Storing pre-trained machine learning models for transfer learning in a wireless communication network

Abstract

Various aspects of the present disclosure relate to a method (1000) performed by a repository entity, comprising: receiving (1002), from a first application entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task; storing (1004) the first information of the base ML model associated with the first ML task; receiving (1006), from a second application entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; identifying (1008) the base ML model as a pre-trained ML model that can be mapped to the second ML task; and sending (1010), to the second application entity, a second information of the base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

Description

STORING PRE-TRAINED MACHINE LEARNING MODELS FOR TRANSFER LEARNING IN A WIRELESS COMMUNICATION NETWORKTECHNICAL FIELD

[0001] The subject matter disclosed herein relates generally to the field of implementing the storing of pre-trained machine learning models for transfer learning in a wireless communication network. This document defines a repository entity for wireless communication and a method performed by a repository entity. This document further defines a first application entity for wireless communication, a second application entity for wireless communication, and methods performed by a first application entity and second application entity.BACKGROUND

[0002] A wireless communications system may include one or multiple network communication devices, such as base stations, which may support wireless communications for one or multiple user communication devices, which may be otherwise known as user equipment (UE), or other suitable terminology. The wireless communications system may support wireless communications with one or multiple user communication devices by utilizing resources of the wireless communication system (e.g., time resources (e.g., symbols, slots, subframes, frames, or the like) or frequency resources (e.g., subcarriers, carriers, or the like). Additionally, the wireless communications system may support wireless communications across various radio access technologies including third generation (3G) radio access technology, fourth generation (4G) radio access technology, fifth generation (5G) radio access technology, among other suitable radio access technologies beyond 5G (e.g., sixth generation (6G)).

[0003] Transfer Learning (TL) is a technique in machine learning where a model trained on one task is used as the starting point for a model on a second task. This can be useful when the second task has similarities with the first task, or when there is limited data available for the second task. By using the learned features from the first task as a startingpoint, the model can learn more quickly and effectively on the second task. This can also help to prevent overfitting, as the model will have already learned general features that are likely to be useful in the second task.SUMMARY

[0004] An article “a” before an element is unrestricted and understood to refer to “at least one” of those elements or “one or more” of those elements. The terms “a,” “at least one,” “one or more,” and “at least one of one or more” may be interchangeable. As used herein, including in the claims, “or” as used in a list of items (e.g., a list of items prefaced by a phrase such as “at least one of’ or “one or more of’ or “one or both of’) indicates an inclusive list such that, for example, a list of at least one of A, B, or C means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Also, as used herein, the phrase “based on” shall not be construed as a reference to a closed set of conditions. For example, an example step that is described as “based on condition A” may be based on both a condition A and a condition B without departing from the scope of the present disclosure. In other words, as used herein, the phrase “based on” shall be construed in the same manner as the phrase “based at least in part on. Further, as used herein, including in the claims, a “set” may include one or more elements.

[0005] There is provided a repository entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the repository entity to: receive, from a first application entity, a first request to store a first information of a base machine learning ‘ML’ model, the base ML model being associated with a first ML task; store the first information of the base ML model associated with the first ML task; receive, from a second application entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; identify the base ML model as a pre-trained ML model that can be mapped to the second ML task; and send, to the second application entity, a second information of the base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

[0006] There is further provided a first application entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least onememory and configured to cause the first application entity to: send, to a repository entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task, wherein the first information comprises an indication that the base ML model can be used as a pre-trained ML model for transfer learning.

[0007] There is further provided a second application entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the second application entity to: send, to a repository entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; receive, from the repository entity, a second information of a base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

[0008] There is further provided a method performed by a repository entity, comprising: receiving, from a first application entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task; storing the first information of the base ML model associated with the first ML task; receiving, from a second application entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; identifying the base ML model as a pre-trained ML model that can be mapped to the second ML task; and sending, to the second application entity, a second information of the base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

[0009] There is further provided a method performed by a first application entity, comprising: sending, to an repository entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task, wherein the first information comprises an indication that the base ML model can be used as a pre-trained ML model for transfer learning.

[0010] There is further provided a method performed by a second application entity, comprising: sending, to a repository entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; and receiving, from the repository entity, a second information of a base ML model indicating the base ML model as a pretrained ML model for transfer learning for the second ML task.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 illustrates an example of a wireless communications system in accordance with aspects of the present disclosure.

[0012] Figure 2 illustrates an example of high-level transfer learning operation in accordance with aspects of the present disclosure.

[0013] Figure 3 illustrates an example of a transfer learning architecture in accordance with aspects of the present disclosure.

[0014] Figure 4 illustrates an example of an AI / ML information storage procedure in accordance with aspects of the present disclosure.

[0015] Figure 5 illustrates a further example of an AI / ML information storage procedure in accordance with aspects of the present disclosure.

[0016] Figure 6 illustrates an example of AI / ML pre-trained model information discovery and an authentication procedure.

[0017] Figure 7 illustrates an example of a user equipment (UE) 700 in accordance with aspects of the present disclosure.

[0018] Figure 8 illustrates an example of a processor 800 in accordance with aspects of the present disclosure.

[0019] Figure 9 illustrates an example of a network equipment (NE) 900 in accordance with aspects of the present disclosure.

[0020] Figure 10 illustrates a flowchart of a method performed by a NE in accordance with aspects of the present disclosure.

[0021] Figure 11 illustrates a flowchart of a method performed by a NE in accordance with aspects of the present disclosure.

[0022] Figure 12 illustrates a flowchart of a method performed by a NE in accordance with aspects of the present disclosure.DETAILED DESCRIPTION

[0023] In the third-generation partnership project (3 GPP) the employment of artificial intelligence (Al) has been discussed in different domains (RAN, core, service enabler layer) either as a tool to optimize existing features and procedures by making them more intelligent, or for assisting Al-enabled apps to communicate via the mobile communications system.

[0024] The work of the 3 GPP in core and application layers has been performed in 3 GPP SA2 which started from Rel-16 to investigate AI / ML support. This focused on two main aspects: 1) AI / ML support in the networks data analytics function (NWDAF); and 2) network assistance for AI / ML services.

[0025] Regarding the first main aspect, network analytics and AI / ML is deployed in the 5G core network via the introducing of NWDAF which considers the support of various analytics types that can be distinguished using different Analytics IDs, e.g., “UE Mobility”, “NF Load”, etc. as elaborated in the 3GPP Technical Specification (TS) 23.288, titled “Architecture enhancements for 5G System to support network data analytics services”. Each NWDAF may support one or more Analytics IDs and may have the role of: (i) AI / ML inference called NWDAF AnLF, or (ii) AI / ML training called NWDAF MTLF or (iii) both. For federated learning (FL) use cases, in 3GPP Release 18, 3GPP defined federated learning amongst different NWDAF MTLFs where ML model training is running in multiple local MTLFs.

[0026] Regarding the second main aspect, as part of AIMLSys, enhancements to 5GC (as specified in the 3GPP TS 23.501 clause 5.46, titled “System Architecture for the 5G system”) are specified for assisting the AI / ML operations in the application layer (between one or more AI / ML users and AI / ML server). In this direction, a network exposure function (NEF) may assist the AI / ML application server in scheduling available UE(s) to participate in the AI / ML operation (e.g. Federated Learning). Also, 5GC may assist the selection of UEs to serve as FL clients, by providing a list of target member UE(s), then subscribing to the NEF to be notified about the subset list of UE(s) (i.e. list of candidate UE(s)) that fulfil certain filtering criteria.

[0027] The 3 GPP SA4 continues to study Al and ML for Media (see 3 GPP TechnicalReport (TR) 26.927 titled, “Study on Artificial Intelligence and Machine Learning in 5G media services”). This study aims to identify relevant interoperability requirements and implementation constraints of AI / ML in 5G media services. This study includes mediabased AI / ML use cases and architecture considerations related to media services.

[0028] The 3 GPP SA5 in Rel-17 has also provided a work on AI / ML management (see3GPP TS 28.105 titled, “Management and orchestration; Artificial Intelligence / Machine Learning management”) focusing on AI / ML capabilities (e.g., ML training MnS / MF) at the operations administration and maintenance (0AM) side. The 3GPP TS 28.105 specifies the AI / ML management capabilities and services for 5GS where AI / ML is used, including management and orchestration (e.g., MDA, see the 3GPP TS 28.104, titled “Management and orchestration; management data analytics (MDA)). In this work, an ML training Function (MLT) playing the role of ML training MnS producer is introduced which may consume various data from 5GS for ML training purpose and provides the training outputs to other management functions in 0 AM.

[0029] The 3 GPP SA6 in Rel-19 continues to study an AIML Enabler (which can be logically placed in AD AES or a new SEAL server) to support AI / ML services via the service enablement layer. This study is discussed in the 3 GPP TR 23.700-82 titled “Study on Application Layer support for AI / ML services

[0030] In 3GPP TR 23.700-82 (AIMLAPP study in Rel-19 of 3GPP SA6), a key issue is presented in respect of support for transfer learning in 3GPP systems. The key issue that arises is how to support transfer learning at application enablement layers.

[0031] In 3GPP TS 22.261 titled “Service requirements for the 5G system” at clause 6.40 it is stated that the 5G system needs to support at least three types of AI / ML operations: AI / ML operation splitting between AI / ML endpoints; Distributed / F ederated Learning over 5G system; and AI / ML model / data distribution and sharing over 5G system.

[0032] In relation to AI / ML model / data distribution and sharing over the 5G system, it is expected to let the AI / ML consumers use an ML model (out-of-a-set of candidate models available in a NW endpoint) for better adapting to task and environment variations.

[0033] Nevertheless, in practical deployments involving AI / ML operations, the number of available models is not that large to cover all the possible environmental conditions in a network. Therefore out-of-the-box usage of the model will imply a lack of accuracy / precision during the inference stage unless a model fine tuning stage is present to ensure that the baseline model is quickly adapted to the current conditions using lesser amount of data (than the one used for training of the baseline model). On that note, Transfer learning (TL) is conceived as a technique used to tackle the lack of labelled data for training a model which is the first limitation that is faced when (re-)training a model. In TL, the training of a model (to solve a particular task) is carried out using information from a "similar" domain in which enough information is available (the so-called source domain). With TL, some parts of the model trained in the source domain are fine-tuned with scarce information available in the target domain. This way, the model reuses the understanding of the structure of the problem and does not have to learn from scratch all the low-level features / structure of the problem: it will only have to learn the higher-level structures / relationships which usually requires less labelled data. TL assumes a baseline model is already available (in a NW endpoint) and can be fine-tuned to quickly perform the same or similar tasks (e.g., prediction, classification, etc.) in a target domain with lesser amount of training data.

[0034] Considering firstly that: in 3GPP TS 23.288 clauses 6.2B.5, 6.2B.6 and 6.2B.7 some operations to allow the storage / retrieval of ML models to / from the Analytical Data Repository Function (ADRF) are available to be consumed by a network function containing an instance of the Model Training Logical Function (MTLF); and as from clause 5.3 in 3GPP TS 23.436 (titled “Functional architecture and information flows for application data analytics enablement service”), the internal architecture of ADAE (Application Data Analytics Enabler) can include an instance of ADRF, the so-called A- ADRF (Application layer - Analytical Data Repository Function), it is possible to enable inter- and intra-VAL domain transfer learning scenarios.

[0035] As mentioned above, Transfer Learning is used to support an entity to ease the training process (if not possible or if the data and time to train is not sufficient). Whenapplying Transfer Learning (TL) to features in 5G and beyond systems, there can be different use cases which can be assumed for the operations to be impacted.

[0036] A first use-case is where TL is employed in AI / ML enabled analytics between network analytics functions in the Core (between NWDAFs / MTLFs).

[0037] A further use-case is where TL is employed in AI / ML enabled analytics between SEAL ADAESs (AD AES as defined in SA6, 3GPP TS 23.436)

[0038] A further use-case is where TL is employed between an AF and a NF, for instance in cross domain analytics of Al-assisted operations.

[0039] A further use-case is where TL is employed to assist AI / ML operations between VAL servers or Afs.

[0040] A further use-case is where TL is employed to assist AI / ML operations between AF / VAL servers and VAL UEs or among VAL UEs.

[0041] For TL enablement in 3 GPP systems focusing more on the application layer, there are certain problems relating to how to store / fetch and distribute pre-trained models if TL is used for application layer ML tasks / operations.

[0042] The disclosure herein mitigates these issues by providing a method at an application repository to store the ML models and that allows external consumers of the pre-trained ML model to discover and fetch pre-trained ML model information. Such solutions provide enhancement to existing application layer registry for ML models. Such solutions enable the benefits of transfer learning as hereinbefore discussed, within 3 GPP systems.

[0043] Aspects of the present disclosure are described in the context of a wireless communications system.

[0044] Figure 1 illustrates an example of a wireless communications system 100 in accordance with aspects of the present disclosure. The wireless communications system 100 may include one or more NE 102, one or more UE 104, and a core network (CN) 106. The wireless communications system 100 may support various radio access technologies. In some implementations, the wireless communications system 100 may be a 4G network,such as an LTE network or an LTE- Advanced (LIE- A) network. In some other implementations, the wireless communications system 100 may be a NR network, such as a 5G network, a 5G- Advanced (5G-A) network, or a 5G ultrawideband (5G-UWB) network. In other implementations, the wireless communications system 100 may be a combination of a 4G network and a 5G network, or other suitable radio access technology including Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20. The wireless communications system 100 may support radio access technologies beyond 5G, for example, 6G. Additionally, the wireless communications system 100 may support technologies, such as time division multiple access (TDMA), frequency division multiple access (FDMA), or code division multiple access (CDMA), etc.

[0045] The one or more NE 102 may be dispersed throughout a geographic region to form the wireless communications system 100. One or more of the NE 102 described herein may be or include or may be referred to as a network node, a base station, a network element, a network function, a network entity, a radio access network (RAN), a NodeB, an eNodeB (eNB), a next-generation NodeB (gNB), or other suitable terminology. An NE 102 and a UE 104 may communicate via a communication link, which may be a wireless or wired connection. For example, an NE 102 and a UE 104 may perform wireless communication (e.g., receive signalling, transmit signalling) over a Uu interface.

[0046] An NE 102 may provide a geographic coverage area for which the NE 102 may support services for one or more UEs 104 within the geographic coverage area. For example, an NE 102 and a UE 104 may support wireless communication of signals related to services (e.g., voice, video, packet data, messaging, broadcast, etc.) according to one or multiple radio access technologies. In some implementations, an NE 102 may be moveable, for example, a satellite associated with a non-terrestrial network (NTN). In some implementations, different geographic coverage areas associated with the same or different radio access technologies may overlap, but the different geographic coverage areas may be associated with different NE 102.

[0047] The one or more UE 104 may be dispersed throughout a geographic region of the wireless communications system 100. A UE 104 may include or may be referred to as aremote unit, a mobile device, a wireless device, a remote device, a subscriber device, a transmitter device, a receiver device, or some other suitable terminology. In some implementations, the UE 104 may be referred to as a unit, a station, a terminal, or a client, among other examples. Additionally, or alternatively, the UE 104 may be referred to as an Internet-of-Things (loT) device, an Internet-of-Everything (loE) device, or machine-type communication (MTC) device, among other examples.

[0048] A UE 104 may be able to support wireless communication directly with other UEs 104 over a communication link. For example, a UE 104 may support wireless communication directly with another UE 104 over a device-to-device (D2D) communication link. In some implementations, such as vehicle-to-vehicle (V2V) deployments, vehicle-to-everything (V2X) deployments, or cellular-V2X deployments, the communication link may be referred to as a sidelink. For example, a UE 104 may support wireless communication directly with another UE 104 over a PC5 interface.

[0049] The NE 102 may comprise application enablement or vertical enablement or edge enablement entities (SEAL, EDGEAPP, CAPIF) which are specified in 3 GPP SA6. An NE 102 may support communications with the CN 106, or with another NE 102, or both. For example, an NE 102 may interface with other NE 102 or the CN 106 through one or more backhaul links (e.g., SI, N2, N2, or network interface). In some implementations, the NE 102 may communicate with each other directly. In some other implementations, the NE 102 may communicate with each other or indirectly (e.g., via the CN 106. In some implementations, one or more NE 102 may include subcomponents, such as an access network entity, which may be an example of an access node controller (ANC). An ANC may communicate with the one or more UEs 104 through one or more other access network transmission entities, which may be referred to as a radio heads, smart radio heads, or transmission-reception points (TRPs).

[0050] The CN 106 may support user authentication, access authorization, tracking, connectivity, and other access, routing, or mobility functions. The CN 106 may be an evolved packet core (EPC), or a 5G core (5GC), which may include a control plane entity that manages access and mobility (e.g., a mobility management entity (MME), an access and mobility management functions (AMF)) and a user plane entity that routes packets orinterconnects to external networks (e.g., a serving gateway (S-GW), a Packet Data Network (PDN) gateway (P-GW), or a user plane function (UPF)). In some implementations, the control plane entity may manage non-access stratum (NAS) functions, such as mobility, authentication, and bearer management (e.g., data bearers, signal bearers, etc.) for the one or more UEs 104 served by the one or more NE 102 associated with the CN 106.

[0051] The CN 106 may communicate with a packet data network over one or more backhaul links (e.g., via an SI, N2, N2, or another network interface). The packet data network may include an application server. In some implementations, one or more UEs 104 may communicate with the application server. A UE 104 may establish a session (e.g., a protocol data unit (PDU) session, or the like) with the CN 106 via an NE 102. The CN 106 may route traffic (e.g., control information, data, and the like) between the UE 104 and the application server using the established session (e.g., the established PDU session). The PDU session may be an example of a logical connection between the UE 104 and the CN 106 (e.g., one or more network functions of the CN 106).

[0052] In the wireless communications system 100, the NEs 102 and the UEs 104 may use resources of the wireless communications system 100 (e.g., time resources (e.g., symbols, slots, subframes, frames, or the like) or frequency resources (e.g., subcarriers, carriers)) to perform various operations (e.g., wireless communications). In some implementations, the NEs 102 and the UEs 104 may support different resource structures. For example, the NEs 102 and the UEs 104 may support different frame structures. In some implementations, such as in 4G, the NEs 102 and the UEs 104 may support a single frame structure. In some other implementations, such as in 5 G and among other suitable radio access technologies, the NEs 102 and the UEs 104 may support various frame structures (i.e., multiple frame structures). The NEs 102 and the UEs 104 may support various frame structures based on one or more numerologies.

[0053] One or more numerologies may be supported in the wireless communications system 100, and a numerology may include a subcarrier spacing and a cyclic prefix. A first numerology (e.g., / r=0) may be associated with a first subcarrier spacing (e.g., 15 kHz) and a normal cyclic prefix. In some implementations, the first numerology (e.g., / r=0) associated with the first subcarrier spacing (e.g., 15 kHz) may utilize one slot per subframe.A second numerology (e.g., / r=l) may be associated with a second subcarrier spacing (e.g., 30 kHz) and a normal cyclic prefix. A third numerology (e.g., / r=2) may be associated with a third subcarrier spacing (e.g., 60 kHz) and a normal cyclic prefix or an extended cyclic prefix. A fourth numerology (e.g., / r=3) may be associated with a fourth subcarrier spacing (e.g., 120 kHz) and a normal cyclic prefix. A fifth numerology (e.g., / r=4) may be associated with a fifth subcarrier spacing (e.g., 240 kHz) and a normal cyclic prefix.

[0054] A time interval of a resource (e.g., a communication resource) may be organized according to frames (also referred to as radio frames). Each frame may have a duration, for example, a 10 millisecond (ms) duration. In some implementations, each frame may include multiple subframes. For example, each frame may include 10 subframes, and each subframe may have a duration, for example, a 1 ms duration. In some implementations, each frame may have the same duration. In some implementations, each subframe of a frame may have the same duration.

[0055] Additionally or alternatively, a time interval of a resource (e.g., a communication resource) may be organized according to slots. For example, a subframe may include a number (e.g., quantity) of slots. The number of slots in each subframe may also depend on the one or more numerologies supported in the wireless communications system 100. For instance, the first, second, third, fourth, and fifth numerologies (i.e., / r=0, jU=l, / r=2, jU=3, / r=4) associated with respective subcarrier spacings of 15 kHz, 30 kHz, 60 kHz, 120 kHz, and 240 kHz may utilize a single slot per subframe, two slots per subframe, four slots per subframe, eight slots per subframe, and 16 slots per subframe, respectively.# Each slot may include a number (e.g., quantity) of symbols (e.g., OFDM symbols). In some implementations, the number (e.g., quantity) of slots for a subframe may depend on a numerology. For a normal cyclic prefix, a slot may include 14 symbols. For an extended cyclic prefix (e.g., applicable for 60 kHz subcarrier spacing), a slot may include 12 symbols. The relationship between the number of symbols per slot, the number of slots per subframe, and the number of slots per frame for a normal cyclic prefix and an extended cyclic prefix may depend on a numerology. It should be understood that reference to a first numerology (e.g., / r=0) associated with a first subcarrier spacing (e.g., 15 kHz) may be used interchangeably between subframes and slots.

[0056] In the wireless communications system 100, an electromagnetic (EM) spectrum may be split, based on frequency or wavelength, into various classes, frequency bands, frequency channels, etc. By way of example, the wireless communications system 100 may support one or multiple operating frequency bands, such as frequency range designations FR1 (410 MHz - 7.125 GHz), FR2 (24.25 GHz - 52.6 GHz), FR3 (7.125 GHz - 24.25 GHz), FR4 (52.6 GHz - 114.25 GHz), FR4a or FR4-1 (52.6 GHz - 71 GHz), and FR5 (114.25 GHz - 300 GHz). In some implementations, the NEs 102 and the UEs 104 may perform wireless communications over one or more of the operating frequency bands. In some implementations, FR1 may be used by the NEs 102 and the UEs 104, among other equipment or devices for cellular communications traffic (e.g., control information, data). In some implementations, FR2 may be used by the NEs 102 and the UEs 104, among other equipment or devices for short-range, high data rate capabilities.

[0057] FR1 may be associated with one or multiple numerologies (e.g., at least three numerologies). For example, FR1 may be associated with a first numerology (e.g., / r=0), which includes 15 kHz subcarrier spacing; a second numerology (e.g., / r=l), which includes 30 kHz subcarrier spacing; and a third numerology (e.g., / r=2), which includes 60 kHz subcarrier spacing. FR2 may be associated with one or multiple numerologies (e.g., at least 2 numerologies). For example, FR2 may be associated with a third numerology (e.g., / r=2), which includes 60 kHz subcarrier spacing; and a fourth numerology (e.g., / r=3), which includes 120 kHz subcarrier spacing.

[0058] Figure 2 illustrates an example of high-level transfer learning operation 200 in accordance with aspects of the present disclosure.

[0059] Transfer learning starts with a model 220 that has previously been trained for a certain task 210 using a large set of data. Frequently trained on extensive datasets, this model 220 has identified general features and patterns relevant to numerous related jobs. This model 220 is known as a pre-trained model or pre-trained ML model.

[0060] The model 220 that has been pre-trained is also known as the base model. It is made up of layers that have utilized the incoming data to learn hierarchical feature representations. This base model forms the starting point for re-training for a second ML task 250.

[0061] In the pre-trained model 220, a set of layers is found that capture generic information and knowledge 230 relevant to the new task 250 as well as the previous task 210. Because they are prone to learning low-level information, these layers are frequently found near the top of the trained ML network.

[0062] Using the dataset from the new task / challenge 250 to retrain the chosen layers is a procedure known as fine-tuning. The goal is to preserve the knowledge from the pretraining while enabling the new model 240 to modify its parameters to better suit the demands of the current assignment.

[0063] Figure 3 illustrates an example of a transfer learning architecture 300 in accordance with aspects of the present disclosure. The transfer learning architecture 300 includes a source 310 of a base model which may be a ML server or a VAL server enabler. Also shown is a transfer learning registry / management entity 320. Also shown is an enabler server 330 which may be an AI / ML enabler or an AD AES. Further shown is a 5GC 340, a RAN 350 and a UE1 360 and UE2 370. The UE1 360 comprises an enabler application client 362. The UE2 370 comprises an enabler application client 372. The UE1 360 has an associated ML enhanced application task#l 364. The UE2 370 has an associated ML enhanced application task #2 374. The UEs 360, 370 interface with RAN 350 over Uu. The UEs 360, 370 interface with each other over PC5. A procedure for storing and retrieving pre-trained ML models for transfer learning will now be generically discussed with respect to the architecture 300 of Figure 3.

[0064] In a first step 301, ML model provider 310 (e.g., VAL server) or the MTME function (in Al enabler 330) requests A-ADRF 320 to register and store a pre-trained ML model for a particular analytics ID # or for a certain application service / service profile or UE / group of UEs; or for an ML Task Job #. This request includes the permissions and the possible use for Transfer Learning by another VAL server or ADAES / AI Enabler. For completeness, the Al Enabler 330 may be equivalent to an AI / ML Enablement Server as reported in 3GPP TR 23.700-82, or any other SEAL or edge enablers as defined in 3GPP TS 23.434 and 3 GPP TS 23.558 which is enhanced to provide support for AI / ML enablement capabilities. Such an Al Enabler 330 may include or can be represented as an AF functionality. AF is defined in 3GPP TS 23.501 as an application function which is partof 5G system (part of the service-based 5G core architecture); however such an AF can be either deployed at the MNO domain or at a trusted or non-trusted 3rd party (e.g., vertical) which has established agreement with the MNO for consuming core network services.

[0065] In certain embodiments, the requestor (310 or MTME function in Al enabler 330) provides a set of context information (i.e., features, dataset requirement, environment, profile, usage / intent) which is matched to the pre-trained ML model.

[0066] The request may also include also the pre-trained ML model to be stored or this can be provided in separate step, or this can be given in form of an address / URL to allow the a consumer of the model to direct download.

[0067] The request may be in the form of a subscription to the A-ADRF 320

[0068] In a further step 302, the A-ADRF 320 authorizes the request and responds to the requestor (ML model provider 310 or Al Enabler 330) with ACK / NACK. The A- ADRF 320 stores the pre-trained ML model as a base ML model.

[0069] In a further step 303, the AD AES or Al Enabler (with MTME functionality) 330 identifies that for a certain ML model task (364 or 374 via enabler client 362 or 372) it requires TL to be used and generates a trigger to discover available pre-trained models.

[0070] The ML model task 364 or 374 can be related to an Analytics ID from AD AES 330. The ML model task can be related to the training by Al Enabler (i.e., 330 and enabler clients 362, 372) functionality. The ML model task 364, 374 can be related to a task initiated by a second VAL server.

[0071] In a further step 304, the AD AES or Al enabler 330 requests the available pretrained models in the repository 320 for a given Analytics ID or for a given Model ID or profile or for a set of features which may be similar. The “repository” may also be referred to as a “model repository” herein. The “model” may also be referred to as a “ML model” herein.

[0072] If the requestor is the VAL server, VAL server may use Al enabler 330 to discover and fetch the pre-trained model or this can be done directly to the repository 320.

[0073] In a further step 305, the repository 320 (A-ADRF) sends to the consumer of the pre-trained model (AD AES or Al enabler 330) one or a list of pre-trained model IDs / addresses / context information which can be mapped to the given ML model task.

[0074] In a further step 306, the consumer (AD AES or Al enabler 330) selects one of the models and may further request to fetch the pre-trained model from A-ADRF 320 or may request the address / URL if not given in this step.

[0075] In a further step 307, the consumer 330 uses the pretrained model to train / retrain the ML model for the target ML model task or analytics ID.

[0076] In a further step 308, the consumer 330 after updating the model, may also send the updated ML model (if allowed by the ML model provider 310) to the A-ADRF 320.

[0077] Figures 4-5 illustrate examples of an AI / ML information storage procedures 400, 500 in accordance with aspects of the present disclosure. The procedures 400, 500 shows a model repository consumer 410, 510 and a model repository 420, 520. The model provider is the model repository consumer 410, 510 and stores to the model repository 420, 520 (A-ADRF or an external registry e.g., CCF, edge repository).

[0078] Two aspects of functionality will be described a first aspect of functionality is the storage of a pre-trained ML model which in certain implementation can enhance the Solution #5 of 3GPP TR 23.700-82 (clause 8.5.2). A second aspect of functionality is enabling the utilization of the pre-trained model by an Al Enabler entity.

[0079] With regard to the example 400 of Figure 4, in a first step 401 the model repository consumer 410 (e.g., ADAE or AIML server with MTME function) sends an AI / ML model information storage request to the model repository 420 to store a pre-trained ML model. The pre-trained ML model included in the request can be one trained by the repository consumer 410 or its information can be provisioned to the repository consumer 410. The request contains information of the AI / ML model (or a model profile) such as the analytics ID(s) for which the model can be used, requirements of using the model (e.g., required computing power), information of the training data with which the model is trained, metadata, etc. In case of TL, the consumer 410 provides the following additional parameters: indication or flag that the model can be used as candidate pre-trained model;context information for supporting the pre-trained model consumer (set of features, data set requirements); permissions and exposure level for the pre-trained model. This may provide a list of entities or services / service profiles or area and time where this can be fetched and used; permissions for CRUD operations by the consumer of the pre-trained model.

[0080] In a further step 402, upon receiving the request, the model repository 420 processes the request and records information of the AI / ML model (e.g., by creating an AI / ML model profile). The model repository 420 sends a response to the model repository consumer 410 (e.g., MTME-enhanced ADAE or AIML server) with an identifier of the created AI / ML model profile. The response may be referred to as an ‘AI / ML model information storage response’.

[0081] In case of TL, the model repository 420 may provide in the response the acknowledgement or result for storing the model as pre-trained model. This may indicate that more information is required for classifying the model as pre-trained (additional context information, environment). Such request for supplementary information may lead to further interaction with the repository 420.

[0082] With regard to the example 500 of Figure 5, the model repository consumer 510 doesn’t actively indicate that the model can be used for TL but the repository 520 requests if it can be used in this way. In a first step 501, the model repository consumer 510 (e.g., ADAE or AIML server with MTME function) sends an AI / ML model information storage request to the model repository 520 to store a model. Following the request, it receives a response from the repository 520.

[0083] In a further step 502, the model repository 520 queries the consumer 510 whether the model can be used for TL, and requests supplementary information to help classify it and store it in a manner that can be fetched by TL consumers. This query can be in form of a request and may include the following parameters: a indication or flag that the model can be used as candidate pre-trained model; context information for supporting the pre-trained model consumer (set of features, data set requirements); permissions and exposure level for the pre-trained model; a list of entities or services / service profiles or area and time, vendors; where the model can be fetched and used; permissions for CRUDoperations by the consumer of the pre-trained model; charging model in case of TL enabled.

[0084] In a further step 503, upon receiving the query, the model repository consumer 510 authorizes the request and sends the requested information. This information may include: whether the model can be used by other entities as pre-trained; context information and permissions; charging model; or any other requested parameter requested in step 502.

[0085] In a further step 504, if additional / supplementary information is provided, the repository 520 updates the information for the stored model and adds a flag that it can be used for TL.

[0086] Figure 6 illustrates an example 600 of AI / ML pre- trained model information discovery and an authentication procedure. In this example 600, following the storage of the pre-trained ML model, an AD AES / AIML Enablement Server discovers from the repository the pre-trained ML models. In certain implementations this may be used as an enhancement of Sol#5 in clause 8.5.3 of 3GPP TR 23.700-82 for supporting TL.

[0087] The example 600 of Figure 6 shows a model repository consumer (i.e., the pretrained ML model provider) 610, a model repository 620 and a consumer of a transfer learning service (i.e., of pre-trained ML models) 630.

[0088] In a first step 601, the expected or prospective consumer 630 of the TL service / pre-trained ML model (e.g., AD AES, AIML Enablement Server, VAL server) identifies that for a certain task there is a need for TL and discovers the repository 620 to request for it.

[0089] In a further step 602, the consumer 630 of the TL service / pre-trained model sends an AI / ML pre-trained model information discovery request to the model repository 620. The request contains filter information of the AI / ML model such as the analytics ID(s) associated with the model, requirements of the model (e.g., required computing power), information of the training data. For TL, this request may also contain a request for a list of pre-trained models which can be used for a certain analytics ID or task ID or for certain features / context information which may be used to find similarities with the pre-trainedmodel. This request may provide some intent / goal for the task and other information such as the UE / group of UEs, vertical to be used, area and time of interest.

[0090] In a further step 603, upon receiving the discovery request, the model repository 620 authorizes the request and may interact with the pre-trained model providers / owners 610 to confirm that the pre-trained ML model can be exposed to be used for TL. If this is already provided at the storage of the pre-trained ML model (i.e., in the example procedures 400, 500), such confirmation is optional.

[0091] In a further step 604, the repository 620 then sends the response message to the consumer of the TL service 630. The response includes information of the discovered AI / ML pre-trained model(s). Optionally, the model repository 620 can initiate a model transfer process to the consumer of the TL service 630.

[0092] Figure 7 illustrates an example of a UE 700 in accordance with aspects of the present disclosure. The UE 700 may include a processor 702, a memory 704, a controller 706, and a transceiver 708. The processor 702, the memory 704, the controller 706, or the transceiver 708, or various combinations thereof or various components thereof may be examples of means for performing various aspects of the present disclosure as described herein. These components may be coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces.

[0093] The processor 702, the memory 704, the controller 706, or the transceiver 708, or various combinations or components thereof may be implemented in hardware (e.g., circuitry). The hardware may include a processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other programmable logic device, or any combination thereof configured as or otherwise supporting a means for performing the functions described in the present disclosure.

[0094] The processor 702 may include an intelligent hardware device (e.g., a general- purpose processor, a DSP, a CPU, an ASIC, an FPGA, or any combination thereof). In some implementations, the processor 702 may be configured to operate the memory 704. In some other implementations, the memory 704 may be integrated into the processor 702.The processor 702 may be configured to execute computer-readable instructions stored in the memory 704 to cause the UE 700 to perform various functions of the present disclosure.

[0095] The memory 704 may include volatile or non-volatile memory. The memory 704 may store computer-readable, computer-executable code including instructions when executed by the processor 702 cause the UE 700 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium such the memory 704 or another type of memory. Computer-readable media includes both non- transitory computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A non-transitory storage medium may be any available medium that may be accessed by a general-purpose or special-purpose computer.

[0096] In some implementations, the processor 702 and the memory 704 coupled with the processor 702 may be configured to cause the UE 700 to perform one or more of the functions described herein (e.g., executing, by the processor 702, instructions stored in the memory 704). For example, the processor 702 may support wireless communication at the UE 700 in accordance with examples as disclosed herein. The UE 700 may be configured to support a means for aspects of the methods disclosed herein.

[0097] The controller 706 may manage input and output signals for the UE 700. The controller 706 may also manage peripherals not integrated into the UE 700. In some implementations, the controller 706 may utilize an operating system such as iOS®, ANDROID®, WINDOWS®, or other operating systems. In some implementations, the controller 706 may be implemented as part of the processor 702.

[0098] In some implementations, the UE 700 may include at least one transceiver 708. In some other implementations, the UE 700 may have more than one transceiver 708. The transceiver 708 may represent a wireless transceiver. The transceiver 708 may include one or more receiver chains 710, one or more transmitter chains 712, or a combination thereof.

[0099] A receiver chain 710 may be configured to receive signals (e.g., control information, data, packets) over a wireless medium. For example, the receiver chain 710 may include one or more antennas for receive the signal over the air or wireless medium.The receiver chain 710 may include at least one amplifier (e.g., a low- noise amplifier (LN A)) configured to amplify the received signal. The receiver chain 710 may include at least one demodulator configured to demodulate the receive signal and obtain the transmitted data by reversing the modulation technique applied during transmission of the signal. The receiver chain 710 may include at least one decoder for decoding the processing the demodulated signal to receive the transmitted data.

[0100] A transmitter chain 712 may be configured to generate and transmit signals (e.g., control information, data, packets). The transmitter chain 712 may include at least one modulator for modulating data onto a carrier signal, preparing the signal for transmission over a wireless medium. The at least one modulator may be configured to support one or more techniques such as amplitude modulation (AM), frequency modulation (FM), or digital modulation schemes like phase-shift keying (PSK) or quadrature amplitude modulation (QAM). The transmitter chain 712 may also include at least one power amplifier configured to amplify the modulated signal to an appropriate power level suitable for transmission over the wireless medium. The transmitter chain 712 may also include one or more antennas for transmitting the amplified signal into the air or wireless medium.

[0101] Figure 8 illustrates an example of a processor 800 in accordance with aspects of the present disclosure. The processor 800 may be an example of a processor configured to perform various operations in accordance with examples as described herein. The processor 800 may include a controller 802 configured to perform various operations in accordance with examples as described herein. The processor 800 may optionally include at least one memory 804, which may be, for example, an L1 / L2 / L3 cache. Additionally, or alternatively, the processor 800 may optionally include one or more arithmetic-logic units (ALUs) 806. One or more of these components may be in electronic communication or otherwise coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces (e.g., buses).

[0102] The processor 800 may be a processor chipset and include a protocol stack (e.g., a software stack) executed by the processor chipset to perform various operations (e.g., receiving, obtaining, retrieving, transmitting, outputting, forwarding, storing, determining, identifying, accessing, writing, reading) in accordance with examples as described herein.The processor chipset may include one or more cores, one or more caches (e.g., memory local to or included in the processor chipset (e.g., the processor 800) or other memory (e.g., random access memory (RAM), read-only memory (ROM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), static RAM (SRAM), ferroelectric RAM (FeRAM), magnetic RAM (MRAM), resistive RAM (RRAM), flash memory, phase change memory (PCM), and others).

[0103] The controller 802 may be configured to manage and coordinate various operations (e.g., signalling, receiving, obtaining, retrieving, transmitting, outputting, forwarding, storing, determining, identifying, accessing, writing, reading) of the processor 800 to cause the processor 800 to support various operations in accordance with examples as described herein. For example, the controller 802 may operate as a control unit of the processor 800, generating control signals that manage the operation of various components of the processor 800. These control signals include enabling or disabling functional units, selecting data paths, initiating memory access, and coordinating timing of operations.

[0104] The controller 802 may be configured to fetch (e.g., obtain, retrieve, receive) instructions from the memory 804 and determine subsequent instruction(s) to be executed to cause the processor 800 to support various operations in accordance with examples as described herein. The controller 802 may be configured to track memory address of instructions associated with the memory 804. The controller 802 may be configured to decode instructions to determine the operation to be performed and the operands involved. For example, the controller 802 may be configured to interpret the instruction and determine control signals to be output to other components of the processor 800 to cause the processor 800 to support various operations in accordance with examples as described herein. Additionally, or alternatively, the controller 802 may be configured to manage flow of data within the processor 800. The controller 802 may be configured to control transfer of data between registers, arithmetic logic units (ALUs), and other functional units of the processor 800.

[0105] The memory 804 may include one or more caches (e.g., memory local to or included in the processor 800 or other memory, such RAM, ROM, DRAM, SDRAM, SRAM, MRAM, flash memory, etc. In some implementations, the memory 804 may residewithin or on a processor chipset (e.g., local to the processor 800). In some other implementations, the memory 804 may reside external to the processor chipset (e.g., remote to the processor 800).

[0106] The memory 804 may store computer-readable, computer-executable code including instructions that, when executed by the processor 800, cause the processor 800 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium such as system memory or another type of memory. The controller 802 and / or the processor 800 may be configured to execute computer- readable instructions stored in the memory 804 to cause the processor 800 to perform various functions. For example, the processor 800 and / or the controller 802 may be coupled with or to the memory 804, the processor 800, the controller 802, and the memory 804 may be configured to perform various functions described herein. In some examples, the processor 800 may include multiple processors and the memory 804 may include multiple memories. One or more of the multiple processors may be coupled with one or more of the multiple memories, which may, individually or collectively, be configured to perform various functions herein.

[0107] The one or more ALUs 806 may be configured to support various operations in accordance with examples as described herein. In some implementations, the one or more ALUs 806 may reside within or on a processor chipset (e.g., the processor 800). In some other implementations, the one or more ALUs 806 may reside external to the processor chipset (e.g., the processor 800). One or more ALUs 806 may perform one or more computations such as addition, subtraction, multiplication, and division on data. For example, one or more ALUs 806 may receive input operands and an operation code, which determines an operation to be executed. One or more ALUs 806 be configured with a variety of logical and arithmetic circuits, including adders, subtractors, shifters, and logic gates, to process and manipulate the data according to the operation. Additionally, or alternatively, the one or more ALUs 806 may support logical operations such as AND, OR, exclusive-OR (XOR), not-OR (NOR), and not- AND (NAND), enabling the one or more ALUs 806 to handle conditional operations, comparisons, and bitwise operations.

[0108] The processor 800 may support wireless communication in accordance with examples as disclosed herein. The processor 800 may be configured to or operable to support a means for aspects of the methods disclosure herein.

[0109] Figure 9 illustrates an example of a NE 900 in accordance with aspects of the present disclosure. The NE 900 may comprise application enablement or vertical enablement or edge enablement entities (SEAL, EDGEAPP, CAPIF) which are specified in 3GPP SA6. The NE 900 may include a processor 902, a memory 904, a controller 906, and a transceiver 908. The processor 902, the memory 904, the controller 906, or the transceiver 908, or various combinations thereof or various components thereof may be examples of means for performing various aspects of the present disclosure as described herein. These components may be coupled (e.g., operatively, communicatively, functionally, electronically, electrically) via one or more interfaces.

[0110] The processor 902, the memory 904, the controller 906, or the transceiver 908, or various combinations or components thereof may be implemented in hardware (e.g., circuitry). The hardware may include a processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other programmable logic device, or any combination thereof configured as or otherwise supporting a means for performing the functions described in the present disclosure.

[0111] The processor 902 may include an intelligent hardware device (e.g., a general- purpose processor, a DSP, a CPU, an ASIC, an FPGA, or any combination thereof). In some implementations, the processor 902 may be configured to operate the memory 904. In some other implementations, the memory 904 may be integrated into the processor 902. The processor 902 may be configured to execute computer-readable instructions stored in the memory 904 to cause the NE 900 to perform various functions of the present disclosure.

[0112] The memory 904 may include volatile or non-volatile memory. The memory 904 may store computer-readable, computer-executable code including instructions when executed by the processor 902 cause the NE 900 to perform various functions described herein. The code may be stored in a non-transitory computer-readable medium such the memory 904 or another type of memory. Computer-readable media includes both non- transitory computer storage media and communication media including any medium thatfacilitates transfer of a computer program from one place to another. A non-transitory storage medium may be any available medium that may be accessed by a general-purpose or special-purpose computer.

[0113] In some implementations, the processor 902 and the memory 904 coupled with the processor 902 may be configured to cause the NE 900 to perform one or more of the functions described herein (e.g., executing, by the processor 902, instructions stored in the memory 904). For example, the processor 902 may support wireless communication at the NE 900 in accordance with examples as disclosed herein. The NE 900 may be configured to support a means for performing aspects of the methods disclosed herein. The NE 900 may be a NE 310, 320, 330 of Figure 3, or an NE 410, 420 of Figure 4, or an NE 510, 520 of Figure 5, or an NE 610, 620, 630 of Figure 6.

[0114] The controller 906 may manage input and output signals for the NE 900. The controller 906 may also manage peripherals not integrated into the NE 900. In some implementations, the controller 906 may utilize an operating system such as iOS®, ANDROID®, WINDOWS®, or other operating systems. In some implementations, the controller 906 may be implemented as part of the processor 902.

[0115] In some implementations, the NE 900 may include at least one transceiver 908. In some other implementations, the NE 900 may have more than one transceiver 908. The transceiver 908 may represent a wireless transceiver. The transceiver 908 may include one or more receiver chains 910, one or more transmitter chains 912, or a combination thereof.

[0116] A receiver chain 910 may be configured to receive signals (e.g., control information, data, packets) over a wireless medium. For example, the receiver chain 910 may include one or more antennas for receive the signal over the air or wireless medium. The receiver chain 910 may include at least one amplifier (e.g., a low- noise amplifier (LN A)) configured to amplify the received signal. The receiver chain 910 may include at least one demodulator configured to demodulate the receive signal and obtain the transmitted data by reversing the modulation technique applied during transmission of the signal. The receiver chain 910 may include at least one decoder for decoding the processing the demodulated signal to receive the transmitted data.

[0117] A transmiter chain 912 may be configured to generate and transmit signals (e.g., control information, data, packets). The transmitter chain 912 may include at least one modulator for modulating data onto a carrier signal, preparing the signal for transmission over a wireless medium. The at least one modulator may be configured to support one or more techniques such as amplitude modulation (AM), frequency modulation (FM), or digital modulation schemes like phase-shift keying (PSK) or quadrature amplitude modulation (QAM). The transmitter chain 912 may also include at least one power amplifier configured to amplify the modulated signal to an appropriate power level suitable for transmission over the wireless medium. The transmiter chain 912 may also include one or more antennas for transmiting the amplified signal into the air or wireless medium.

[0118] Figure 10 illustrates a flowchart of a method in accordance with aspects of the present disclosure. The operations of the method may be implemented by a NE as described herein. In some implementations, the NE may execute a set of instructions to control the function elements of the NE to perform the described functions.

[0119] At 1002, the method may include receiving, from a first application entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task. The operations of 1002 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1002 may be performed by a NE as described with reference to Figure 9.

[0120] At 1004, the method may include storing the first information of the base ML model associated with the first ML task. The operations of 1004 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1004 may be performed by a NE as described with reference to Figure 9.

[0121] At 1006, the method may include receiving, from a second application entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task. The operations of 1006 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1006 may be performed a NE as described with reference to Figure 9.

[0122] At 1008, the method may include identifying the base ML model as a pretrained ML model that can be mapped to the second ML task. The operations of 1008 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1008 may be performed by a NE as described with reference to Figure 9.

[0123] At 1010, the method may include sending, to the second application entity, a second information of the base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task. The operations of 1010 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1010 may be performed by a NE as described with reference to Figure 9.

[0124] Figure 11 illustrates a flowchart of a method in accordance with aspects of the present disclosure. The operations of the method may be implemented by a NE as described herein. In some implementations, the NE may execute a set of instructions to control the function elements of the NE to perform the described functions.

[0125] At 1102, the method may include sending, to a repository entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task, wherein the first information comprises an indication that the base ML model can be used as a pre-trained ML model for transfer learning. The operations of 1102 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1102 may be performed by a NE as described with reference to Figure 9.

[0126] Figure 12 illustrates a flowchart of a method in accordance with aspects of the present disclosure. The operations of the method may be implemented by a NE as described herein. In some implementations, the NE may execute a set of instructions to control the function elements of the NE to perform the described functions.

[0127] At 1202, the method may include sending, to a repository entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task. The operations of 1202 may be performed in accordance with examples as describedherein. In some implementations, aspects of the operations of 1202 may be performed by a NE as described with reference to Figure 9.

[0128] At 1204, the method may include receiving, from the repository entity, a second information of a base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task. The operations of 1204 may be performed in accordance with examples as described herein. In some implementations, aspects of the operations of 1204 may be performed by a NE as described with reference to Figure 9.

[0129] It should be noted that the methods described herein describe possible implementations, and that the operations and the steps may be rearranged or otherwise modified and that other implementations are possible.

[0130] The disclosure herein provides a repository entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the repository entity to: receive, from a first application entity, a first request to store a first information of a base machine learning ‘ML’ model, the base ML model being associated with a first ML task; store the first information of the base ML model associated with the first ML task; receive, from a second application entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; identify the base ML model as a pre-trained ML model that can be mapped to the second ML task; and send, to the second application entity, a second information of the base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

[0131] Transfer learning is a technique in ML where a model trained on a first ML task is used as the starting point for a model on a second ML task. This can be useful when the second ML task has similarities with the first ML task and / or when there is limited data available for training an entirely new ML model for the second ML task. By using the learned features from the first ML task as a starting point, a ML model can learn more quickly and effectively on the second ML task. This can also help to prevent overfitting, as the ML model will have already learned general features that are likely to be useful in the second ML task.

[0132] A pre-trained ML model is a ML model that has previously been trained for a certain ML task. The pre-trained ML model may have been trained on extensive training datasets. The pre-trained ML model may have identified general features and patterns relevant to numerous related ML tasks. A base ML model or base model is the pre-trained model. It may be made up of layers that have utilized the training data from a first ML task to learn hierarchical feature representations. In transfer learning, a set of layers of the base model can be found that capture generic information related to a second ML task. Using available training data for the second ML task, the chosen layers of the base model can be retrained - a process known as ‘fine tuning’. This preserves the knowledge from the pretraining (for the first ML task) while enabling the ML model to be modified to better suit the demands of the second ML task.

[0133] In the 3GPP Technical Specification 22.261, titled “Service Requirements for the 5G System”, it is stated that the 5G system needs to support AI / ML model / data distribution and sharing. There is no current mechanism to store / fetch and distribute pretrained ML models for application layer ML tasks.

[0134] The disclosure herein provides a solution at a repository entity (i.e., an application-layer repository entity) wherein pre-trained ML models or information thereof can be stored. External consumers (the second application entity directly, or another entity acting via the second application entity) of the pre-trained ML models can discover and fetch the pre-trained model information by interacting with the repository entity. The repository entity identifies whether a base ML model can be mapped to a second ML task. This may involve a mapping of one or more requirements provided to the registry entity (in the second request) to the first information of the base ML model. If this is the case, the base ML model or information thereof can be provided as the ‘second information’ provided to the consumer entity (the second application entity) indicating the base ML model as a pre-trained ML model suitable for transfer learning for the second ML task.

[0135] Such a solution provides enhancement of existing application layer registries for ML models. Furthermore, such a solution enables the benefits of transfer learning within 3 GPP systems.

[0136] The base ML model may alternatively be referred to herein as a ‘first’ ML model. The repository entity may be referred to herein as an ‘application-layer repository entity’. The second information may comprise at least part of the first information.

[0137] The repository entity may be part of a wireless communication system (i.e., may reside at the application layer of a wireless communication system).

[0138] The first application entity may operate as an ML model provider. Examples of a first application entity include an ADAE or AIML server, or an MTME function. The base ML model may be an ML model trained by the first application entity or it may be an ML model trained by another entity that has provided the first information to the first application entity.

[0139] The second application entity may operate as a consumer of a transfer learning service / a consumer of pre-trained models. Examples of a second application entity include AD AES, AIML server, or a VAL server.

[0140] The at least one processor may be configured to cause the repository entity to: determine an indication that the base ML model can be used as a pre-trained ML model for transfer learning.

[0141] The at least one processor may be configured to cause the repository entity to determine the indication by causing the repository entity to: determine that the first information comprises the indication that the base ML model can be used as a pre-trained ML model for transfer learning.

[0142] The first application entity may provide directly to the repository entity an indication that the base ML model can be used for transfer learning. This indication is provided in the first information.

[0143] The first information may further comprise at least one of: a context information for supporting the use of the base ML model for transfer learning; the base ML model or an address where the base ML model can be accessed; one or more permissions for use and / or exposure of the base ML model; one or more permissions for create read update delete ‘CRUD’ operations; and a charging model.

[0144] The context information may include data set requirements, environmental requirements, intended usage of the base ML model, for instance.

[0145] The one or more permissions for use and / or exposure may include permitted entities, services, service profiles, areas, times, to which the base ML model can be exposed and used.

[0146] The at least one processor may be configured to cause the repository entity to determine the indication by causing the repository entity to: send, to the first application entity, a third request querying whether the base ML model can be used as a pre-trained ML model for transfer learning; and receive, from the first application entity, a third response comprising supplementary information indicating the base ML model can be used as a pre-trained ML model for transfer learning.

[0147] The first information may not include a flag or other indication that directly indicates that the base ML model can be used for transfer learning. The repository entity may query the first application entity, to ascertain whether permission / authorization is given for the base ML model to be exposed for transfer learning to the second application entity. The ‘third request’ may be referred to herein as a ‘query’.

[0148] The supplementary information may comprise at least one of: the indication the base ML model can be used as a pre-trained ML model for transfer learning; a context information for supporting the use of the base ML model for transfer learning; the base ML model or an address where the base ML model can be accessed; one or more permissions for use and / or exposure of the base ML model; one or more permissions for create read update delete ‘CRUD’ operations; and a charging model.

[0149] The charging model may be used to provide awareness of how the user of transfer-learning / pre- trained models will be charged (i.e., because this is a ‘service’ being provided by the entity that owns the model). The charging model may be based on one or more of: the duration and / or data used; criticality of the operation; which CRUD operations are used; the service level agreement between the MNO and the vertical / application provider; the energy consumption requirement for pre-training the model.

[0150] The at least one processor may be configured to cause the repository entity to store the first information by causing the repository entity to: update the first information, based on the supplementary information; and store the updated first information.

[0151] The repository entity when updating the first information may introduce a flag or indication that the base ML model can be used for transfer learning.

[0152] The first information may comprise at least one of: an identifier of an application service associated with the base ML model; a service profile associated with the base ML model; an identifier for one or more user equipment associated with the base ML model; an identifier for the first ML task; a model profile for the base ML model; information of the training data used to train the base ML model for the first ML task; an identifier for analytics tasks with which the base ML model can be used.

[0153] The second request may comprise one or more requirements for pre-trained ML models for transfer learning for the second ML task, wherein the one or more requirements comprise one or more of: an identifier for an analytics task; an identifier for an ML model; a model profile for an ML model; one or more attributes of an ML model; and one or more training data requirements.

[0154] The one or more requirements may be considered as filtering information that can be used by the repository entity to identify whether the base ML model maps to the second ML task. For instance, the repository entity may only identify base ML models that map to the identifier for the analytics task specified in the second request.

[0155] The first and / or second ML tasks may be analytics tasks.

[0156] Transfer learning can be employed for one or more AI / ML enabled analytics tasks. This includes analytics tasks between network analytics functions in the core network, analytics tasks between SEAL ADAESs; operations between an AF and NF, such as in cross-domain analytics; operations between VAL servers or AFs; operations between AF / VAL servers and VAL UEs or among VAL UEs. The network analytics functions may comprise any combination of NWDAFs and / or MTLFs.

[0157] The first and / or second ML tasks may be application-layer ML tasks.

[0158] The disclosure herein provides a mechanism to store / fetch and distribute pretrained models when transfer learning is used for application layer ML tasks. Currently, there is no such mechanism available for the application layer.

[0159] The at least one processor may be further configured to cause the repository entity, upon storing the first information, to: send, to the first application entity, a first response confirming the storing of the first information.

[0160] The first response may comprise a first identifier for the first information stored by the repository entity.

[0161] The first identifier may be an identifier of an AI / ML model profile created by the repository entity. The first response may contain an acknowledgement or result for storing the base ML model as a pre-trained model. This may indicate that more information is required for classifying the model as pre-trained (additional context information, environmental information, etc.). This requirement for additional information may lead to further interactions between the first application entity and the repository entity.

[0162] The at least one processor may be further configured to cause the repository entity to authorize the second request by causing the repository entity to: send a fourth request, to the first application entity, for confirmation that the base ML model can be exposed to the second application entity as a pre-trained ML model for transfer learning.

[0163] Alternatively, the authorization may have already been given (for instance in the first information).

[0164] The at least one processor may be further configured to cause the repository entity to: initiate a transfer of the base ML model to the second application entity.

[0165] A transfer of the base ML model may include the repository entity providing the base ML model to the second application entity, or may include the repository entity initiating a transfer from the first application entity to the second application entity, for instance.

[0166] The at least one processor may be further configured to cause the repository entity to: receive, from the second application entity, an updated base ML model, whereinthe updated base ML model comprises the base ML model retrained for the second ML task; and store, the updated base ML model.

[0167] The first application entity or a consumer acting therethrough, may retrain the base ML model for the second ML task. After updating the base ML model, if allowed by the ML model provider, the updated base ML model may be provided back to the repository entity.

[0168] The first application entity or the second application entity may comprise: a vertical application layer ‘VAL’ server; a model training and management entity ‘MTME’; an application entity; an application data analytics enabler server ‘AD AES’; or an artificial intelligence machine learning ‘AIML’ server.

[0169] The repository entity may comprise: an application layer repository entity; an application layer analytics data repository function ‘AADRF’; a common API framework core function ‘CCF’; or an edge repository.

[0170] The disclosure herein further provides a first application entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the first application entity to: send, to a repository entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task, wherein the first information comprises an indication that the base ML model can be used as a pre-trained ML model for transfer learning.

[0171] The disclosure herein further provides a second application entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the second application entity to: send, to a repository entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; receive, from the repository entity, a second information of a base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

[0172] The disclosure herein further provides a method performed by a repository entity, comprising: receiving, from a first application entity, a first request to store a firstinformation of a base ML model, the base ML model being associated with a first ML task; storing the first information of the base ML model associated with the first ML task; receiving, from a second application entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; identifying the base ML model as a pre-trained ML model that can be mapped to the second ML task; and sending, to the second application entity, a second information of the base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

[0173] The method may further comprise determining an indication that the base ML model can be used as a pre-trained ML model for transfer learning.

[0174] The determining the indication may comprise determining that the first information comprises the indication that the base ML model can be used as a pre-trained ML model for transfer learning.

[0175] The first information may further comprise at least one of: a context information for supporting the use of the base ML model for transfer learning; the base ML model or an address where the base ML model can be accessed; one or more permissions for use and / or exposure of the base ML model; one or more permissions for create read update delete ‘CRUD’ operations; and a charging model.

[0176] The determining the indication may comprise: sending, to the first application entity, a third request querying whether the base ML model can be used as a pre-trained ML model for transfer learning; and receiving, from the first application entity, a third response comprising supplementary information indicating the base ML model can be used as a pre-trained ML model for transfer learning.

[0177] The supplementary information may comprise at least one of: the indication the base ML model can be used as a pre-trained ML model for transfer learning; a context information for supporting the use of the base ML model for transfer learning; the base ML model or an address where the base ML model can be accessed; one or more permissions for use and / or exposure of the base ML model; one or more permissions for create read update delete ‘CRUD’ operations; and a charging model.

[0178] The storing the first information may comprise: updating the first information, based on the supplementary information; and storing the updated first information.

[0179] The first information may comprise at least one of: an identifier of an application service associated with the base ML model; a service profile associated with the base ML model; an identifier for one or more user equipment associated with the base ML model; an identifier for the first ML task; a model profile for the base ML model; information of the training data used to train the base ML model for the first ML task; an identifier for analytics with which the base ML model can be used.

[0180] The second request may comprise one or more requirements for pre-trained ML models for transfer learning for the second ML task, wherein the one or more requirements comprise one or more of: an identifier for an analytics task; an identifier for an ML model; a model profile for an ML model; one or more attributes of an ML model; and one or more training data requirements.

[0181] The first and / or second ML tasks may be analytics tasks.

[0182] The first and / or second ML tasks may be application-layer ML tasks.

[0183] The method may comprise, upon the storing the first information: sending, to the first application entity, a first response confirming the storing of the first information.

[0184] The first response may comprise a first identifier for the first information stored by the repository entity.

[0185] The method may comprise authorizing the second request by sending a fourth request, to the first application entity, for confirmation that the base ML model can be exposed to the second application entity as a pre-trained ML model for transfer learning.

[0186] The method may comprise initiating a transfer of the base ML model to the second application entity.

[0187] The method may comprise receiving, from the second application entity, an updated base ML model, wherein the updated base ML model comprises the base ML model retrained for the second ML task; and storing, the updated base ML model.

[0188] The first application entity and / or the second application entity may comprise: a vertical application layer ‘VAL’ server; a model training and management entity ‘MTME’; an application entity; an application data analytics enabler server ‘AD AES’; or an artificial intelligence machine learning ‘AIML’ server.

[0189] The repository entity may comprise: an application layer repository entity; an application layer analytics data repository function ‘AADRF’; a common API framework core function ‘CCF’; or an edge repository.

[0190] There is further provided a method performed by a first application entity, comprising: sending, to an repository entity, a first request to store a first information of a first machine learning ‘ML’ model, the base ML model being associated with a first ML task, wherein the first information comprises an indication that the base ML model can be used as a pre-trained ML model for transfer learning.

[0191] The disclosure herein further provides a method performed by a second application entity, comprising: sending, to a repository entity, a second request for one or more pre-trained machine learning ‘ML’ models for transfer learning for a second ML task; and receiving, from the repository entity, a second information of a base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

[0192] Transfer Learning (TL) is used to support an entity to ease the training process (if conventional training is not possible or if the data and time to train is not sufficient). In 3GPP SA6 Al-enabled functionalities there is no mechanism to store / fetch and distribute pre-trained models if TL is used for application layer ML tasks / operations. Hence, the disclosure herein focuses to on this issue.

[0193] The solutions provided herein, provide a method at an application repository to store the ML models and allows external consumers of the pre-trained ML model to discover and fetch pre-trained ML model information. Such solutions provide enhancement to existing application layer registry for ML models defined for 3 GPP systems.

[0194] Current solutions do not consider transfer learning techniques (i.e., using a pretrained model from one task to another), hence the storage / fetching for supporting other consumers is a novel procedure which enhances existing capabilities and interfaces.

[0195] There is provided, a method (i.e., at an application layer repository) for storing and maintaining a pre-trained ML model for one or more ML model tasks, the method comprises: receiving an application requirement from an application entity to store a base ML model for a first ML model task; storing the base ML model for the first ML model task; receiving a request from a further application entity to provide information on one or more pre-trained ML models for a second ML model task; identifying the base ML model for the first ML model task as one of requested pre-trained ML models for the second ML model task; mapping the stored base ML model as a candidate pre-trained ML model for the second ML model task; and sending to the further application entity information of the base ML model as one of the pre-trained ML models for the second ML model task

[0196] The first and / or second ML model task may be an analytics task identified by an analytics identity.

[0197] The requirements may comprise one or more requirements as hereinbefore discussed with respect to the examples disclosed herein.

[0198] The first and / or second ML model task may be application layer ML model tasks.

[0199] The method may further comprise sending a notification to the application entity of the base ML model upon storing the base ML model.

[0200] The method may further comprise sending a query for supplementary information with respect to the base ML model; and receiving supplementary information based on the request.

[0201] The supplementary information may comprise the supplementary information as hereinbefore discussed with respect to the examples disclosed herein.

[0202] The method may further comprise updating the stored information of the base ML model based on the supplementary information.

[0203] The request from the further application entity may comprise a request as hereinbefore discussed with respect to the examples disclosed herein.

[0204] The method may further comprise authorizing the request from the further application entity.

[0205] The authorizing the request may comprise obtaining the authorization from the application entity.

[0206] It should be noted that the method described herein describes a possible implementation, and that the operations and the steps may be rearranged or otherwise modified and that other implementations are possible.

[0207] The description herein is provided to enable a person having ordinary skill in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to a person having ordinary skill in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the disclosure is not limited to the examples and designs described herein but is to be accorded the broadest scope consistent with the principles and novel features disclosed herein.

[0208] As used herein, an “ML model” may comprise a mathematical algorithm that can be "trained" by data and human expert input as examples to replicate a decision an expert would make when provided that same information (see 3GPP TS 28.105). As used herein, “ML model training” may include capabilities of an ML training function or service to take data, run it through an ML model, derive the associated loss and adjust the parameterization of that ML model based on the computed loss (see 3GPP TS 28.105). As used herein, “ML model inference may include capabilities of an ML model inference function that employs an ML model and / or Al decision entity to conduct inference (see 3GPP TS 28.105). As used herein “AI / ML enablement” may comprise an application enablement framework consisting of one or more AI / ML enabler capabilities based on the SA6 provider implementation. Such function can be deployed as (or within) an enablement layer server (e.g. SEAL or AD AES) or client. As used herein the term “repository” may be used interchangeably with the term “model repository”. The term “model” may be used interchangeably with the term “ML model”.

[0209] The following abbreviations are used throughout the disclosure: TL, Transfer Learning; MDA, Management Domain Analytics; AIML , Artificial Intelligence / Machine Learning; AD AES , Application Data Analytics Enabler Server; SEAL , Service Enabler Architecture Layer; NW , Network, ; MTME , Model Training and Management Entity; CCF, CAPIF (Common API Framework) Core Function; CRUD , Create-Read-Update- Delete; and A-ADRF, Application layer - Analytics Data Repository Function.

Claims

CLAIMS1. A repository entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the repository entity to: receive, from a first application entity, a first request to store a first information of a base machine learning ‘ML’ model, the base ML model being associated with a first ML task; store the first information of the base ML model associated with the first ML task; receive, from a second application entity, a second request for one or more pretrained ML models for transfer learning for a second ML task; identify the base ML model as a pre-trained ML model that can be mapped to the second ML task; and send, to the second application entity, a second information of the base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

2. The repository entity of claim 1 , wherein the at least one processor is configured to cause the repository entity to: determine an indication that the base ML model can be used as a pre-trained ML model for transfer learning.

3. The repository entity of claim 2, wherein the at least one processor is configured to cause the repository entity to determine the indication by causing the repository entity to: determine that the first information comprises the indication that the base ML model can be used as a pre-trained ML model for transfer learning.

4. The repository entity of claim 3, wherein the first information further comprises at least one of:a context information for supporting the use of the base ML model for transfer learning; the base ML model or an address where the base ML model can be accessed; one or more permissions for use and / or exposure of the base ML model; one or more permissions for create read update delete ‘CRUD’ operations; and a charging model.

5. The repository entity of claim 2, wherein the at least one processor is configured to cause the repository entity to determine the indication by causing the repository entity to: send, to the first application entity, a third request querying whether the base ML model can be used as a pre-trained ML model for transfer learning; and receive, from the first application entity, a third response comprising supplementary information indicating the base ML model can be used as a pre-trained ML model for transfer learning.

6. The repository entity of claim 5, wherein the supplementary information comprises at least one of: the indication the base ML model can be used as a pre-trained ML model for transfer learning; a context information for supporting the use of the base ML model for transfer learning; the base ML model or an address where the base ML model can be accessed; one or more permissions for use and / or exposure of the base ML model; one or more permissions for create read update delete ‘CRUD’ operations; and a charging model.

7. The repository entity of any one of claims 5-6, wherein the at least one processor is configured to cause the repository entity to store the first information by causing the repository entity to: update the first information, based on the supplementary information; and store the updated first information.

8. The repository entity of any preceding claim, wherein the first information comprises at least one of: an identifier of an application service associated with the base ML model; a service profile associated with the base ML model; an identifier for one or more user equipment associated with the base ML model; an identifier for the first ML task; a model profile for the base ML model; information of the training data used to train the base ML model for the first ML task; an identifier for analytics tasks with which the base ML model can be used.

9. The repository entity of any preceding claim, wherein the second request comprises one or more requirements for pre-trained ML models for transfer learning for the second ML task, wherein the one or more requirements comprise one or more of: an identifier for an analytics task; an identifier for an ML model; a model profile for an ML model; one or more attributes of an ML model; and one or more training data requirements.

10. The repository entity of any preceding claim, wherein the first and / or second ML tasks are analytics tasks.

11. The repository entity of claim 10, wherein the first and / or second ML tasks are application-layer ML tasks.

12. The repository entity of any preceding claim, wherein the at least one processor is further configured to cause the repository entity, upon storing the first information, to: send, to the first application entity, a first response confirming the storing of the first information.

13. The repository entity of claim 12, wherein the first response comprises a first identifier for the first information stored by the repository entity.

14. The repository entity of any preceding claim, wherein the at least one processor is further configured to cause the repository entity to authorize the second request by causing the repository entity to: send a fourth request, to the first application entity, for confirmation that the base ML model can be exposed to the second application entity as a pre-trained ML model for transfer learning.

15. The repository entity of any preceding claim, wherein the at least one processor is further configured to cause the repository entity to: initiate a transfer of the base ML model to the second application entity.

16. The repository entity of any preceding claim, wherein the at least one processor is further configured to cause the repository entity to: receive, from the second application entity, an updated base ML model, wherein the updated base ML model comprises the base ML model retrained for the second ML task; and store, the updated base ML model.

17. The repository entity of any preceding claim, wherein the first application entity or the second application entity comprise: a vertical application layer ‘ VAL’ server; a model training and management entity ‘MTME’; an application entity; an application data analytics enabler server ‘ADAES’; or an artificial intelligence machine learning ‘AIML’ server; optionally wherein the repository entity comprises: an application layer repository entity;an application layer analytics data repository function ‘AADRF’; a common API framework core function ‘CCF’; or an edge repository.

18. A first application entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the first application entity to: send, to a repository entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task, wherein the first information comprises an indication that the base ML model can be used as a pre-trained ML model for transfer learning.

19. A second application entity for wireless communication, comprising: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the second application entity to: send, to a repository entity, a second request for one or more pre-trained ML models for transfer learning for a second ML task; receive, from the repository entity, a second information of a base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

20. A method performed by a repository entity, comprising: receiving, from a first application entity, a first request to store a first information of a base ML model, the base ML model being associated with a first ML task; storing the first information of the base ML model associated with the first ML task; receiving, from a second application entity, a second request for one or more pretrained ML models for transfer learning for a second ML task; identifying the base ML model as a pre-trained ML model that can be mapped to the second ML task; andsending, to the second application entity, a second information of the base ML model indicating the base ML model as a pre-trained ML model for transfer learning for the second ML task.

Images