Managing Machine Learning Models in 5G Core Networks

Abstract

An embodiment includes a method for a first network node or function (NNF) configured for machine learning (ML) model management in a communication network. The method includes receiving a first message from a second NNF of the communication network, the first message including one or more ML model identifiers corresponding to one or more ML models maintained by the first node and an identifier of an analysis based on the ML models. The method includes sending a second message to the second NNF including one of a plurality of tuples corresponding to the plurality of ML models on which the analysis is based, each tuple including a different ML model identifier and an information element associated with the corresponding ML model, or a single tuple including an analysis identifier and an information element associated with a single ML model on which the analysis is based. Other embodiments include a complementary method for the second NNF.

Description

[Technical Field]

[0001] The present application relates generally to the field of communication networks, and more particularly to techniques for generating analytical results in communication networks based on machine learning (ML) models, including those used in 5G Core (5GC) networks. [Background technology]

[0002] At a high level, a 5G system (5GS) consists of an access network (AN) and a core network (CN). The AN provides access to the CN via base stations, e.g., gNB or ng-eNB, as described below. Ko Connectivity To UE The CN includes various network functions (NFs) that provide a wide range of different functions, such as session management, connection management, accounting, and authentication.

[0003] FIG. 1 shows a high-level view of an exemplary 5G network architecture consisting of a Next Generation Radio Access Network (NG-RAN) 199 and a 5G Core (5GC) 198. The NG-RAN 199 may include one or more gNodeBs (gNBs) connected to the 5GC via one or more NG interfaces, such as gNBs 100, 150 connected via interfaces 102, 152, respectively. More specifically, the gNBs 100, 150 may be connected to one or more Access and Mobility Management Functions (AMFs) in the 5GC 198 via their respective NG-C interfaces. Similarly, the gNBs 100, 150 may be connected to one or more User Plane Functions (UPFs) in the 5GC 198 via their respective NG-U interfaces. As described in more detail below, various other Network Functions (NFs) may be included in the 5GC 198.

[0004] Additionally, the multiple gNBs may be connected to each other via one or more Xn interfaces, such as Xn interface 140 between gNB 100 and gNB 150. The radio technology for NG-RAN is often referred to as “New Radio” (NR). With respect to the NR interface to the UE, each of the gNBs may support frequency division duplex (FDD), time division duplex (TDD), or a combination thereof. Each of the multiple gNBs may provide service in a geographic coverage area including one or more cells and, in some cases, may use different directional beams to provide coverage in each cell.

[0005] The NG-RAN 199 is layered into a Radio Network Layer (RNL) and a Transport Network Layer (TNL). The NG-RAN architecture, i.e., the NG-RAN logical nodes and the interfaces between them, are specified as part of the RNL. For each NG-RAN interface (NG, Xn, F1), the associated TNL protocols and functions are specified. The TNL provides services for user plane transport and signaling transport. In some example configurations, each gNB is connected to all 5GC nodes within an "AMF domain," where the term "AMF" refers to the Access and Mobility Management Function in 5GC.

[0006] The NG RAN logical node shown in FIG. 1 includes a central unit (CU or gNB-CU) and one or more distributed units (DU or gNB-DU). For example, gNB 100 includes gNB-CU 110 and gNB-DUs 120 and 130. The CU (e.g., gNB-CU 110) is a logical node that hosts upper layer protocols and performs various gNB functions, such as controlling the operation of the DUs. The DUs (e.g., gNB-DUs 120, 130) are distributed logical nodes that host lower layer protocols and may include various subsets of gNB functions depending on the functional partitioning option. Thus, each of the CU and DU may include various circuits necessary to perform its functions, including processing circuits, transceiver circuits (e.g., for communication), and power circuits.

[0007] A gNB-CU connects to one or more gNB-DUs via respective F1 logical interfaces, such as interfaces 122 and 132 shown in Figure 1, but a gNB-DU can only connect to a single gNB-CU. The gNB-CU and connected gNB-DU can only appear as a gNB to other gNBs and 5GCs; that is, the F1 interface beyond the gNB-CU is not recognized (is invisible).

[0008] Another change in 5G networks (e.g., in 5GC) is that the traditional peer-to-peer interfaces and protocols found in previous generation networks are modified and / or replaced by a service-based architecture (SBA) in which a network function (NF) provides one or more services to one or more service consumers. This can be done, for example, through HTTP / REST (Hypertext Transfer Protocol / Representational State Transfer) application programming interfaces (APIs). Generally, various services are self-contained functions that can be changed and modified in an isolated manner without affecting other services.

[0009] Furthermore, a service is composed of various "service operations", which are finer divisions of the overall service functionality. The type of interaction between service consumers and producers can be "request / response" or "subscribe / notify". In 5G SBA, a Network Repository Function (NRF) enables every network function to discover services provided by other network functions, and a Data Storage Function (DSF) enables every network function to remember its context. This 5G SBA model is based on principles including modularity, reusability, and self-sufficiency of NFs, which means that network deployment can be achieved with the latest virtualization and software technologies. of Interest Score points This makes it possible to

[0010] A 5GC NF of particular interest in this disclosure is the Network Data Analysis Function (NWDAF), which provides network analysis information (e.g., statistics and / or forecast information of past events) to other NFs.

[0011] Machine learning (ML) is a type of artificial intelligence (AI) that focuses on using data and algorithms to mimic the way humans learn, gradually improving its accuracy. ML algorithms build models based on example (or "training") data, which are then used to make predictions or decisions. ML algorithms can be used in a wide variety of applications (e.g., medicine, email filtering, speech recognition, etc.) where it is difficult or infeasible to develop traditional algorithms to perform the required tasks.

[0012] According to 3GPP TS 23.288 (v17.2.0), the NWDAF is the main network function for computing analytics reports and specifies the classification of the NWDAF into two subfunctions (or logical functions): the NWDAF Analysis Logical Function (NWDAFAnLF), which executes the analytics procedures, and the NWDAF Model Training Logical Function (NWDAFMTLF), which trains and retrains the ML models used by the NWDAFAnLF. The services used by these NWDAF logical functions operate under the implicit assumption of a one-to-one relationship between ML models and analytics identifiers (IDs), also known as "NwdafEvents." In this context, Analytics IDs are used to identify the types of analytics the NWDAF can generate or that a consumer of analytics results is interested in receiving. Summary of the Invention

[0013] However, in certain scenarios, multiple ML models may be used to generate the analytics reports provided to a particular Analytics ID. For example, different ML models may be used to generate these analytics reports depending on location, time of day, network load, or any other information filtering parameter. As another example, these analytics reports may be generated by several cascaded ML models. These ambiguities may cause various problems, challenges, and / or difficulties in managing ML models and the analytics they generate.

[0014] Embodiments of the present disclosure address these and other issues, challenges, and / or difficulties, thereby facilitating other advantageous deployments of ML models for analysis in 5G networks.

[0015] Some embodiments include an example method (e.g., procedure) for a first network node or function (NNF) configured for ML model management in a communications network (e.g., 5GC).

[0016] These example methods may include receiving, from a second NNF of the communication network, a first message including one of the following: ● one or more ML model identifiers corresponding to one or more ML models maintained by the first node; or If no ML model identifier is included, an identifier for the analysis based on one or more ML models.

[0017] These example methods may also include sending a second message to the second NNF that includes one of the following: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of a plurality of ML model identifiers and one or more information elements associated with the corresponding ML model, or ● A single tuple containing an analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

[0018] According to some embodiments, for each tuple included in the second message, the one or more information elements include a network address that is one of a Universal Resource Locator (URL) or a Fully Qualified Domain Name (FQDN). According to some embodiments, each network address included in the second message is one of an address that allows for retrieving the corresponding ML model, or an address that allows for retrieving a manifest or metadata file associated with the corresponding ML model. According to some of these embodiments, the manifest or metadata file associated with each of the ML models includes: ML model identifier Identifiers for one or more analyses based on ML models An updated version of the ML model, where "updated" refers to the latest version of the ML model that has been trained and validated by the MTLF and can therefore be used for inference by the AnLF. Creation timestamp of the updated version of the ML model. A network address that allows retrieving the ML model.

[0019] According to some embodiments, for each tuple included in the second message, the one or more information elements also include an updated version of the corresponding ML model. According to some embodiments, if the first message includes an analysis identifier, these example methods may also include using the (received) analysis identifier to determine one or more ML model identifiers corresponding to one or more ML models on which the analysis is based.

[0020] According to other embodiments, if the first message includes one or more of the ML model identifiers, the first message also indicates a version associated with the corresponding ML model. According to some of these embodiments, sending the second message includes one of the following: selectively sending a second message including a single tuple based on the updated version of the single ML model being newer than the version indicated by the first message; or Selectively including each of the plurality of tuples in a second message based on the updated version of the corresponding ML model being newer than the version indicated by the first message.

[0021] According to some embodiments, the first NNF is an MTLF of the NWDAF and the second NNF is an AnLF of the NWDAF. According to some of these embodiments, the first message is an Nnwdaf_MLModelInfo_Request message and the second message is a response to the Nnwdaf_MLModelInfo_Request message.

[0022] According to other embodiments, the first message is a Nnwdaf_MLModelProvision_Subscribe message and the second message is a Nnwdaf_MLModelProvision_Notify message. According to some of these embodiments, the first message also includes one or more conditions that must be met in order to receive the second message, including one or more of the following: The performance metrics (or identifiers) of one or more ML models on which the analysis is based - Thresholds for each performance metric • The logical relationship between performance metrics and thresholds, and ●ML / AI framework constraints for running one or more ML models.

[0023] According to some variations, these example methods may also include determining whether to retrain one or more ML models based on whether current values ​​of the performance metrics of each model satisfy respective thresholds and logical relationships (e.g., as indicated in the first message). According to some variations, sending the second message includes one of the following: selectively sending a second message that includes a single tuple based on a single ML model that satisfies one or more conditions included in the first message; or • Selectively including each of the plurality of tuples in the second message based on a corresponding ML model that satisfies one or more conditions included in the first message.

[0024] According to some further variations, each tuple included in the second message also includes an identifier of a performance metric indicated in the first message and a value of the identified performance metric for the corresponding ML model.

[0025] Other embodiments include a method (e.g., procedure) for a second NNF configured for ML model management in a communications network (e.g., 5GC).

[0026] These example methods may include transmitting, to a first NNF of a communication network, a first message including one of the following: ● one or more ML model identifiers corresponding to one or more ML models maintained by the first node; or If no ML model identifier is included, an identifier for one or more ML model-based analyses.

[0027] These example methods may also include receiving, from the first node, a second message including one of the following: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; or ● A single tuple containing an analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

[0028] According to some embodiments, for each tuple included in the second message, the one or more information elements include a network address that is a URL or an FQDN. According to some embodiments, each network address included in the second message is one of an address that allows obtaining a corresponding ML model or an address that allows obtaining a manifest or metadata file associated with the corresponding ML model.

[0029] According to some of these embodiments, the manifest or metadata file associated with each of the ML models includes: ML model identifier Identifiers for one or more analyses based on ML models ● Updated version of the ML model Creation timestamp of the updated version of the ML model. A network address that allows retrieving the ML model.

[0030] According to other of these embodiments, these example methods may also include the following operations: obtaining a single ML model or multiple ML models based on the network address in the corresponding single tuple or multiple tuples (e.g., included in the second message); and applying the obtained single ML model or multiple ML models to determine an analysis identified by the analysis identifier.

[0031] According to some embodiments, for each tuple included in the second message, one or more information elements also include an updated version of the corresponding ML model. According to some embodiments, if the first message includes one or more of the ML model identifiers, the first message also indicates the associated version of the corresponding ML model. In some of these embodiments, one or more of the following apply: The second message containing a single tuple is received only if the updated version of the single ML model is newer than the version indicated by the first message. Each of the multiple tuples is received in the second message only if the updated version of the corresponding ML model is newer than the version indicated by the first message.

[0032] According to some embodiments, the first NNF is an MTLF of the NWDAF and the second NNF is an AnLF of the NWDAF. According to some of these embodiments, the first message is an Nnwdaf_MLModelInfo_Request message and the second message is a response to the Nnwdaf_MLModelInfo_Request message.

[0033] According to some of these embodiments, the first message is a Nnwdaf_MLModel-Provision_Subscribe message and the second message is a Nnwdaf_MLModelProvision_Notify message. According to some of these embodiments, the first message also includes one or more conditions for receiving the second message, including one or more of the following: The performance metrics (or identifiers) of one or more ML models on which the analysis is based - Thresholds for each performance metric • The logical relationship between performance metrics and thresholds, and ●ML / AI framework constraints for running one or more ML models.

[0034] According to some variants, one or more of the following apply: A second message containing a single tuple is received only if a single ML model satisfies one or more conditions contained in the first message. Each of the plurality of tuples is received in the second message only if the corresponding ML model satisfies one or more conditions included in the first message.

[0035] According to some further variations, or in each tuple included in the second message, the one or more information elements also include an identifier of the performance metric indicated in the first message and a value of the identified performance metric for the corresponding ML model.

[0036] According to some embodiments, the example methods may also include using the analysis identifier to determine one or more ML model identifiers corresponding to one or more ML models on which the analysis is based. In such cases, the first message includes the determined one or more ML model identifiers.

[0037] Other embodiments include first and second NNFs (e.g., an MTLF and an AnLF, or network nodes hosting them) configured to perform operations corresponding to any of the example methods described herein. Other embodiments also include a non-transitory computer-readable medium storing computer-executable instructions that, when executed by processing circuitry associated with such NNFs, configure the NNFs to perform operations corresponding to any of the example methods described herein.

[0038] These and other disclosed embodiments may enable an MTLF to identify the ML model it has retrained when notifying the AnLF, thereby enabling the AnLF to swap the particular ML model it is using for inference. As another example, embodiments may facilitate an AnLF to express conditions under which the MTLF should retrain an ML model. As another example, embodiments may facilitate an MTLF to express training performance results so that the MTLF can decide whether to swap ML models. At a high level, embodiments improve the management of ML models used for analysis in communications networks (e.g., 5GC).

[0039] These and other objects, features, and advantages of the present disclosure will become apparent from a reading of the following detailed description in light of the drawings briefly described below. [Brief explanation of the drawings]

[0040] [Figure 1] illustrates various aspects of an exemplary 5G network architecture. [Figure 2] illustrates various aspects of an exemplary 5G network architecture.

[0041] [Figure 3]1 illustrates an exemplary procedure for an NWDAF service consumer to subscribe to notifications about the availability of ML models from the NWDAF (MTLF).

[0042] [Figure 4] 1 illustrates an example procedure for an NWDAF service consumer to obtain information about an ML model from the NWDAF (MTLF).

[0043] [Figure 5] shows various example scenarios where a single Analytics ID can be processed by multiple ML models. [Figure 6] shows various example scenarios where a single Analytics ID can be processed by multiple ML models. [Figure 7] shows various example scenarios where a single Analytics ID can be processed by multiple ML models.

[0044] [Figure 8] 1 shows a block diagram of an NWDAF according to various embodiments of the present disclosure.

[0045] [Figure 9] 1 illustrates a data structure of an exemplary Nnwdaf_MLModelProvision_Subscribe request message, according to various embodiments of the present disclosure.

[0046] [Figure 10] 1 illustrates a data structure for an exemplary Nnwdaf_MLModelProvision_Notify request message, according to various embodiments of the present disclosure.

[0047] [Figure 11] 1 illustrates a data structure of an exemplary Nnwdaf_MLModelInfo_Request request message, according to various embodiments of the present disclosure.

[0048] [Figure 12] 1 illustrates a data structure of an exemplary Nnwdaf_MLModelInfo_Request response message, according to various embodiments of the present disclosure.

[0049] [Figure 13] 1 illustrates an example method (e.g., procedure) for a first node of a communication network, according to various embodiments of the present disclosure.

[0050] [Figure 14] 1 illustrates an example method (e.g., procedure) for a second node of a communication network, according to various embodiments of the present disclosure.

[0051] [Figure 15] 1 illustrates a communication system according to various embodiments of the present disclosure.

[0052] [Figure 16] 1 illustrates a UE according to various embodiments of the present disclosure.

[0053] [Figure 17] 1 illustrates a network node according to various embodiments of the present disclosure.

[0054] [Figure 18] 1 illustrates a host computing system according to various embodiments of the present disclosure.

[0055] [Figure 19] 1 is a block diagram of a virtualization environment in which functionality implemented by some embodiments of the present disclosure may be virtualized.

[0056] [Figure 20] 1 illustrates communication between a host computing system, a network node, and a terminal over multiple connections, according to various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0057] The above-described embodiments will be described in more detail below with reference to the accompanying drawings. These descriptions are provided as examples to explain the subject matter to those skilled in the art and should not be construed as limiting the scope of the subject matter to only the embodiments described herein. More specifically, examples illustrating the operation of various embodiments according to the above-described advantages are provided below.

[0058] In general, all terms used herein should be interpreted according to their ordinary meaning in the relevant technical field unless a different meaning is clearly given and / or is implied from the context in which it is used. All references to a / an / the+ element, apparatus, component, means, step, etc. should be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless otherwise specified. The steps of any method and / or procedure disclosed herein do not have to be performed in the order disclosed, unless a step is expressly described as following or preceding another step and / or it is implied that a step must follow or precede another step. Any feature of any embodiment disclosed herein can be applied to any other embodiment, as appropriate. Similarly, the advantages of any embodiment can be applied to other embodiments, and vice versa. Other objects, features, and advantages of the disclosed embodiments will become apparent from the following description.

[0059] Furthermore, the following terminology is used in the following description: Core network node: As used herein, a "core network node" is any type of node in a core network. Some examples of a core network node include, for example, a Mobility Management Entity (MME), a Serving Gateway (SGW), a Packet Data Network Gateway (P-GW), etc. A core network node may also be a node that implements a specific Core Network Function (NF), such as an Access and Mobility Management Function (AMF), a Session Management Function (SMF), a User Plane Function (UPF), a Service Capability Exposure Function (SCEF), etc. Network node: As used herein, a "network node" is any node that is part of a core network (e.g., the core network node described above) of a telecommunications network. Functionally, a network node is equipment that is capable, configured, arranged, and / or operable to communicate, directly or indirectly, with wireless or wired devices and / or with other network nodes or equipment in the telecommunications network, enable and / or provide wireless or wired access to telecommunications devices, and / or perform other functions (e.g., management) in the telecommunications network. ● Node: As used herein, the term “node” (without any prefix) may be any type of node capable of operating in or with a telecommunications network (including a RAN and / or core network), including a radio access node (or equivalent term), a core network node, or a telecommunications device. ● Service: As used herein, the term "service" generally refers to a set of data associated with one or more applications to be transferred over a network, with specific delivery requirements that must be met for the application to be successful. Component: As used herein, the term "component" generally refers to any element required for the delivery of a service. Examples of components are the RAN (e.g., E-UTRAN, NG-RAN, or parts thereof such as eNB, gNB, base station (BS)), the CN (e.g., EPC, 5GC, or parts thereof including all types of links between the RAN and CN entities), and a cloud infrastructure with associated resources such as computing, storage, etc. Generally, each component may have a "manager," which is an entity that can collect historical information about resource utilization and provide information about the current and predicted future availability of resources associated with that component (e.g., the RAN manager).

[0060] It should be noted that the description provided herein focuses on 3GPP® telecommunications systems, and therefore 3GPP® terminology or terminology similar to 3GPP® terminology is generally used. However, the concepts disclosed herein are not limited to 3GPP® systems. Other wireless systems, including, but not limited to, Wideband Code Division Multiple Access (WCDMA®), Worldwide Interoperability for Microwave Access (WiMax), Ultra Mobile Broadband for Mobile Communications (UMB), and Global System for Mobile Communications (GSM), may also benefit from the concepts, principles, and / or embodiments described herein.

[0061] In addition, functions and / or operations described herein as being performed by a telecommunications device or network node may be distributed across multiple telecommunications devices and / or network nodes.

[0062] 2 illustrates an example non-roaming 5G reference architecture with service-based interfaces and various NFs defined by 3GPP in the control plane (CP), including the following NFs, the most relevant to this disclosure of which will be described in more detail: ● The Application Function (AF, with Naf interface) interacts with the 5GC to provide information to the network operator and subscribe to specific events occurring in the operator's network. The AF provides applications where services are delivered at a layer (i.e., transport layer) different from the layer where the service is requested (i.e., signaling layer), and the control of flow resources is performed according to what has been negotiated with the network. The AF communicates dynamic session information to the PCF (via the N5 interface), including a description of the media delivered by the transport layer. The Policy Control Function (PCF, with Npcf interface) supports a unified policy framework that governs network behavior by providing PCC rules (e.g., regarding the treatment of each service data flow under PCC control) to the SMF via the N7 reference point. The PCF provides policy control decisions and flow-based charging control, including service data flow discovery, gate control, QoS control, and flow-based charging (excluding credit management), to the SMF. The PCF receives session and media-related information from the AF and notifies the AF of traffic (or user) plane events. The User Plane Function (UPF) supports the processing of user plane traffic based on rules received from the SMF, including packet inspection and various enforcement actions such as event detection and reporting. The UPF communicates with the RAN (e.g., NG-RAN) via the N3 reference point, with the SMF (described below) via the N4 reference point, and with the external packet data network (PDN) via the N6 reference point. The N9 reference point is for communication between two UPFs. The Session Management Function (SMF, with its Nsmf interface) interacts with the separated traffic (or user) plane, including creating, updating, and removing Protocol Data Unit (PDU) sessions and managing session context with the User Plane Function (UPF), for example, for event reporting. For example, the SMF performs data flow detection (based on filter definitions contained in PCC rules), online and offline charging interactions, and policy enforcement. The Charging Function (CHF, with Nchf interface) is responsible for centralized online and offline charging functions. It provides quota management (for online charging), re-authorization triggers, rating conditions, etc. and is informed about usage reports from the SMF. Quota management involves allowing a certain number of units (e.g., bytes, seconds) for a service. The CHF also interacts with the billing system. The Access and Mobility Management Function (AMF, with Namf interface) terminates the RAN CP interface and handles all mobility and connection management for the UE (similar to MME in EPC). The AMF communicates with the UE via the N1 reference point and with the RAN (e.g., NG-RAN) via the N2 reference point. Network Exposure Function (NEF) with Nnef interface - acts as an entry point into the operator's network by securely exposing network capabilities and events offered by 3GPP® NFs to the AF and by providing a way for the AF to securely provide information to the 3GPP® network. For example, the NEF provides services that allow the AF to provision specific subscription data for various UEs (e.g., expected UE behavior). Network Repository Function (NRF) with Nnrf interface - provides service registration and discovery, allowing NFs to identify suitable services available from other NFs. Network Slice Selection Function (NSSF) with Nnssf interface - A "network slice" is a logical partition of a 5G network that provides specific network capabilities and characteristics, for example, in support of a specific service. A network slice instance is a set of NF instances and the necessary network resources (e.g., compute, storage, communication) that provide the capabilities and characteristics of the network slice. The NSSF enables other NFs (e.g., AMF) to identify the appropriate network slice instance for the service desired by the UE. Authentication server with Nausf interface machine AUSF - User of Based on the home network (HPLMN), the user recognition It performs authentication and computes security keying material for a variety of purposes. • The Network Data Analysis Facility (NWDAF) with Nnwdaf interface is described in more detail above and below. A Location Management Function (LMF) with an Nlmf interface supports various functions related to determining the UE location, including determining the location of the UE and obtaining either DL location measurements or location estimates from the UE, UL location measurements from the NG RAN, and non-UE related assistance data from the NG RAN.

[0063] The Unified Data Management (UDM) function supports the generation of 3GPP authentication credentials, user identification processing, access authorization based on subscription data, and other subscriber-related functions. To provide this functionality, the UDM uses subscription data (including authentication data) stored in the 5GC Unified Data Repository (UDR). In addition to the UDM, the UDR supports the storage and retrieval of policy data by the PCF and application data by the NEF.

[0064] The NRF allows all NFs to discover services provided by other NFs, and the Data Storage Function (DSF) allows all NFs to store their context. Furthermore, the NEF exposes 5GC capabilities and events to AFs both within and outside the 5GC. For example, the NEF provides a service that allows AFs to provision specific subscription data (e.g., expected UE behavior) for various UEs.

[0065] The communication link between the UE and the 5G network (AN and CN) can be grouped into two different hierarchies. The UE communicates with the CN via the Non-Access Stratum (NAS) and with the AN via the Access Stratum (AS). All NAS communication takes place between the UE and the AMF via the NAS protocol (N1 interface in Figure 2). Security for communication over this layer is provided by the NAS protocol (for the NAS) and the PDCP protocol (for the AS).

[0066] 3GPP® Release 17 enhances SBA by adding a data management framework including a Data Collection and Coordination Function (DCCF) and a Messaging Framework Adapter Function (MFAF), which are detailed in 3GPP® TR 23700-91 (v17.0.0). The data management framework is backward compatible with the Release 16 NWDAF functions described above. For Release 17, the baseline services provided by the DCCF (e.g., to the NWDAF analysis function) are the Release 16 NF services used to obtain data. For example, the baseline DCCF service used by an NWDAF consumer to obtain UE mobility data is Namf_EventExposure.

[0067] As mentioned above, 3GPP TS 23.288 (v17.2.0) specifies that the NWDAF is the primary network function for computing analysis reports. The 5G system architecture allows any NF to obtain analysis from the NWDAF using the DCCF function and associated DCCF services. The NWDAF can also store and retrieve analysis information from the Analysis Data Repository Function (ADRF).

[0068] 3GPP® TS 23.288 also categorizes the NWDAF into two sub-functions (or logical functions): the NWDAF Analysis Logical Function (NWDAFAnLF), which performs analysis procedures, and the NWDAF Model Training Logical Function (NWDAFMTLF), which performs training and retraining of ML models used by the NWDAFAnLF.

[0069] 3GPP TS 23.288 specifies a subscribe / notify procedure for a consumer NF to retrieve ML models associated with one or more analysis IDs whenever new ML models are trained and available by the NWDAFMTLF. This is called ML model provisioning and is implemented by the Nnwdaf_MLModelProvision service.

[0070] 3 shows an example procedure for an NWDAF service consumer (e.g., NWDAFAnLF) to subscribe to notifications about ML model availability from the NWDAF (MTLF). As mentioned above, the procedure is implemented based on the Nnwdaf_MLModel-Provision_Subscribe and Nnwdaf_MLModelProvision_Notify messages (part of the Nnwdaf_MLModelProvision service). 3GPP TS 23.288 Section 6.2A describes the procedure in more detail.

[0071] 3GPP® TS 23.288 also specifies a request / response procedure for a consumer NF (e.g., NWDAFAnLF) to retrieve information about ML models associated with one or more analysis IDs. This procedure is implemented by the Nnwdaf_MLModelInfo service and is shown in Figure 4.

[0072] As described above, these services used by the two NWDAF logical functions operate based on the implicit assumption of a one-to-one relationship between ML models and Analytics IDs. However, in certain scenarios, multiple ML models can be used to generate the analytics reports provided to a particular Analytics ID. For example, these analytics reports can be generated using different ML models depending on location, time of day, network load, or any other information filtering parameters. As another example, these analytics reports can be generated by several cascaded ML models.

[0073] Figure 5 shows an example scenario in which one Analytics ID can be provided by several ML models. In this scenario, three different ML models (A-C) generate their own predictions (A-C) based on the same inputs (a-c). The aggregation / voting function aggregates the predictions to provide a system output Generate.

[0074] Figure 6 shows another example scenario in which one Analytics ID may be provided by several ML models. In this situation, three different ML models (A-C) receive system inputs (a-c), ML model (B) provides two predictions (b-c) that serve as inputs to two other ML models (a, c), and ML models (a, c) provide system outputs (a, b), respectively. In some scenarios, one or both of the ML models (a, c) may be used depending on various conditions.

[0075] Figure 7 illustrates another example scenario in which a single Analytics ID may be provided by several ML models. In this situation, the system includes three different ML models (A-C), each receiving different inputs and each generating different predictions than the other two ML models. However, only one of the ML models is used at any given time, based on various conditions related to each input. Thus, the system output at a particular time depends on which ML model is being used.

[0076] In the case of the Nnwdaf_MLModelProvision service described above, if the NWDAF(MTLF) retrains one of the ML models associated with an Analytics ID and this ML model does not have a one-to-one relationship with an AnalyticsFilter, the notification is not meaningful enough for the NWDAF(AnLF). In other words, because the subscription request does not include an Analytics Filter (only an Analytics ID), the NWDAF(MTLF) returns the location of only the retrained ML model (out of multiple ML models). That information alone is not sufficient for the NWDAF(AnLF) to determine which ML model it should swap.

[0077] A similar problem arises for the Nnwdaf_MLModelInfo service mentioned above: the standard does not specify in detail how to specify the particular ML model associated with the meta-information carried by the service response. Rather, the service consumer is expected to specify the ML model of interest, which, as mentioned above, is not possible.

[0078] Additionally, the response to the Nnwdaf_MLModelInfo request includes ML model metadata, but the actual content is not specified. NWDAF (AnLF) typically requires additional information to execute the ML model for inference. Such information may include the ML model's dependencies, supported execution platforms, instructions for executing additional software components that may be required to execute the ML model, etc. Because AI / ML is a constantly evolving field with new platforms and new technologies emerging, it is impossible to standardize all existing options in advance.

[0079] Accordingly, embodiments of the present disclosure address these and other problems, challenges, and / or difficulties by providing improved techniques for provisioning ML models by introducing explicit references to ML models, conditions for training ML models within an NWDAF (MTLF), and / or conditions for swapping ML models within an NWDAF (AnLF). These techniques provide various benefits and / or advantages. For example, they facilitate an NWDAF (MTLF) to identify the ML model it has retrained when notifying an NWDAF (AnLF), thereby enabling the NWDAF (AnLF) to swap the specific ML model it is using for inference. As another example, they facilitate an NWDAF (AnLF) to express conditions under which the NWDAF (MTLF) should seek to retrain an ML model and whether it should notify the NWDAF (AnLF) when retraining an ML model. As another example, they facilitate expressing training performance results so that the NWDAF (MTLF) can decide whether to swap ML models. At a high level, embodiments improve the management of ML models used for analysis in communications networks (e.g., 5GC).

[0080] The abbreviations "MTLF," "NWDAF(MTLF)," and "MTLF of NWDAF" are used interchangeably herein. Similarly, the abbreviations "AnLF," "NWDAF(AnLF)," and "AnLF of NWDAF" are used interchangeably herein.

[0081] FIG. 8 illustrates a block diagram of an NWDAF according to various embodiments of the present disclosure. The NWDAF illustrated in FIG. 8 includes an AnLF and an MTLF that interact through the Nnwdaf_MLModelProvision and Nnwdaf_MLModelInfo services. The embodiment maintains the existing AnLF / MTLF functional division but updates the service behavior in the Nnwdaf_MLModelProvision and Nnwdaf_MLModelInfo services. Additionally, the MTLF functionality is improved so that upon receiving a Nnwdaf_MLModelProvision_Subscribe request, the MTLF considers conditions in existing subscriptions before training a new ML model.

[0082] According to various embodiments, a new ML model identifier information element is added to the Nnwdaf_MLModelProvision_Subscribe request and the Nnwdaf_MLModelInfo_Request. This ML model identifier can replace the Analytics ID in the request and can be used to filter the request. Several instances of the ML model identifier can be included in a request, enabling requests for several ML models that affect the same Analytics ID. According to some embodiments, a consumer (e.g., AnLF) can include a list of its supported capabilities as an additional filtering option in the Nnwdaf_MLModel-Provision_Subscribe request, so that the consumer receives notifications only for retrained ML models that match the indicated capabilities.

[0083] According to various embodiments, the Nnwdaf_MLModelProvision_Notify message can retain its existing structure for carrying a set of tuples, each of which can include the following information: ┌────────────┬───────────┬────────────┐ │IE name │Existence │Existing / New / Modified │ ├────────────┼───────────┼────────────┤ │Analytics ID│Options (ML model│Existing │ │(Analysis ID) │No identifier) ​​│ │ ├────────────┼───────────┼────────────┤ │ML Model Identifier │Options │New │ │ │ │ │ ├────────────┼───────────┼────────────┤ │ML Model Version │Options │New │ │ │ │ │ ├────────────┼───────────┼────────────┤ │ML Model Manifest │Options │Modifications │ │ or artifacts │ │ │ │URL for │ │ │ └────────────┴───────────┴────────────┘

[0084] According to various embodiments, the Nnwdaf_MLModelInfo_Request response message can retain its existing structure for carrying a set of tuples, each of which can include the following information: ┌────────────┬───────────┬────────────┐ │IE name │Existence │Existing / New / Modified │ ├────────────┼───────────┼────────────┤ │Analytics ID│Options (ML model│Existing │ │(Analysis ID) │No identifier) ​​│ │ ├────────────┼───────────┼────────────┤ │ML Model Identifier │Options │New │ │ │ │ │ ├────────────┼───────────┼────────────┤ │ML Model Version │Options │New │ │ │ │ │ ├────────────┼───────────┼────────────┤ │URL for ML metadata │Optional │Existing │ │ │ │ │ └────────────┴───────────┴────────────┘

[0085] ML model metadata (also called a "manifest") can include information about version, location, capabilities, requirements, etc.

[0086] Various embodiments will now be described in more detail, first with respect to the Nnwdaf_MLModelProvision service and then with respect to the Nnwdaf_MLModelInfo service.

[0087] 3GPP TS 23.288 section 7.5.2 specifies the following data to be included in the Nnwdaf_MLModelProvision_Subscribe request: ● Input, required: (Set of) Analytics IDs defined in Table 7.1-2 "Notification Target Address (+ Notification Correlation ID)". ● Input, optional: Subscription correlation ID (in case of modifying ML model subscription), Analysis filter information indicating the conditions that the ML model needs to have for analysis, and Target of analysis report indicating the objects that the ML model needs to have for analysis, an entity such as a specific UE, a group of UEs, or any UE (all UEs), Target period of the ML model, Expiration date.

[0088] Various embodiments of the present disclosure provide one or more of the following improvements to the existing Nnwdaf_MLModelProvision_Subscribe request: ●Make the Analytics ID(s) optional (i.e., if an ML model identifier is included, this one will not be included). Include (a set of) ML model identifiers as an optional parameter. The request must contain either an Analytics ID or a model identifier. If only the Analytics ID is included, this is used as a filter to select the ML models referenced by the notification. If an ML model identifier is included, the ML model version may also be included. As a consumer, the NWDAF (AnLF) can subscribe to receive notifications about new ML model versions after the first version has been deployed. By including the ML model version in the request, the consumer states that it only wants to be notified if the version is greater (newer) than the version identified by the ML model version. Additional conditions can be included as filters for the NWDAF (MTLF) to notify consumers when an ML model is retrained, e.g., no notification will be issued by the NWDAF (MTLF) unless the new ML model meets the following conditions: The performance metric you are interested in (or its identifier, e.g., "mse" for mean squared error, "rmse" for root mean squared error, or "mae" for mean absolute error; this list is not exhaustive and depends on the type of ML model). ○ Performance metric thresholds. The logical relationship between the performance metric and the threshold (i.e., "gt" for "greater than," "lt" for "less than," etc.). ML / AI framework constraints, e.g., new ML models must be executable on a specific type and version of an ML / AI framework.

[0089] 3GPP TS 29.520 (v17.3.0) section 5.4.6.2.2 defines the existing Nnwdaf_MLModel-Provision_Subscribe request message. Figure 9 illustrates the data structure of an example Nnwdaf_MLModelProvision_Subscribe request message according to various embodiments of the present disclosure. Each arrow leads from an instance of an object in the data structure to the corresponding description (or type) of that object. New fields include mlModelInfo for MLEventSubscription and mlModelConditions for MLAnalyticsFilter, which are defined by the corresponding types shown in Figure 9.

[0090] 3GPP TS 23.288 Section 6.2A.1 specifies that when a subscription request is received, the NWDAF (MTLF) may determine whether further training for an existing trained ML model is required to fulfill the subscription. The standard states that this determination is made based on implementation logic without specifying such logic.

[0091] According to some embodiments, the NWDAF (MTLF) explicitly checks the conditions provided in the notification subscription to determine whether retraining is necessary. Additionally, the NWDAF (MTLF) informs the consumer / subscriber that even if retraining is performed according to the conditions, notifications may be provided conditionally. For example, the NTWDAF (MTLF) can inform the consumer that it will only be notified if the performance metrics of the retrained ML model match the conditions in the subscription.

[0092] 3GPP TS 23.288 section 7.5.4 specifies the following data to be included in the Nnwdaf_MLModelProvision_Notify message: ● Input, required: A set of tuples (analysis ID, address of model file (e.g., URL or FQDN)) and notification correlation information. ● Input, optional: validity period, space validity.

[0093] Various embodiments of the present disclosure provide one or more of the following improvements to the existing Nnwdaf_MLModelProvision_Notify request for each tuple in the message: Adding an ML model identifier, which can be omitted if your Analytics ID implementation includes only one ML model. Add ML model version, and ● If a performance metric identifier is present in the corresponding subscription request, add the performance metric identifier and the performance metric value obtained when training the ML model.

[0094] The above URL can be an ML model URL (which allows for retrieving the ML model) or an ML model manifest URL that points to the ML model metadata (or a description thereof). For example, the ML model metadata can include additional information for the NWDAF (AnLF) to execute the ML model, as well as the actual URL of the ML model. This allows more flexibility to include further information related to model execution without having to encode each attribute as part of the service operation.

[0095] 3GPP TS 29.520 (v17.3.0) section 5.4.6.2.5 specifies the existing Nnwdaf_MLModel-Provision_Notify request message. Figure 10 illustrates the data structure of an example Nnwdaf_MLModel-Provision_Notify request message according to various embodiments of the present disclosure. Each arrow leads from an instance of an object in the data structure to the corresponding description (or type) of that object. New fields include mlModelInfo in MLEventNotify and mlModelConditions in MLAnalyticsFilter, which are defined by the corresponding types. Note that the MLModelPerformanceMetric type is the same as in Figure 9. The MLModelMetadata type is updated to include several additional fields.

[0096] According to another embodiment, the Nnwdaf_MLModelInfo service described in 3GPP® TS 23.288 section 7.6 can be enhanced in a similar manner to the Nnwdaf_MLModelProvision service described above, but without specificity for ML model training. 3GPP® TS 23.288 specifies the following data to be included in the Nnwdaf_MLModelInfo_Request request message: ● Input, required: (Set of) analysis IDs as defined in Table 7.1-2. ● Input, optional: analysis filter information indicating the conditions required by the ML model for analysis, and the target of the analysis report indicating the objects required by the ML model for analysis (e.g., a specific UE, a group of UEs, or any UEs (i.e., all UEs)), the ML model target period.

[0097] Various embodiments of the present disclosure provide one or more of the following improvements to the existing Nnwdaf_MLModelInfo_Request request message: ●Make Analytics ID(s) optional (do not include ML model identifiers, if any). Include (a set of) ML model identifiers as an optional parameter. The request must contain either an Analytics ID or a model identifier. If only the Analytics ID is included, it will be used as a filter to select the ML models referenced by the notification. If an ML model identifier is included, the ML model version may also be included.

[0098] 11 illustrates the data structure of an example Nnwdaf_MLModelInfo_Request request message, according to various embodiments of the present disclosure. Each arrow leads from an instance of an object in the data structure to the corresponding description (or type) of that object. A new field includes mlModelInfo within MLEventSubscription.

[0099] 3GPP TS 23.288 specifies the following data to be included in the Nnwdaf_MLModelInfo_Request response message: Output, required: A set of tuples (Analytics ID, address of the model file). This address can be a URL or a fully qualified domain name (FQDN). ● Output, optional: validity period, spatial validity.

[0100] Various embodiments of the present disclosure provide one or more of the following improvements to the existing Nnwdaf_MLModelInfo_Request response message, particularly for each tuple in the message: Adding an ML model identifier, which is optional if your Analytics ID implementation only includes one ML model. Indicates that the URL references an ML model manifest, which must contain the following required elements: ○ML model identifier. The Analytics ID where the ML model will be used. ○Updated ML model version. Creation timestamp of the updated version of the ML model. ○ML model URL.

[0101] 12 illustrates the data structure of an example Nnwdaf_MLModelInfo_Request response message according to various embodiments of the present disclosure. Each arrow leads from an instance of an object in the data structure to the corresponding description (or type) of that object. New fields include mlModelInfo for MLEventSubscription, which is defined by the corresponding type MLModelInfo. The MLModelMetadata type is updated to include several additional fields.

[0102] The above-described embodiments can be further described with reference to FIGS. 13-14 , which illustrate example methods (e.g., procedures) for the MTLF and AnLF of the NWDAF, respectively. In other words, various features of the operations described below correspond to the various embodiments described above. The example methods illustrated in FIGS. 13-14 may be used in conjunction (e.g., with each other and with other procedures described herein) to provide benefits, advantages, and / or solutions to problems described herein. Although the example methods are illustrated in FIGS. 13-14 by particular blocks in a particular order, the operations corresponding to the blocks may be performed in a different order than shown, and may be combined and / or divided into blocks and / or operations having different functionality than shown. Optional blocks and / or operations are indicated by dashed lines.

[0103] Specifically, Figure 13 shows Disclosure13 illustrates an example method (e.g., procedure) for a first network node or function (NNF) configured for ML model management in a communications network (e.g., 5GC), according to various embodiments. For example, the example method illustrated in FIG. 13 may be performed by a Network Node Driver for Mobile Communications (NWDAF) (MTLF) or a network node hosting a NWDAF (MTLF) as described elsewhere herein.

[0104] An example method may include the operations of block 1310, where a first NNF may receive, from a second NNF of the communication network, a first message including one of the following: one or more ML model identifiers corresponding to one or more ML models maintained by the MTLF; or ● Identifiers of the analysis based on one or more ML models (e.g., if no ML model identifiers are included).

[0105] The example method may also include the operation of block 1340, where the first NNF may send a second message to the AnLF that includes one of the following: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; or ● A single tuple containing an analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

[0106] According to some embodiments, for each tuple included in the second message, the one or more information elements include a network address that is one of a Universal Resource Locator (URL) or a Fully Qualified Domain Name (FQDN). According to some embodiments, each network address included in the second message is one of an address that allows for retrieving the corresponding ML model or an address that allows for retrieving a manifest or metadata file associated with the corresponding ML model. According to some of these embodiments, the manifest or metadata file associated with each of the ML models includes: ●ML model identifier. ●Identifiers of one or more analyses based on the ML model. An updated version of the ML model, where "updated" refers to the latest version of the ML model that has been trained and validated by the MTLF and can therefore be used for inference by the AnLF. Creation timestamp of the updated version of the ML model. A network address that allows retrieving the ML model.

[0107] According to some embodiments, for each tuple included in the second message, the one or more information elements also include an updated version of the corresponding ML model. According to some embodiments, if the first message includes an analysis identifier (e.g., at block 1310), the example method may also include the operation of block 1320, where the first NNF may use the (received) analysis identifier to determine one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based.

[0108] According to other embodiments, if the first message includes one or more of the ML model identifiers, the first message also indicates a version associated with the corresponding ML model. For example, the version included in the first message is the version of the ML model used for inference by the second NNF when the first message is sent. When a new version of the ML model is trained by the first NNF, it is assigned a larger (newer) version identifier. According to some of these embodiments, sending the second message in block 1340 includes one of the following operations, identified by a corresponding subblock number: (1341) selectively sending a second message including a single tuple based on the updated version of the single ML model being newer than the version indicated by the first message; or ● (1342) Selectively including each of the plurality of tuples in a second message based on the updated version of the corresponding ML model being newer than the version indicated by the first message.

[0109] According to some embodiments, the first NNF is an MTLF of the NWDAF, and the second NNF is an AnLF of the NWDAF. According to some of these embodiments, the first message is an Nnwdaf_MLModelInfo_Request message (e.g., FIG. 11), and the second message is a response to the Nnwdaf_MLModelInfo_Request message (e.g., FIG. 12). FIG. 4 shows an example signaling diagram of these messages.

[0110] According to these other embodiments, the first message is a Nnwdaf_MLModel-Provision_Subscribe message (e.g., FIG. 9), and the second message is a Nnwdaf_MLModelProvision_Notify message (e.g., FIG. 10). FIG. 3 shows an example signaling diagram of these messages. According to some of these embodiments, the first message also includes one or more conditions for receiving the second message, including one or more of the following: ●The performance metrics (or identifiers thereof) of one or more ML models on which the analysis is based. • Thresholds for each of the performance metrics. • The logical relationship between performance metrics and thresholds, and ●ML / AI framework constraints for running one or more ML models.

[0111] According to some variations, the example method may also include the operation of block 1330, where the first NNF may determine whether to retrain one or more ML models based on whether the current values ​​of the performance metrics of the respective models satisfy respective thresholds and logical relationships (e.g., as indicated in the first message).

[0112] According to some variations, sending the second message in block 1340 includes one of the following operations, identified by the corresponding sub-block number: ● (1343) selectively sending a second message including a single tuple based on a single ML model that satisfies one or more conditions included in the first message; or ● (1344) Selectively including each of the plurality of tuples in the second message based on a corresponding ML model that satisfies one or more conditions included in the first message.

[0113] According to some further variations, for each tuple included in the second message, the one or more information elements also include an identifier of a performance metric indicated in the first message and a value of the identified performance metric for the corresponding ML model.

[0114] 14 illustrates an example method (e.g., procedure) for a second NNF configured for ML model management in a communications network (e.g., 5GC) according to various embodiments of the present disclosure. For example, the example method illustrated in FIG. 14 may be performed by an NWDAF (AnLF) or a network node hosting an NWDAF (AnLF), as described elsewhere herein.

[0115] The example method may include the operation of block 1420, where the second NNF may send a first message to the first NNF of the communication network, the first message including one of the following: one or more ML model identifiers corresponding to one or more ML models maintained by the MTLF; or ● Identifiers of the analysis based on one or more ML models (e.g., if no ML model identifiers are included).

[0116] The example method may also include the operation of block 1430, where the second NNF may receive, from the MTLF, a second message including one of the following: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; or ● A single tuple containing an analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

[0117] According to some embodiments, for each tuple included in the second message, the one or more information elements include a network address that is a URL or an FQDN. According to some embodiments, each network address included in the second message is one of an address that allows for retrieving a corresponding ML model or an address that allows for retrieving a manifest or metadata file associated with the corresponding ML model.

[0118] According to some of these embodiments, the manifest or metadata file associated with each of the ML models includes: ●ML model identifier. ●An identifier for one or more analyses based on the ML model. ●Updated version of the ML model. Creation timestamp of the updated version of the ML model. A network address that allows retrieving the ML model.

[0119] According to these other embodiments, the example method may also include the operations of blocks 1440-1450, where the second NNF may retrieve a single ML model or multiple ML models based on the network address in the corresponding single tuple or multiple tuples (e.g., included in the second message) and apply the retrieved single ML model or multiple ML models to determine the analysis identified by the analysis identifier.

[0120] According to some embodiments, for each tuple included in the second message, one or more information elements also include an updated version of the corresponding ML model. According to some embodiments, if the first message includes one or more of the ML model identifiers, the first message also indicates the version associated with the corresponding ML model. As described above, the version included in the first message may be the version of the ML model used for inference by the second NNF when the first message is sent. When a new version of the ML model is trained by the first NNF, it can be assigned a larger (newer) version identifier.

[0121] In some of these embodiments, one or more of the following apply: The second message containing a single tuple is received only if the updated version of the single ML model is newer than the version indicated by the first message. Each of the multiple tuples is received in the second message only if the updated version of the corresponding ML model is newer than the version indicated by the first message.

[0122] According to some embodiments, the first NNF is an MTLF of the NWDAF, and the second NNF is an AnLF of the NWDAF. According to some of these embodiments, the first message is an Nnwdaf_MLModelInfo_Request message (e.g., FIG. 11), and the second message is a response to the Nnwdaf_MLModelInfo_Request message (e.g., FIG. 12). FIG. 4 shows an example signaling diagram of these messages.

[0123] According to these other embodiments, the first message is a Nnwdaf_MLModel-Provision_Subscribe message (e.g., FIG. 9), and the second message is a Nnwdaf_MLModelProvision_Notify message (e.g., FIG. 10). FIG. 3 shows an example signaling diagram of these messages. According to some of these embodiments, the first message also includes one or more conditions for receiving the second message, including one or more of the following: ●The performance metrics (or identifiers thereof) of one or more ML models on which the analysis is based. • Thresholds for each of the performance metrics. The logical relationship between performance metrics and thresholds; and ●ML / AI framework constraints for running one or more ML models.

[0124] According to some variants, one or more of the following apply: A second message containing a single tuple is received only if a single ML model satisfies one or more conditions contained in the first message. ● Each of the multiple tuples is received in the second message only if the corresponding ML model satisfies one or more conditions included in the first message.

[0125] According to some further variations, or in each tuple included in the second message, the one or more information elements also include an identifier of the performance metric indicated in the first message and a value of the identified performance metric for the corresponding ML model.

[0126] According to some embodiments, the example method may also include the operation of block 1410, where the second NNF may use the analysis identifier to determine one or more ML model identifiers corresponding to one or more ML models on which the analysis is based. In such a case, the first message includes the determined one or more ML model identifiers.

[0127] Although various embodiments are described herein above with respect to methods, apparatus, devices, computer-readable media, and receivers, such methods may be implemented using hardware and software in various systems, communication devices, computing devices, control devices, apparatus, non-transitory computer-readable media, etc. Eh Those skilled in the art will readily appreciate that the present invention can be implemented by various combinations of the above.

[0128] 15 illustrates an example communications system 1500 according to some embodiments. In this illustration, the communications system 1500 includes a telecommunications network 1502 including an access network 1504, such as a radio access network (RAN), and a core network 1506 including one or more core network nodes 1508. The access network 1504 includes one or more access network nodes, such as network nodes 1510a and 1510b (one or more of which may be generally referred to as network nodes 1510), or any other similar Third Generation Partnership Project (3GPP) access nodes or non-3GPP access points. The network nodes 1510 enable direct or indirect connectivity of user equipment (UE) 1512a, 1512b, 1512c, and 1512d (one or more of which may be generally referred to as UE 1512) to the core network 1506 via one or more wireless connections.

[0129] Exemplary wireless communication over a wireless connection includes transmitting and / or receiving radio signals using electromagnetic waves, radio waves, infrared, and / or other types of signals suitable for conveying information without the use of wires, cables, or other data conductors. Further, according to various embodiments, communication system 1500 may include any number of wired or a wireless network, a network node, a UE, and / or wired or any other components or systems that may facilitate or be involved in the communication of data and / or signals, whether via a wireless connection or not. Communications system 1500 may include and / or interface with any type of communications, telecommunications, data, cellular, wireless network, and / or other similar type systems.

[0130] The UE 1512 may be any of a wide variety of communication devices, including wireless devices designed, configured, and / or operable to communicate wirelessly with the network node 1510 and other communication devices. Similarly, the network node 1510 is designed, enabled, configured, and / or operable to communicate directly or indirectly with the UE 1512 and / or other network nodes or apparatuses within the telecommunications network 1502 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as management in the telecommunications network 1502.

[0131] According to the illustrated embodiment, core network 1506 connects network node 1510 to one or more hosts, such as host 1516. These connections may be direct or indirect through one or more intermediate networks or devices. In other examples, the network node may be directly coupled to the host. Core network 1506 may also include hardware Eh Art and software EhThe UE includes one or more core network nodes (e.g., core network node 1508) comprised of various components. Features of these components may be substantially similar to those described with respect to the UE, network node, and / or host, and therefore, those descriptions are generally applicable to the corresponding components of the core network node 1508. An example core network node includes one or more functions of a Mobile Switching Center (MSC), a Mobility Management Entity (MME), a Home Subscriber Server (HSS), an Access and Mobility Management Function (AMF), a Session Management Function (SMF), an Authentication Server Function (AUSF), a Subscription Identifier Unhiding Function (SIDF), a Unified Data Management (UDM), a Security Edge Protection Proxy (SEPP), a Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0132] As a specific example, one or more core network nodes 1508 may be configured to perform operations attributed to a first NNF and a second NNF in the descriptions of various methods or procedures herein. More specifically, one or more core network nodes 1508 may be configured to perform operations attributed to an MTLF of an NWDAF and an AnLF of an NWDAF.

[0133] The host 1516 may be under the ownership or control of, and may be operated by, or on behalf of, a service provider other than the operator or provider of the access network 1504 and / or the telecommunications network 1502. The host 1516 may host various applications to provide one or more services. Examples of such applications include live and pre-recorded audio / video content, data collection services such as searching and compiling data about various ambient conditions detected by multiple UEs, analytics functions, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and monitoring center, or any other such function performed by a server.

[0134] Overall, the communication system 1500 of Figure 15 enables connectivity between UEs, network nodes, and hosts. In that sense, the communication system may include, but is not limited to: Mobile Global System for Communications (GSM), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standards (e.g., 6G), wireless local area network (WLAN) standards such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard (WiFi), and / or World Wide Web standards for microwave access. interoperability The wireless communication device may be configured to operate according to predetermined rules or procedures, such as a particular standard, including any other suitable wireless communication standard, such as WiMax, Bluetooth, Z-Wave, Near Field Communication (NFC), ZigBee, LiFi, and / or any Low Power Wide Area Network (LPWAN) standard, such as LoRa and Sigfox.

[0135] In some examples, the telecommunications network 1502 implements features of 3GPP standardization. Thus, the telecommunications network 1502 may support network slicing to provide different logical networks to different devices connected to the telecommunications network 1502. For example, the telecommunications network 1502 may provide Ultra-Reliable Low-Latency Communications (URLLC) services to some UEs, while providing enhanced Mobile Broadband (eMBB) services to other UEs and / or Massive Machine Type Communications (mMTC) / Massive IoT services to additional UEs.

[0136] In some examples, the UE 1512 is configured to transmit and / or receive information without direct human interaction. For example, the UE may be designed to transmit information to the access network 1504 on a predetermined schedule, when triggered by an internal or external event, or in response to a request from the access network 1504. Furthermore, the UE may be configured to operate in a single or multi-RAT or multi-standard mode. For example, the UE may operate in any one or combination of Wi-Fi, NR (New Radio), and LTE, i.e., configured with Multi-Radio Dual Connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio-Dual Connectivity (EN-DC).

[0137] In the illustrated example, the hub 1514 communicates with the access network 1504 to facilitate indirect communication between one or more UEs (e.g., UEs 1512c and / or 1512d) and a network node (e.g., network node 1510b). In some examples, the hub 1514 may be a controller, a router, a content source and analysis, or any of the other communication devices described herein with respect to UEs. For example, the hub 1514 may be a broadband router that enables access to the core network 1506 for the UE. As another example, the hub 1514 may be a controller that sends commands or instructions to one or more actuators within the UE. The instructions or instructions may be received from the UE, the network node 1510, or by executable code, scripts, processes, or other instructions within the hub 1514. As another example, the hub 1514 may be a data collector that acts as a temporary storage for UE data and may perform analysis or other processing of the data, according to some embodiments. As another example, the hub 1514 may be a content source. For example, in the case of a UE that is a VR headset, display, loudspeaker, or other media distribution device, the hub 1514 may retrieve data related to VR assets, video, audio, or other media or sensory information via a network node, which the hub 1514 then provides directly to the UE, either after performing local processing and / or adding additional local content. In yet another example, the hub 1514 acts as a proxy server or orchestrator for the UEs, particularly if one or more of the UEs are low-energy IoT devices.

[0138] The hub 1514 may have a constant / persistent or intermittent connection to the network node 1510b. The hub 1514 may also enable other communication schemes and / or schedules between the hub 1514 and the UEs (e.g., UEs 1512c and / or 1512d) and between the hub 1514 and the core network 1506. According to other embodiments, the hub 1514 may: wiredThe hub 1514 may be configured to connect to the core network 1506 and / or one or more UEs via a direct connection. Additionally, the hub 1514 may be configured to connect to an M2M service provider via the access network 1504 and / or to another UE via a direct connection. In some situations, the UE may wired Alternatively, the UE may establish a wireless connection with the network node 1510b while still connected via the hub 1514 via a wireless connection. According to some embodiments, the hub 1514 may be a dedicated hub, i.e., a hub whose primary function is to route communications to / from the network node 1510b to the UE. According to other embodiments, the hub 1514 may be a non-dedicated hub, i.e., a device that is capable of operating to route communications between the UE and the network node 1510b, but that is also capable of operating as a communication origination and / or termination point for a particular data channel.

[0139] 16 illustrates a UE 1600 according to some embodiments. As used herein, a UE refers to a device capable of, configured, configured, and / or operable to wirelessly communicate with network nodes and / or other UEs. Examples of UEs include smartphones, move Examples include, but are not limited to, telephones, mobile phones, voice-over-IP (VoIP) phones, wireless local loop phones, desktop computers, personal digital assistants (PDAs), wireless cameras, gaming consoles or devices, music storage devices, playback appliances, wearable terminal devices, wireless endpoints, mobile stations, tablets, laptops, laptop embedded equipment (LEE), laptop mounted equipment (LME), smart devices, wireless customer premises equipment (CPE), in-vehicle or embedded / integrated wireless devices, etc. Other examples include any UE specified by the 3rd Generation Partnership Project (3GPP®), including narrowband Internet of Things (NB-IoT) UE, machine type communication (MTC) UE, and / or enhanced MTC (eMTC) UE.

[0140] A UE may support device-to-device (D2D) communications, for example, by implementing 3GPP standards for sidelink communications, dedicated short-range communications (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates an associated device. Instead, a UE may represent a device that is intended for sale to or operation by a human user, but that may or may not initially be associated with a particular human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to or operation by an end user, but that may be associated with or operated for a user (e.g., a smart electricity meter).

[0141] The UE 1600 includes a processing circuit 1602 operably coupled to an input / output interface 1606, a power source 1608, a memory 1610, a communication interface 1612, and / or any other components, or any combination thereof, via a bus 1604. Some UEs may utilize all or a subset of the components shown in FIG. 16. The level of integration between components may vary from UE to UE. Additionally, some UEs may include multiple instances of components, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0142] The processing circuit 1602 is configured to process instructions and data and may be configured to implement any sequential state machine operable to execute instructions stored as a machine-readable computer program in the memory 1610. The processing circuit 1602 may be implemented as one or more hardware-implemented state machines (e.g., discrete logic, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.), programmable logic with appropriate firmware, one or more stored computer programs such as a microprocessor or digital signal processor (DSP) with appropriate software, a general-purpose processor, or any combination of the above. For example, the processing circuit 1602 may include multiple central processing units (CPUs).

[0143] In this example, the input / output interface 1606 may be configured to provide an interface to an input device, an output device, or one or more input and / or output devices. Examples of output devices include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smart card, another output device, or any combination thereof. An input device may allow a user to capture information on the UE 1600. Examples of input devices include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a webcam, etc.), a microphone, a sensor, a mouse, a trackball, a directional keypad, a trackpad, a scroll wheel, a smart card, etc. A presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. The sensor may be, for example, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, a light sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device can use the same type of interface port as an input device. For example, a universal serial bus (USB) port may be used to provide an input device and an output device.

[0144] According to some embodiments, the power source 1608 is configured as a battery or battery pack. Other types of power sources can be used, such as an external power source (e.g., an electrical outlet), a photovoltaic device, or a power cell. The power source 1608 may further include power circuitry for delivering power to various portions of the UE 1600 from the power source 1608 itself and / or from the external power source via an interface, such as an input circuit or a power cable. The power delivery may be for charging the power source 1608, for example. The power circuitry may perform any formatting, conversion, or other modification of the power from the power source 1608 to make it suitable for each component of the UE 1600 being powered.

[0145] The memory 1610 may be or be configured to include random access memory (RAM), read-only memory (ROM), field programmable gate array read-only memory (PROM), erasable field programmable gate array read-only memory (EPROM), electrically erasable field programmable gate array read-only memory (EEPROM), magnetic disk, optical disk, hard disk, removable cartridge, flash drive, or other memory. According to one embodiment, the memory 1610 includes one or more application programs 1614, such as an operating system, a web browser application, a widget, a gadget engine, or other applications, and corresponding data 1616. The memory 1610 may store any of a variety of operating systems or combinations of operating systems for use by the UE 1600.

[0146] The memory 1610 may be configured to include several physical drives, such as a redundant array of independent disks (RAID), flash memory, a USB flash drive, an external hard disk drive, a thumb drive, a pen drive, a key drive, a high-density digital versatile disc (HD-DVD) optical disc drive, an internal hard disk drive, a Blu-ray optical disc drive, a holographic digital data storage (HDDS) optical disc drive, an external mini dual in-line memory module (DIMM), a synchronous dynamic random access memory (SDRAM), an external micro-DIMM SDRAM, a smart card memory such as a tamper-resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), e.g., a USIM and / or ISIM, other memory, or any combination thereof. The UICC may be, for example, an embedded UICC (eUICC), an integrated UICC (iUICC), or a removable UICC commonly known as a "SIM card." The memory 1610 may enable the UE 1600 to access, offload, or upload data, instructions, application programs, etc. stored on a temporary or non-transitory memory medium. An article of manufacture, such as one utilizing a communications system, may be tangibly embodied as or in memory 1610, which may be or include a device-readable storage medium.

[0147] The processing circuit 1602 may be configured to communicate with an access network or other networks using a communication interface 1612. The communication interface 1612 may comprise one or more communication subsystems and may include, or be communicatively coupled to, an antenna 1622. The communication interface 1612 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or network node in the access network). Each transceiver may include a transmitter 1618 and / or a receiver 1620 suitable for providing network communication (e.g., optical, electrical, frequency allocation, etc.). Furthermore, the transmitter 1618 and receiver 1620 may be coupled to one or more antennas (e.g., antenna 1622) and may share circuit components, software, or firmware or may be implemented separately.

[0148] According to the illustrated embodiment, the communication capabilities of communication interface 1612 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communication such as Bluetooth®, short-range communication, location-based communication such as use of the Global Positioning System (GPS) to determine location, another similar communication capability, or any combination thereof. Communications may be implemented in accordance with one or more communication protocols and / or standards such as IEEE 802.11, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Synchronous Optical Networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), etc.

[0149] Regardless of the type of sensor, the UE can provide data output captured by its sensor to a network node via a wireless connection via its communications interface 1612. Data captured by a sensor in a UE can be communicated to a network node via a wireless connection via another UE. The output can be periodic (e.g., once every 15 minutes when reporting a sensed temperature), random (e.g., over a load from reports from several sensors), in response to a trigger event (e.g., when moisture is detected, when an alert is sent), upon request (e.g., a user-initiated request), or in response to a continuous stream (e.g., a live video feed of a patient). (ringing) It could be.

[0150] As another example, the UE may receive wireless signals from a network node via a wireless connection. input and an actuator, motor, or switch associated with a communication interface configured to receive wireless input. In response to the received wireless input, the state of the actuator, motor, or switch may change. For example, the UE may control a flying robot arm to perform a medical procedure in accordance with the received input. drone The aircraft may include motors for adjusting the control surfaces or rotors of the aircraft.

[0151] When a UE is in the form of an Internet of Things (IoT) device, it may be a device for use in one or more applications, including, but not limited to, urban wearable technology, extended industrial applications, and healthcare. Non-limiting examples of such IoT devices are devices such as or mounted on a connected refrigerator or freezer, a TV, a connected lighting device, an electric meter, a robotic vacuum cleaner, a voice-controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electric door lock, a connected doorbell, an air conditioning system such as a heat pump, an autonomous vehicle, a surveillance system, a weather monitor, a vehicle parking monitor, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for augmented reality (AR) or virtual reality (VR), a wearable for haptic or sensory augmentation, a water sprinkler, a device for tracking an animal or an object, a sensor for monitoring flora and fauna, an industrial robot, an unmanned aerial vehicle (UAV), and any type of medical device such as a heart rate monitor or a remotely operated surgical robot. A UE in the form of an IoT device comprises circuitry and / or software depending on the intended application of the IoT device, in addition to other components as described in relation to UE 1600 shown in FIG. 16.

[0152] As yet another specific example, in an IoT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may, in this case, be an M2M device, which may be referred to as an MTC device in a 3GPP® context. As one particular example, the UE may implement the 3GPP® NB-IoT standard. In other scenarios, a UE may represent a vehicle, such as an automobile, bus, truck, ship, and aircraft, or other equipment that can monitor and / or report its operating state or other functions related to its operation.

[0153] In practice, any number of UEs may be used together for a single use case. For example, a first UE may be a drone or may be integrated into a drone and may provide drone speed information (obtained via a speed sensor) to a second UE that is a remote controller operating the drone. When a user makes a change from the remote controller, the first UE may adjust a throttle on the drone (e.g., by controlling an actuator) to increase or decrease the drone's speed. The first and / or second UE may also include two or more of the functions described above. For example, a UE may be equipped with a sensor and an actuator and handle communication of data for both the speed sensor and the actuator.

[0154] 17 illustrates a network node 1700 according to some embodiments. As used herein, a network node refers to a configured, arranged, and / or operative device that can communicate directly or indirectly with UEs and / or other network nodes or devices in a telecommunications network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., wireless access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs), and NR Node Bs (gNBs)).

[0155] Base stations may be classified based on the size of the coverage they provide (or, stated differently, their transmit power level) and may therefore be referred to as femto, pico, micro, or macro base stations depending on the size of the coverage provided. A base station may also be a relay node or a relay donor node that controls a relay. A network node may also include one or more (or all) parts of a distributed radio base station, such as a centralized digital unit and / or a remote radio unit (RRU), sometimes referred to as a remote radio head (RRH). Such remote radio units may or may not be integrated with an antenna as an antenna-integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0156] Other examples of network nodes include a multi-transmission point (multi-TRP) 5G access node, multi-standard radio (MSR) equipment such as an MSR BS, a network controller such as a radio network controller (RNC) or base station controller (BSC), a base transceiver station (BTS), a transmission point, a transmitting node, a multi-cell / multicast coordination entity (MCE), an operations and maintenance (O&M) node, an operations support system (OSS) node, a self-organizing network (SON) node, a positioning node (e.g., an evolved serving mobile location center (E-SMLC)), and / or a minimization of drive test (MDT).

[0157] The network node 1700 includes a processing circuit 1702, a memory 1704, a communication interface 1706, and a power source 1708. The network node 1700 may be comprised of multiple physically separate components (e.g., a Node B component and an RNC component, or a BTS component and a BSC component, etc.), each of which may have its own components. In certain situations where the network node 1700 includes multiple separate components (e.g., a BTS and a BSC component), one or more of the separate components may be shared among multiple network nodes. For example, a single RNC may control multiple Node Bs. In such a scenario, each unique Node B and RNC pair may potentially be considered a single individual network node. In some embodiments, the network node 1700 may be comprised of multiple Nothing The network node 1700 may be configured to support multiple wireless access technologies (RATs). According to such an embodiment, some components may be duplicated (e.g., separate memory 1704 for different RATs) and some components may be reused (e.g., the same antenna 1710 may be shared by different RATs). The network node 1700 may also include multiple sets of the various illustrated components for the various wireless technologies integrated into the network node 1700, e.g., GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, radio frequency identification device (RFID), or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chips or chipsets and other components within the network node 1700.

[0158] The processing circuit 1702 may include one or more combinations of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software, and / or coded logic, which combination is operable, alone or in conjunction with other network node 1700 components, such as memory 1704, to provide the network node 1700 functionality.

[0159] According to some embodiments, the processing circuit 1702 comprises a system on a chip (SOC). According to some embodiments, the processing circuit 1702 includes one or more of a radio frequency (RF) transceiver circuit 1712 and a baseband processing circuit 1714. In some embodiments, the radio frequency (RF) transceiver circuit 1712 and the baseband processing circuit 1714 may be on separate chips (or chipsets), boards, or units, such as a radio unit and a digital unit. In alternative embodiments, some or all of the RF transceiver circuit 1712 and the baseband processing circuit 1714 may be on the same chip or chipset, board, or unit.

[0160] The memory 1704 may comprise any form of volatile or non-volatile computer-readable memory, including, but not limited to, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (e.g., hard disk), removable storage media (e.g., flash drive, compact disc (CD) or digital video disc (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory device that stores information, data, and / or instructions that can be used by the processing circuit 1702. The memory 1704 may store computer programs, software, EhThe memory 1704 may store any suitable instructions, data, or information that can be executed by the processing circuit 1702 and utilized by the network node 1700, including applications including one or more of: software, logic, rules, code, tables, and / or other instructions (collectively referred to as computer program product 1704a). The memory 1704 may be used to store any operations performed by the processing circuit 1702 and / or any data received via the communications interface 1706. According to some embodiments, the processing circuit 1702 and the memory 1704 are integrated.

[0161] The communications interface 1706 is used for wired or wireless communication of signaling and / or data between network nodes, access networks, and / or UEs. As shown, the communications interface 1706 includes a port / terminal 1716, for example, for transmitting and receiving data to and from a network via a wired connection. The communications interface 1706 also includes a radio front-end circuit 1718, which may be coupled to the antenna 1710, or to portions thereof according to an embodiment. The radio front-end circuit 1718 includes a filter 1720 and an amplifier 1722. The radio front-end circuit 1718 may be connected to the antenna 1710 and the processing circuit 1702. The radio front-end circuit may be configured to condition signals communicated between the antenna 1710 and the processing circuit 1702. The radio front-end circuit 1718 may receive digital data to be sent to other network nodes or UEs via a wireless connection. The radio front-end circuitry 1718 may convert the digital data into a radio signal having appropriate channel and bandwidth parameters using a combination of filters 1720 and / or amplifiers 1722. The radio signal may then be transmitted via the antenna 1710. Similarly, when receiving data, the antenna 1710 may collect the radio signal, which is converted into digital data by the radio front-end circuitry 1718. The digital data may be passed to the processing circuit 1702. In other embodiments, the communication interface may include different components and / or different combinations of components.

[0162] According to certain alternative embodiments, the network node 1700 does not include a separate radio front-end circuit 1718; instead, the processing circuit 1702 includes the radio front-end circuitry and is connected to the antenna 1710. Similarly, according to some embodiments, all or some of the RF transceiver circuitry 1712 is part of the communications interface 1706. According to yet other embodiments, the communications interface 1706 includes one or more ports or terminals 1716, the radio front-end circuitry 1718, and the RF transceiver circuitry 1712 as part of a radio unit (not shown), and the communications interface 1706 communicates with baseband processing circuitry 1714 that is part of a digital unit (not shown).

[0163] The antenna 1710 may include one or more antennas or an antenna array configured to transmit and / or receive wireless signals. The antenna 1710 may be coupled to the radio front-end circuitry 1718 and may be any type of antenna capable of wirelessly transmitting and receiving data and / or signals. According to an embodiment, the antenna 1710 is separate from the network node 1700 and may be connectable to the network node 1700 through an interface or port.

[0164] The antenna 1710, the communication interface 1706, and / or the processing circuit 1702 may be configured to perform any receiving operations and / or some obtaining operations described herein as being performed by a network node. Any information, data, and / or signals may be transmitted to or received by the UE, another network node, and / or any other Network devices Similarly, the antenna 1710, the communication interface 1706, and / or the processing circuit 1702 may be configured to perform any transmission operation described herein as being performed by a network node. Any information, data, and / or signals may be transmitted to or received from a UE, another network node, and / or any other Network devices may be sent to

[0165] The power supply 1708 provides power to the various components of the network node 1700 in a form appropriate for each component (e.g., at voltage and current levels required by each component). The power supply 1708 may further comprise or be coupled to power management circuitry for supplying power to the components of the network node 1700 to perform the functions described herein. For example, the network node 1700 may be connectable to an external power source (e.g., a power grid, an electrical outlet) via an input circuit or interface, such as a wire, whereby the external power source provides power to the power circuitry of the power supply 1708. As a further example, the power supply 1708 may comprise a power source in the form of a battery or battery pack connected to or integrated into the power circuitry. In the event of a failure of the external power source, backup power may be provided by the battery.

[0166] Embodiments of network node 1700 may include additional components beyond those shown in Figure 17 to provide particular aspects of the network node's functionality, including any of the functionality described herein and / or any functionality essential to supporting the subject matter described herein. For example, network node 1700 may include user interface devices that allow for the input of information into network node 1700 and the output of information from network node 1700. This allows a user to perform diagnostic, maintenance, repair, and other management functions on network node 1700.

[0167] As a specific example, one or more network nodes 1700 may be configured to perform operations attributed to a first NNF and a second NNF in the descriptions of various methods or procedures herein. More specifically, one or more network nodes 1700 may be configured to perform operations attributed to an MTLF of an NWDAF and an AnLF of an NWDAF.

[0168] 18 is a block diagram of a host 1800, which may be an embodiment of the host 1516 of FIG. 15, in accordance with various aspects described herein. As used herein, the host 1800 may be or comprise various combinations of hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, a container, or processing resources in a server farm. The host 1800 may provide one or more services to one or more UEs.

[0169] Host 1800 includes a processing circuit 1802 operably coupled to an input / output interface 1806, a network interface 1808, a power supply 1810, and a memory 1812 via a bus 1804. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 16 and 17, such that the descriptions are generally applicable to corresponding components of host 1800.

[0170] The memory 1812 may include one or more computer programs, including one or more host application programs 1814 and data 1816, which may include user data, e.g., data generated by the UE for the host 1800 or data generated by the host 1800 for the UE. An embodiment of the host 1800 may utilize only a subset or all of the components shown. The host application programs 1814 may be implemented in a container-based architecture, allowing for multiple different classes, types, or implementations of UE (e.g., handsets, desktop computers, EhThe host application program 1814 may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for mobile display systems, head-up display systems, and other applications. The host application program 1814 may also provide user authentication and license checks and may periodically report health, route, and content availability to a central node, such as a device within the core network or on its edge. Thus, the host 1800 may select and / or indicate different hosts for over-the-top services for the UE. The host application program 1814 may support various protocols, such as HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), and Dynamic Adaptive Streaming over HTTP (MPEG-DASH).

[0171] FIG. 19 is a block diagram illustrating a virtualization environment 1900 in which functionality implemented by some embodiments may be virtualized. In this context, virtualization refers to creating a virtual version of a device or device, including virtualizing a hardware platform, storage, and networking resources. As used herein, virtualization may apply to any device described herein, or components thereof, and relates to implementations in which at least a portion of functionality is implemented as one or more virtual components. Some or all of the functionality described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1900 hosted by one or more hardware nodes, such as a network node, a UE, a core network node, or a hardware computing device acting as a host. Furthermore, in embodiments in which the virtual node does not require wireless connectivity (e.g., a core network node or a host), the node may be fully virtualized.

[0172] An application 1902 (which may alternatively be referred to as a software instance, a virtual appliance, a network function, a virtual node, a virtual network function, etc.) executes in the virtualized environment Q400 to implement some of the features, functions, and / or advantages of some of the embodiments disclosed herein.

[0173] As a particular example, the first NNF and / or second NNF described herein with respect to other figures may be implemented as a virtual network function 1902 in the virtualized environment 1900. As a more specific example, the MTLF of the NWDAF and / or the AnLF of the NWDAF may be implemented as a virtual network function 1902 in the virtualized environment 1900.

[0174] The hardware 1904 includes processing circuitry, memory that stores software and / or instructions executable by the hardware processing circuitry (collectively referred to as computer program product 1904a), and / or other hardware devices described herein, such as network interfaces, input / output interfaces, etc. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1906 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1908a and 1908b (one or more of which may be generally referred to as VMs 1908), and / or perform any of the functions, features, and / or benefits described in connection with some embodiments described herein. The virtualization layer 1906 may present a virtual operating platform to the VMs 1908 that appears to be networking hardware.

[0175] The VMs 1908 may comprise virtual processing, virtual memory, virtual networking or interfaces, and virtual storage and may be executed by a corresponding virtualization layer 1906. Various embodiments of instances of virtual appliances 1902 may be implemented on one or more of the VMs 1908, and the implementation may be done in different ways. Hardware virtualization occurs in some contexts called network functions virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry-standard high-capacity server hardware, physical switches, and physical storage that may be located in data centers, as well as customer premises equipment.

[0176] In the context of NFV, VM 1908 may be a software implementation of a physical machine that executes programs as if they were running on a physical, non-virtualized machine. Each VM 1908, and that portion of hardware 1904 on which it runs, is hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forming a separate virtual network element. Furthermore, in the context of NFV, virtual network functions run in one or more VMs 1908 on hardware 1904 and are responsible for handling specific network functions corresponding to applications 1902.

[0177] The hardware 1904 may be implemented in a standalone network node having generic or specific components. The hardware 1904 may implement some functions via virtualization. Alternatively, the hardware 1904 may be part of a larger cluster of hardware (e.g., in a data center or CPE) where many hardware nodes cooperate and are managed via a management and orchestration 1910 that oversees, among other things, the lifecycle management of the application 1902. According to some embodiments, the hardware 1904 is coupled to one or more radio units, each including one or more transmitters and one or more receivers that may be coupled to one or more antennas. The radio units can communicate directly with other hardware nodes via one or more appropriate network interfaces and wirelessly to virtual nodes. access It may be used in combination with virtual components to provide wireless functionality such as a node or base station. According to some embodiments, some signaling may be provided using the control system 1912, which may alternatively be hardware. Eh It can be used for communication between the anode and the radio unit.

[0178] Figure 20 illustrates a communication diagram of a host 2002 communicating over a partial wireless connection with a UE 2006 via a network node 2004, according to some embodiments. Exemplary embodiments of the UE (such as the UE 1512a of Figure 15 and / or the UE 1600 of Figure 16), network node (such as the network node 1510a of Figure 15 and / or the network node 1700 of Figure 17), and host (such as the host 1516 of Figure 15 and / or the host 1800 of Figure 18) described in the previous paragraphs, according to various embodiments, are described using Figure 20.

[0179] Similar to the host 1800, an embodiment of the host 2002 includes hardware such as a communications interface, processing circuitry, and memory. The host 2002 also includes software stored on or accessible by the host 2002 and executable by the processing circuitry. This software includes a host application that may be operable to provide services to a remote user, such as a UE 2006, connecting via an over-the-top (OTT) connection 2050 extending between the UE 2006 and the host 2002. When providing services to a remote user, the host application may provide user data that is transmitted using the OTT connection 2050.

[0180] The network node 2004 is a hardware device for communicating with the host 2002 and the UE 2006. Eh The connection 2060 can be direct or pass through one or more other intermediate networks, such as a core network (such as core network 1506 in FIG. 15) and / or one or more public, private, or hosted networks. For example, the intermediate network may be a backbone network or the Internet.

[0181] The UE 2006 includes hardware and software stored on or accessible by the UE 2006 and executable by the UE's processing circuitry. The software includes a client application, such as a web browser or operator-specific "app," that may be operable to provide services to a human or non-human user via the UE 2006 with the support of the host 2002. A host application running on the host 2002 can communicate with a client application running on the UE 2006 via an OTT connection 2050 that terminates at the UE 2006 and the host 2002. In providing services to the user, the client application on the UE can receive request data from the host application on the host and provide user data in response to the request data. The OTT connection 2050 may carry both request data and user data. The client application on the UE can interact with the user and generate user data to provide to the host application via the OTT connection 2050.

[0182] The OTT connection 2050 may provide a connection between the host 2002 and the UE 2006 via a connection 2060 between the host 2002 and the network node 2004 and via a wireless connection 2070 between the network node 2004 and the UE 2006. The connections 2060 and wireless connections 2070 over which the OTT connection 2050 may be provided are depicted abstractly to illustrate communication between the host 2002 and the UE 2006 via the network node 2004, although any intermediate devices and the exact routing of messages through these devices are not explicitly mentioned.

[0183] As an example of transmitting data over the OTT connection 2050, in step 2008, the host 2002 provides user data that can be executed by executing a host application. According to some embodiments, the user data is associated with a particular human user interacting with the UE 2006. According to other embodiments, the user data is associated with a UE 2006 that shares data with the host 2002 without explicit human interaction. In step 2010, the host 2002 initiates a transmission carrying the user data toward the UE 2006. The host 2002 can initiate the transmission in response to a request sent by the UE 2006. The request can be caused by human interaction with the UE 2006 or by the operation of a client application running on the UE 2006. The transmitted signal can be used to Disclosure In accordance with the teachings of the embodiments described throughout, the network node 2004 may pass through the network node 2004. Thus, in step 2012, the network node 2004 Disclosure In accordance with the teachings of the embodiments described herein, the host 2002 transmits user data carried in a transmission initiated by the host 2002 to the UE 2006. In step 2014, the UE 2006 receives the user data carried in the transmission, which may be executed by a client application executing on the UE 2006 associated with the host application executed by the host 2002.

[0184] In some examples, the UE 2006 executes a client application that provides user data to the host 2002. The user data may be provided in reaction to or in response to data received from the host 2002. Thus, in step 2016, the UE 2006 may provide the user data, which may be executed by executing the client application. In providing the user data, the client application may further consider user input received from a user via an input / output interface of the UE 2006. Regardless of the specific manner in which the user data is provided, the UE 2006 initiates transmission of the user data to the host 2002 via the network node 2004 in step 2018. In step 2020, in accordance with the teachings of embodiments described throughout this disclosure, the network node 2004 receives the user data from the UE 2006 and initiates transmission of the received user data to the host 2002. In step 2022, the host 2002 receives the user data carried in a transmission initiated by the UE 2006.

[0185] One or more of the various embodiments improve the performance of an OTT service provided to a UE 2006 using an OTT connection 2050 in which the wireless connection 2070 forms the final leg. The embodiments disclosed herein may improve the management of ML models used for analysis in a communications network (e.g., 5GC). More precisely, embodiments may enable an MTLF to identify the ML model it has retrained when notifying an AnLF, thereby enabling the AnLF to swap the particular ML model it is using for inference. As another example, embodiments may enable an AnLF to express conditions under which the MTLF should retrain an ML model. As another example, embodiments may enable an MTLF to express training performance results so that the MTLF can decide whether to swap ML models. In this manner, embodiments may facilitate improved analysis (particularly based on ML models) within the network, which may result in improved network performance. Furthermore, improved network performance may increase the value of OTT services delivered over the network to both service providers and end users.

[0186] In an exemplary scenario, factory status information may be collected and analyzed by the host 2002. As another example, the host 2002 may process audio and video data retrieved from UEs for use in creating maps. As another example, the host 2002 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 2002 may store surveillance video uploaded by UEs. As another example, the host 2002 may store or control access to media content, such as video, audio, VR, or AR, that can be broadcast, multicast, or unicast to UEs. As another example, the host 2002 may be used for remote control of non-time-critical electrical loads, to balance energy pricing, power generation needs, location services, presentation services (such as compiling diagrams from data collected from remote devices), or any other function that collects, retrieves, stores, analyzes, and / or transmits data.

[0187] In some examples, measurement procedures may be provided for purposes of monitoring data rates, latency, and other factors that one or more embodiments improve. Additionally, there may be optional network functionality for reconfiguring the OTT connection 2050 between the host 2002 and the UE 2006 in response to fluctuations in the measurement results. The measurement procedures and / or network functionality for reconfiguring the OTT connection may be implemented in software or hardware in the host 2002 and / or the UE 2006. According to some embodiments, sensors (not shown) may be deployed in or associated with other devices through which the OTT connection 2050 passes, and the sensors may provide values ​​for the monitored quantities exemplified above or may be implemented in software. Eh By providing values ​​of other physical quantities from which the monitored quantity can be calculated or estimated. Measurement procedureThe host 2002 may participate in the OTT connection 2050. Reconfiguration of the OTT connection 2050 may include message formats, retransmission settings, preferred routing, etc., and the reconfiguration need not directly change the operation of the network node 2004. Such procedures and functionality may be known and practiced in the art. According to some embodiments, the measurements may involve proprietary UE signaling that facilitates measurements by the host 2002 of throughput, propagation time, latency, etc. The measurements may be software-implemented such that messages, particularly empty or "dummy" messages, are sent using the OTT connection 2050 while monitoring propagation times, communication errors, etc.

[0188] As described herein, a device and / or apparatus may be represented by a semiconductor chip, chipset, or (hardware) module comprising such a chip or chipset, which may include: The functionality of the device or apparatus is Instead of being implemented in hardware, it may be implemented as a software module, such as a computer program or computer program product, comprising executable software code portions for execution or running on a processor. , the possibility The present invention does not exclude the use of hardware or software. Furthermore, the functionality of a device or apparatus may be implemented by any combination of hardware and software. A device or apparatus may also be considered an assembly of multiple devices and / or apparatus, whether functionally cooperating with one another or independent of one another. Furthermore, devices and apparatus may be implemented in a distributed manner throughout a system, so long as the functionality of the device or apparatus is maintained. Such and similar principles are believed to be known to those skilled in the art.

[0189] Furthermore, functionality described herein as being performed by a wireless device or network node may be distributed across multiple wireless devices and / or network nodes. In other words, it is contemplated that the functionality of the network nodes and wireless devices described herein is not limited to performance by a single physical device, but may in fact be distributed among several physical devices.

[0190] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which this disclosure belongs. Terms used herein should be interpreted to have a meaning consistent with their meaning in the context of the present specification and related art, and will be further understood not to be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0191] Additionally, certain terms used in this disclosure, including the specification, drawings, and exemplary embodiments thereof, may be used synonymously in certain instances, including, but not limited to, for example, data and information. These and / or other words that may be synonymous with each other may be used synonymously herein, but it should be understood that there may be cases where it is not intended that such words be used synonymously. Furthermore, knowledge of the prior art is expressly incorporated herein in its entirety, unless expressly incorporated above by reference. All referenced publications are incorporated herein by reference in their entirety.

[0192] The foregoing merely illustrates the principles of the present disclosure. Various modifications and alterations to the described embodiments will be apparent to those skilled in the art in light of the teachings herein. It will thus be appreciated that those skilled in the art will be able to devise numerous systems, configurations, and procedures that, although not explicitly shown or described herein, embody the principles of the present disclosure and thus may be within the spirit and scope of the present disclosure. As will be understood by those skilled in the art, the various exemplary embodiments can be used in conjunction with, and interchangeably with, one another.

[0193] Exemplary embodiments of the techniques and apparatus described herein include, but are not limited to, the embodiments listed below:

[0194] A1. A method for a first node of a communication network, said method comprising: receiving a first message from a second node of the communication network, wherein the first message includes one of the following: one or more ML model identifiers corresponding to one or more ML models maintained by the first node; or An identifier for the analysis based on said one or more ML models. and transmitting a second message to the second node, wherein the second message includes one of: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, where each tuple includes a distinct one of a plurality of ML model identifiers and a network address associated with the corresponding ML model; or A single tuple comprising the analysis identifier and a network address associated with a single ML model on which the analysis is based.

[0195] A2. The method of embodiment A1, wherein each network address included in the second message is one of a universal resource locator (URL) or a fully qualified domain name (FQDN).

[0196] A3. The method of any of embodiments A1-A2, wherein each network address included in the second message is one of the following: an address that allows the corresponding ML model to be obtained, or An address that allows retrieving a manifest or metadata file associated with the corresponding ML model.

[0197] A4. The method of embodiment A3, wherein the manifest or the metadata file associated with each of the ML models includes: an ML model identifier; and an identifier of one or more analyses based on the ML model; an updated version of the ML model; and A creation timestamp of an updated version of the ML model; and A network address where the ML model can be retrieved.

[0198] A5. The method of any of embodiments A1-A4, wherein each tuple in the second message also includes an updated version of the corresponding ML model.

[0199] A6. A method according to any of embodiments A1 to A5, further comprising, if the first message includes the analysis identifier, using the analysis identifier to determine one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based.

[0200] A7. A method according to any one of embodiments A1 to A5, wherein if the first message includes one or more of the ML model identifiers, the first message also indicates the version associated with the corresponding ML model.

[0201] A8. The method of embodiment A7, wherein sending the second message includes one of the following: Selectively sending the second message including the single tuple based on the updated version of the single ML model being newer than the version indicated by the first message; or Selectively including each of the plurality of tuples in the second message based on the updated version of the corresponding ML model being newer than the version indicated by the first message.

[0202] A8a. A method according to any of embodiments A1 to A8, wherein the first node is a Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF), and the second node is an Analysis Logic Function (AnLF) of the NWDAF.

[0203] A9. The method of embodiment A8a, wherein the first message is a Nnwdaf_MLModelInfo_Request message and the second message is a response to the Nnwdaf_MLModelInfo_Request message.

[0204] A10. The method of embodiment A8a, wherein the first message is a Nnwdaf_MLModelProvision_Subscribe message and the second message is a Nnwdaf_MLModelProvision_Notify message.

[0205] A11. The method of embodiment A10, wherein the first message also includes one or more conditions for receiving the second message, including, inter alia, one or more of the following: performance metrics for the one or more ML models on which the analysis is based; and Respective thresholds for said performance metrics; a logical relationship between the performance metrics and the thresholds; ML / AI framework constraints for executing the one or more ML models.

[0206] A12. The method of embodiment A11, wherein sending the second message includes one of the following: Selectively transmitting the second message including the single tuple based on the single ML model satisfying the one or more conditions included in the first message; or Selectively including each of the plurality of tuples in the second message based on the corresponding ML model satisfying the one or more conditions included in the first message.

[0207] A13. The method of embodiment A12, wherein each tuple included in the second message further includes: an identifier of a performance metric indicated in the first message; The value of the performance metric determined for the corresponding ML model.

[0208] A14. The method of any one of embodiments A11 to A13, further comprising determining whether to retrain the one or more ML models based on whether current values ​​of the performance metrics for each of the models satisfy respective thresholds and logical relationships.

[0209] B1. A method for a second node of a communication network, the method comprising: sending a first message to a first node of the communication network, wherein the first message includes one of the following: one or more ML model identifiers corresponding to one or more ML models maintained by the first node; or An identifier for the analysis based on said one or more ML models. receiving a second message from the first node, the second message including one of the following: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the plurality of ML model identifiers and a network address associated with the corresponding ML model; or A single tuple comprising the analysis identifier and a network address associated with a single ML model on which the analysis is based.

[0210] B2. The method of embodiment B1, wherein each network address included in the second message is one of a universal resource locator (URL) or a fully qualified domain name (FQDN).

[0211] B3. The method of any of embodiments B1-B2, wherein each network address included in the second message is one of the following: an address allowing the corresponding ML model to be obtained, or An address that allows retrieving a manifest or metadata file associated with the corresponding ML model.

[0212] B4. The method of embodiment B3, wherein the manifest or the metadata file associated with each of the ML models includes: an ML model identifier; and an identifier of one or more analyses based on the ML model; an updated version of the ML model; and A creation timestamp of an updated version of the ML model; and A network address where the ML model can be retrieved.

[0213] B5. The method of any of embodiments B1-B4, wherein each tuple in the second message also includes an updated version of the corresponding ML model.

[0214] B6. A method according to any of embodiments B1 to B5, wherein if the first message includes the one or more ML model identifiers, the first message further indicates the version associated with the corresponding ML model.

[0215] B7. The method of embodiment B6, wherein one or more of the following applies: The second message including the single tuple is received only if the updated version of the single ML model is newer than the version indicated by the first message. Each of the plurality of tuples is received in the second message only if the updated version of the corresponding ML model is newer than the version indicated by the first message.

[0216] B7a. A method according to any of embodiments B1 to B7, wherein the first node is a Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) and the second node is an Analysis Logic Function (AnLF) of the NWDAF.

[0217] B8. The method of embodiment B7a, wherein the first message is a Nnwdaf_MLModelInfo_Request message and the second message is a response to the Nnwdaf_MLModelInfo_Request message.

[0218] B9. The method of embodiment B7a, wherein the first message is a Nnwdaf_MLModelProvision_Subscribe message and the second message is a Nnwdaf_MLModelProvision_Notify message.

[0219] B10. The method of embodiment B9, wherein the first message further includes one or more conditions for receiving the second message, including, inter alia, one or more of the following: performance metrics of the one or more ML models on which the analysis is based; and a threshold for each of said performance metrics; a logical relationship between the performance metrics and the thresholds; ML / AI framework constraints for executing the one or more ML models.

[0220] B11. The method of embodiment B10, wherein one or more of the following applies: The second message containing the single tuple is received only if the single ML model satisfies the one or more conditions contained in the first message. Each of the plurality of tuples is received in the second message only if the corresponding ML model satisfies the one or more conditions included in the first message.

[0221] B12. The method of embodiment B11, wherein each tuple included in the second message also includes: an identifier of a performance metric indicated in the first message; The value of the identified performance metric for the corresponding ML model.

[0222] B13. The method of any one of embodiments B1-B12, further comprising: Obtaining the ML model or the plurality of ML models based on the network addresses in the corresponding tuple or tuples; applying the obtained ML model or models to determine the analysis identified by the analysis identifier; Includes.

[0223] B14. A method according to any of embodiments B1 to B13, further comprising using the analysis identifier to determine one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based, wherein the first message includes the determined one or more ML model identifiers.

[0224] C1. A Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) of a communications network, comprising: the MTLF is implemented by an operably coupled communications interface circuit and processing circuit; The processing circuitry and the communications interface circuitry are configured to perform operations corresponding to any of the methods of embodiments A1-A14.

[0225] C2. A Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) of a communications network, the MTLF configured to perform operations corresponding to the method of any of embodiments A1-A14.

[0226] C3. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by processing circuitry associated with a Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) of a communications network, configure the MTLF to perform operations corresponding to the method of any of embodiments A1-A14.

[0227] C4. A computer program product comprising computer-executable instructions that, when executed by processing circuitry associated with a Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) of a communications network, configure the MTLF to perform operations corresponding to the method of any of embodiments A1-A14.

[0228] D1. An Analysis Logic Function (AnLF) of a Network Data Analysis Function (NWDAF) of a communications network, comprising: the AnLF is implemented by an operably coupled communication interface circuit and processing circuit; The processing circuitry and the communication interface circuitry are configured to perform operations corresponding to any of the methods of embodiments B1 to B12.

[0229] D2. An Analysis Logic Function (AnLF) of a Network Data Analysis Function (NWDAF) of a communications network, the AnLF configured to perform operations corresponding to the method of any of embodiments B1-B12.

[0230] D3. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by processing circuitry associated with an Analysis Logic Function (AnLF) of a Network Data Analysis Function (NWDAF) of a communications network, configure the AnLF to perform operations corresponding to the methods of any of embodiments B1-B12.

[0231] D4. A computer program product comprising computer-executable instructions that, when executed by processing circuitry associated with an Analysis Logic Function (AnLF) of a Network Data Analysis Function (NWDAF) of a communications network, configure the AnLF to perform operations corresponding to the methods of any of embodiments B1-B12.

Claims

1. 1. A method for a first network node or function (NNF) configured for machine learning (ML) model management in a communication network, the method comprising: receiving 1310 a first message from a second NNF of the communication network, the first message comprising: one or more machine learning (ML) model identifiers corresponding to one or more ML models maintained by the first NNF; and an analysis identifier of the one or more ML model-based analyses; and sending a second message to the second NNF (1340), the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including the analysis identifier and one or more information elements associated with a single ML model on which the analysis is based; Each network address included in the second message is an address allowing the corresponding ML model to be obtained; an address allowing for retrieval of a manifest or metadata file associated with the corresponding ML model; It is one of the The manifest or the metadata file associated with each of the ML models an ML model identifier; an identifier of one or more analyses based on said ML model; an updated version of the ML model; a creation timestamp of the updated version of the ML model; and a network address allowing the ML model to be obtained; The method is one of the above.

2. A method for a first network node or function (NNF) configured for machine learning (ML) model management in a communications network, the method comprising: receiving 1310 a first message from a second NNF of the communication network, the first message comprising: one or more machine learning (ML) model identifiers corresponding to one or more ML models maintained by the first NNF; and an analysis identifier of the one or more ML model-based analyses; and sending a second message to the second NNF (1340), the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including the analysis identifier and one or more information elements associated with a single ML model on which the analysis is based; the one or more information elements for each tuple included in the second message further include an updated version of the corresponding ML model; Transmitting the second message (1340) comprises: Selectively sending the second message including the single tuple based on the updated version of the single ML model being newer than the version indicated by the first message (1341); Selectively including each of the plurality of tuples in the second message based on the updated version of the corresponding ML model being newer than the version indicated by the first message (1342); The method includes one of the following:

3. 2. The method of claim 1, further comprising: if the first message includes the analysis identifier, using the analysis identifier to determine (1320) one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based.

4. 2. The method of claim 1, wherein if the first message includes the one or more of the ML model identifiers, the first message also indicates a version associated with the corresponding ML model.

5. A method for a first network node or function (NNF) configured for machine learning (ML) model management in a communications network, the method comprising: receiving 1310 a first message from a second NNF of the communication network, the first message comprising: one or more machine learning (ML) model identifiers corresponding to one or more ML models maintained by the first NNF; and an analysis identifier of the one or more ML model-based analyses; and sending a second message to the second NNF (1340), the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including the analysis identifier and one or more information elements associated with a single ML model on which the analysis is based; The first message includes one or more conditions for receiving the second message, the one or more conditions including: performance metrics of the one or more ML models on which the analysis is based; and a threshold for each of said performance metrics; a logical relationship between the performance metrics and the thresholds; ML / AI framework constraints for executing the one or more ML models; and and Transmitting the second message (1340) comprises: Selectively sending the second message including the single tuple based on the single ML model satisfying the one or more conditions included in the first message (1343); Selectively including each of a plurality of tuples in the second message based on the corresponding ML model satisfying the one or more conditions included in the first message (1344); and The one or more information elements for each tuple included in the second message further comprise: an identifier of a performance metric indicated in the first message; the value of the performance metric determined for the corresponding ML model; and A method comprising:

6. 6. The method of claim 5, further comprising: determining (1330) whether to retrain the one or more ML models based on whether current values ​​of the performance metrics for the corresponding ML models satisfy the corresponding thresholds and logical relationships.

7. 1. A method for a second network node or function (NNF) configured for machine learning (ML) model management in a communications network, the method comprising: Sending a first message to a first NNF of the communication network (1420), the first message comprising: one or more ML model identifiers corresponding to one or more ML models maintained by the first NNF; and an analysis identifier of the one or more ML model-based analyses; and receiving a second message from the first NNF (1430), the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including the analysis identifier and one or more information elements associated with a single ML model on which the analysis is based; Each network address included in the second message is an address allowing the corresponding ML model to be obtained; an address allowing retrieval of a manifest or metadata file associated with the corresponding ML model; and The manifest or the metadata file associated with each of the ML models an ML model identifier; an identifier of one or more analyses based on said ML model; an updated version of the ML model; a creation timestamp of the updated version of the ML model; and a network address allowing the ML model to be obtained; The method is one of the above.

8. 8. The method of claim 7, further comprising: Obtaining the ML model or the plurality of ML models based on a network address in the corresponding tuple or tuples (1440); applying 1450 the obtained ML model or models to determine the analysis identified by the analysis identifier; A method comprising:

9. A method for a second network node or function (NNF) configured for machine learning (ML) model management in a communications network, the method comprising: Sending a first message to a first NNF of the communication network (1420), the first message comprising: one or more ML model identifiers corresponding to one or more ML models maintained by the first NNF; and an analysis identifier of the one or more ML model-based analyses; and receiving a second message from the first NNF (1430), the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including the analysis identifier and one or more information elements associated with a single ML model on which the analysis is based; the one or more information elements for each tuple included in the second message further include an updated version of the corresponding ML model; the second message containing the single tuple is received only if the updated version of the single ML model is newer than the version indicated by the first message; each of the plurality of tuples is received in the second message only if the updated version of the corresponding ML model is newer than the version indicated by the first message; The method according to claim 1, wherein one or more of the following is applied:

10. 8. The method of claim 7, wherein if the first message includes the one or more ML model identifiers, the first message also indicates a version associated with the corresponding ML model.

11. A method for a second network node or function (NNF) configured for machine learning (ML) model management in a communications network, the method comprising: Sending a first message to a first NNF of the communication network (1420), the first message comprising: one or more ML model identifiers corresponding to one or more ML models maintained by the first NNF; and an analysis identifier of the one or more ML model-based analyses; and receiving a second message from the first NNF (1430), the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including the analysis identifier and one or more information elements associated with a single ML model on which the analysis is based; The first message further includes one or more conditions for receiving the second message, the one or more conditions including: performance metrics of the one or more ML models on which the analysis is based; and a threshold for each of said performance metrics; a logical relationship between the performance metrics and the thresholds; ML / AI framework constraints for executing the one or more ML models; and and the second message containing the single tuple is received only if the single ML model satisfies the one or more conditions contained in the first message; each of the plurality of tuples is received in the second message only if the corresponding ML model satisfies the one or more conditions included in the first message; apply, The one or more information elements for each tuple included in the second message further comprise: an identifier of a performance metric indicated in the first message; the value of the performance metric determined for the corresponding ML model; and A method comprising:

12. 8. The method of claim 7, further comprising: using the analysis identifier to determine (1410) one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based; and wherein the first message includes the determined one or more ML model identifiers.

13. A first network node or function (NNF) (810, 1508, 1700, 1902) configured for machine learning (ML) model management in a communication network (199, 200, 1506), the first NNF configured to perform the method of any one of claims 1 to 6.

14. A computer program (1704a, 1904a) that, when executed by a processing circuit (1702, 1904) associated with a first network node or function (NNF) (810, 1508, 1700, 1902) configured for machine learning (ML) model management in a communications network (199, 200, 1506), configures the first NNF to perform operations corresponding to the method of any of claims 1 to 6.

15. A second network node or function (NNF) (820, 1508, 1700, 1902) configured for machine learning (ML) model management in a communication network (199, 200, 1506), the second NNF configured to perform the method of any one of claims 7 to 12.

16. A computer program (1704a, 1904a) that, when executed by a processing circuit (1702, 1904) associated with a second network node or function (NNF) (820, 1508, 1700, 1902) configured for machine learning (ML) model management in a communications network (199, 200, 1506), configures the second NNF to perform operations corresponding to the method of any of claims 7 to 12.

Images