Management of Machine Learning Models in 5G Core Networks Background of the Invention

Abstract

An embodiment includes a method for a first network node or function (NNF) configured for machine learning (ML) model management in a communication network. The method includes receiving a first message from a second NNF of the communication network, the first message including one or more ML model identifiers corresponding to one or more ML models maintained by the first node and an identifier of an analysis based on the ML models. The method includes sending a second message to the second NNF including one of a plurality of tuples corresponding to the plurality of ML models on which the analysis is based, each tuple including a different ML model identifier and an information element associated with the corresponding ML model, or a single tuple including an analysis identifier and an information element associated with a single ML model on which the analysis is based. Other embodiments include a complementary method for the second NNF.

Description

[Technical field]

[0001] The present application relates generally to the field of communication networks, and more particularly to techniques for managing machine learning (ML) used in 5G Core (5GC) networks and generating analytical results in communication networks based on such models. [Background technology]

[0002] At a high level, the 5G system (5GS) consists of an access network (AN) and a core network (CN). The AN provides access to the CN, for example via a base station, such as a gNB or ng-eNB, as described below. Ko Connectivity To UE The CN contains various Network Functions (NFs) that provide a wide range of different functions, such as session management, connection management, charging, and authentication.

[0003] FIG. 1 illustrates a high-level view of an exemplary 5G network architecture consisting of a Next Generation Radio Access Network (NG-RAN) 199 and a 5G Core (5GC) 198. The NG-RAN 199 may include one or more gNodeBs (gNBs) connected to the 5GC via one or more NG interfaces, such as gNBs 100, 150 connected via interfaces 102, 152, respectively. More specifically, the gNBs 100, 150 may be connected to one or more Access and Mobility Management Functions (AMFs) in the 5GC 198 via respective NG-C interfaces. Similarly, the gNBs 100, 150 may be connected to one or more User Plane Functions (UPFs) in the 5GC 198 via respective NG-U interfaces. Various other Network Functions (NFs) may be included in the 5GC 198, as described in more detail below.

[0004] Further, the multiple gNBs may be connected to each other via one or more Xn interfaces, such as Xn interface 140 between gNB 100 and gNB 150. The radio technology for NG-RAN is often referred to as "New Radio" (NR). With respect to the NR interface to the UE, each of the gNBs may support frequency division duplex (FDD), time division duplex (TDD), or a combination thereof. Each of the multiple gNBs may provide service in a geographic coverage area that includes one or more cells, and in some cases may use various directional beams to provide coverage in the respective cells.

[0005] The NG-RAN 199 is layered into a Radio Network Layer (RNL) and a Transport Network Layer (TNL). The NG-RAN architecture, i.e., the NG-RAN logical nodes and the interfaces between them, are specified as part of the RNL. For each NG-RAN interface (NG, Xn, F1), the associated TNL protocols and functions are specified. The TNL provides services for user plane transport and signaling transport. In some example configurations, the gNBs are each connected to all 5GC nodes within an "AMF domain," and the term "AMF" refers to the Access and Mobility Management Function in 5GC.

[0006] The NG RAN logical node shown in FIG. 1 includes a central unit (CU or gNB-CU) and one or more distributed units (DU or gNB-DU). For example, gNB 100 includes gNB-CU 110 and gNB-DU 120 and 130. The CU (e.g., gNB-CU 110) is a logical node that hosts higher layer protocols and performs various gNB functions such as controlling the operation of the DU. The DU (e.g., gNB-DU 120, 130) is a distributed logical node that hosts lower layer protocols and can include various subsets of gNB functions depending on the function partitioning option. Thus, each of the CU and DU may include various circuits, including processing circuits, transceiver (e.g., for communication) circuits, and power circuits, necessary to perform its functions.

[0007] A gNB-CU connects to one or more gNB-DUs via respective F1 logical interfaces, such as interfaces 122 and 132 shown in Figure 1, but a gNB-DU can only connect to a single gNB-CU. The gNB-CU and connected gNB-DU can only be seen as a gNB to other gNBs and 5GCs, i.e., the F1 interface beyond the gNB-CU is not recognized (is invisible).

[0008] Another change in 5G networks (e.g., in 5GC) is that traditional peer-to-peer interfaces and protocols found in previous generation networks are modified and / or replaced by a service-based architecture (SBA) in which a network function (NF) provides one or more services to one or more service consumers. This can be done, for example, by HTTP / REST (HyperText Transfer Protocol / Representational State Transfer) Application Programming Interfaces (APIs). In general, the various services are self-contained functions and can be changed and modified in an isolated manner without affecting other services.

[0009] Furthermore, a service is composed of various "service operations", which are finer divisions of the overall service functionality. The type of interaction between service consumers and producers can be "request / response" or "subscribe / notify". In 5G SBA, a Network Repository Function (NRF) enables every network function to discover services offered by other network functions, and a Data Storage Function (DSF) enables every network function to remember its context. This 5G SBA model is based on principles including modularity, reusability, and self-sufficiency of NFs, which allows network deployment to leverage the latest virtualization and software technologies. of Interest Score a point This makes it possible to

[0010] A 5GC NF of particular interest in this disclosure is the Network Data Analysis Function (NWDAF), which provides network analysis information (e.g., statistics and / or forecast information of past events) to other NFs.

[0011] Machine learning (ML) is a type of artificial intelligence (AI) that focuses on using data and algorithms to mimic the way humans learn, gradually improving its accuracy. ML algorithms build models based on example (or "training") data, which are then used to make predictions or decisions. ML algorithms can be used in a wide variety of applications (e.g., healthcare, email filtering, speech recognition, etc.) where it is difficult or not feasible to develop traditional algorithms to perform the required tasks.

[0012] According to 3GPP TS 23.288 (v17.2.0), the NWDAF is the main network function for computing analytics reports and specifies the classification of the NWDAF into two sub-functions (or logical functions): the NWDAF Analytics Logical Function (NWDAFAnLF), which performs the analytics procedures, and the NWDAF Model Training Logical Function (NWDAFMTLF), which performs the training and retraining of ML models used by the NWDAFAnLF. The services used by these NWDAF logical functions work on the implicit assumption of a one-to-one relationship between ML models and analytics identifiers (IDs), also called "NwdafEvents". In this context, the Analytics IDs are used to identify the types of analytics that the NWDAF can generate or that a consumer of analytics results is interested in receiving. Summary of the Invention

[0013] However, in certain scenarios, multiple ML models may be used to generate the analytics reports provided to a particular Analytics ID. For example, different ML models may be used to generate these analytics reports depending on location, time of day, network load, or any other information filtering parameters. As another example, these analytics reports may be generated by several cascaded ML models. These ambiguities may cause various issues, challenges, and / or difficulties in managing ML models and the analytics they generate.

[0014] Embodiments of the present disclosure address these and other issues, challenges, and / or difficulties, thereby facilitating otherwise advantageous deployment of ML models for analysis in 5G networks.

[0015] Some embodiments include an example method (e.g., a procedure) for a first network node or function (NNF) configured for ML model management in a communications network (e.g., 5GC).

[0016] These example methods can include receiving, from a second NNF of the communication network, a first message including one of the following: ● one or more ML model identifiers corresponding to one or more ML models maintained by the first node; or ●If the ML model identifier is not included, an identifier of the analysis based on one or more ML models.

[0017] These example methods may also include sending a second message to the second NNF that includes one of the following: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a distinct one of a plurality of ML model identifiers and one or more information elements associated with the corresponding ML model; or ● A single tuple containing an analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

[0018] According to some embodiments, for each tuple included in the second message, the one or more information elements include a network address that is one of a Universal Resource Locator (URL) or a Fully Qualified Domain Name (FQDN). According to some embodiments, each network address included in the second message is one of an address that allows for obtaining the corresponding ML model, or an address that allows for obtaining a manifest or metadata file associated with the corresponding ML model. According to some of these embodiments, the manifest or metadata file associated with each of the ML models includes: ML model identifier Identifier of one or more analyses based on the ML model An updated version of the ML model, where "updated" refers to the latest version of the ML model that has been trained and validated by the MTLF and therefore may be used for inference by AnLF. The creation timestamp of the updated version of the ML model. A network address that allows retrieving the ML model.

[0019] According to some embodiments for each tuple included in the second message, the one or more information elements also include an updated version of the corresponding ML model. According to some embodiments, if the first message includes an analysis identifier, these example methods may also include using the (received) analysis identifier to determine one or more ML model identifiers that correspond to the one or more ML models on which the analysis is based.

[0020] According to other embodiments, if the first message includes one or more of the ML model identifiers, the first message also indicates a version associated with the corresponding ML model. According to some of these embodiments, sending the second message includes one of the following: selectively sending a second message including a single tuple based on the updated version of the single ML model being newer than the version indicated by the first message; or ● Selectively including each of the plurality of tuples in a second message based on an updated version of the corresponding ML model being newer than the version indicated by the first message.

[0021] According to some embodiments, the first NNF is a MTLF of the NWDAF and the second NNF is an AnLF of the NWDAF. According to some of these embodiments, the first message is a Nnwdaf_MLModelInfo_Request message and the second message is a response to the Nnwdaf_MLModelInfo_Request message.

[0022] According to other embodiments, the first message is a Nnwdaf_MLModelProvision_Subscribe message and the second message is a Nnwdaf_MLModelProvision_Notify message. According to some of these embodiments, the first message also includes one or more conditions that must be met in order to receive the second message, including one or more of the following: The performance metrics (or identifiers) of one or more ML models on which the analysis is based - Thresholds for each performance metric ● The logical relationship between performance metrics and thresholds, and ●ML / AI framework constraints for running one or more ML models.

[0023] According to some variations, these example methods may also include determining whether to retrain one or more ML models based on whether current values ​​of the performance metrics of the respective models satisfy respective thresholds and logical relationships (e.g., as indicated in the first message). According to some variations, sending the second message includes one of the following: selectively sending a second message that includes a single tuple based on a single ML model that satisfies one or more conditions included in the first message; or ● Selectively including each of the plurality of tuples in the second message based on a corresponding ML model that satisfies one or more conditions included in the first message.

[0024] According to some further variations, each tuple included in the second message also includes an identifier of a performance metric indicated in the first message and a value of the identified performance metric for the corresponding ML model.

[0025] Other embodiments include a method (e.g., a procedure) for a second NNF configured for ML model management in a communications network (e.g., 5GC).

[0026] These example methods may include transmitting, to a first NNF of the communication network, a first message including one of the following: ● one or more ML model identifiers corresponding to one or more ML models maintained by the first node; or ●If no ML model identifier is included, an identifier of the analysis based on one or more ML models.

[0027] These example methods may also include receiving, from the first node, a second message including one of the following: A plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a distinct one of the ML model identifiers and one or more information elements associated with the corresponding ML model; or ● A single tuple containing an analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

[0028] According to some embodiments, for each tuple included in the second message, the one or more information elements include a network address that is a URL or a FQDN. According to some embodiments, each network address included in the second message is one of an address that allows to obtain a corresponding ML model or an address that allows to obtain a manifest or a metadata file associated with the corresponding ML model.

[0029] According to some of these embodiments, the manifest or metadata file associated with each of the ML models includes: ML model identifier Identifier of one or more analyses based on the ML model ● Updated version of the ML model The creation timestamp of the updated version of the ML model. A network address that allows retrieving the ML model.

[0030] According to other of these embodiments, these example methods may also include the following operations: obtaining a single ML model or multiple ML models based on a network address in a corresponding single tuple or multiple tuples (e.g., included in the second message); and applying the obtained single ML model or multiple ML models to determine an analysis identified by the analysis identifier.

[0031] According to some embodiments, for each tuple included in the second message, the one or more information elements also include an updated version of the corresponding ML model. According to some embodiments, if the first message includes one or more of the ML model identifiers, the first message also indicates the associated version of the corresponding ML model. In some of these embodiments, one or more of the following apply: A second message containing a single tuple is received only if the updated version of the single ML model is newer than the version indicated by the first message. Each of the multiple tuples is received in the second message only if the updated version of the corresponding ML model is newer than the version indicated by the first message.

[0032] According to some embodiments, the first NNF is a MTLF of the NWDAF and the second NNF is an AnLF of the NWDAF. According to some of these embodiments, the first message is a Nnwdaf_MLModelInfo_Request message and the second message is a response to the Nnwdaf_MLModelInfo_Request message.

[0033] According to these other embodiments, the first message is a Nnwdaf_MLModel-Provision_Subscribe message and the second message is a Nnwdaf_MLModelProvision_Notify message. According to some of these embodiments, the first message also includes one or more conditions for receiving the second message, including one or more of the following: The performance metrics (or identifiers) of one or more ML models on which the analysis is based - Thresholds for each performance metric ● The logical relationship between performance metrics and thresholds, and ●ML / AI framework constraints for running one or more ML models.

[0034] According to some variants, one or more of the following apply: A second message containing a single tuple is received only if the single ML model satisfies one or more conditions contained in the first message. ● Each of the multiple tuples is received in the second message only if the corresponding ML model satisfies one or more conditions included in the first message.

[0035] According to some further variants, or in each tuple included in the second message, the one or more information elements also include an identifier of the performance metric indicated in the first message and a value of the identified performance metric for the corresponding ML model.

[0036] According to some embodiments, these example methods may also include using the analysis identifier to determine one or more ML model identifiers that correspond to one or more ML models on which the analysis is based. In such cases, the first message includes the determined one or more ML model identifiers.

[0037] Other embodiments include first and second NNFs (e.g., the MTLF and AnLF, or network nodes hosting them) configured to perform operations corresponding to any of the example methods described herein. Other embodiments also include a non-transitory computer-readable medium storing computer-executable instructions that, when executed by processing circuitry associated with such NNFs, configure the NNFs to perform operations corresponding to any of the example methods described herein.

[0038] These and other disclosed embodiments can enable the MTLF to identify the ML models it has retrained when notifying the AnLF, thereby enabling the AnLF to swap the particular ML model it is using for inference. As another example, embodiments can facilitate the AnLF to express conditions under which the MTLF should retrain an ML model. As another example, embodiments can facilitate the MTLF to express training performance results so that the MTLF can decide whether to swap ML models. At a high level, embodiments improve management of ML models used for analysis in communication networks (e.g., 5GC).

[0039] These and other objects, features and advantages of the present disclosure will become apparent from a reading of the following detailed description in consideration of the drawings briefly described below. [Brief description of the drawings]

[0040] [Figure 1] FIG. 1 illustrates various aspects of an exemplary 5G network architecture. [Diagram 2] FIG. 1 illustrates various aspects of an exemplary 5G network architecture.

[0041] [Diagram 3]1 illustrates an example procedure for an NWDAF service consumer to subscribe to notifications about the availability of ML models from the NWDAF (MTLF).

[0042] [Figure 4] 1 illustrates an example procedure of an NWDAF service consumer for obtaining information about an ML model from the NWDAF (MTLF).

[0043] [Diagram 5] gives various example scenarios where one Analytics ID can be processed by multiple ML models. [Figure 6] gives various example scenarios where one Analytics ID can be processed by multiple ML models. [Figure 7] gives various example scenarios where one Analytics ID can be processed by multiple ML models.

[0044] [Figure 8] 1 shows a block diagram of an NWDAF according to various embodiments of the present disclosure.

[0045] [Figure 9] 1 illustrates a data structure of an example Nnwdaf_MLModelProvision_Subscribe request message in accordance with various embodiments of the present disclosure.

[0046] [Figure 10] 1 illustrates a data structure for an example Nnwdaf_MLModelProvision_Notify request message, according to various embodiments of the present disclosure.

[0047] [Figure 11] 1 illustrates a data structure of an example Nnwdaf_MLModelInfo_Request request message in accordance with various embodiments of the present disclosure.

[0048] [Figure 12] 1 illustrates a data structure of an example Nnwdaf_MLModelInfo_Request response message in accordance with various embodiments of the present disclosure.

[0049] [Figure 13] 1 illustrates an example method (e.g., a procedure) for a first node of a communication network, in accordance with various embodiments of the present disclosure.

[0050] [Figure 14] 1 illustrates an example method (e.g., a procedure) for a second node in a communication network, in accordance with various embodiments of the present disclosure.

[0051] [Figure 15] 1 illustrates a communication system according to various embodiments of the present disclosure.

[0052] [Figure 16] 1 illustrates a UE in accordance with various embodiments of the present disclosure.

[0053] [Figure 17] 1 illustrates a network node according to various embodiments of the present disclosure.

[0054] [Figure 18] 1 illustrates a host computing system according to various embodiments of the present disclosure.

[0055] [Figure 19] 1 is a block diagram of a virtualization environment in which functions implemented by some embodiments of the present disclosure may be virtualized.

[0056] [Figure 20] 1 illustrates communication between a host computing system, a network node, and a terminal via multiple connections, according to various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0057] The above-mentioned embodiments will be described in more detail below with reference to the accompanying drawings. These descriptions are provided as examples to explain the subject matter to those skilled in the art, and should not be construed as limiting the scope of the subject matter only to the embodiments described herein. More specifically, examples illustrating the operation of various embodiments according to the above-mentioned advantages are provided below.

[0058] In general, all terms used herein should be interpreted according to their ordinary meaning in the relevant technical field unless a different meaning is expressly given and / or implied from the context in which it is used. All references to a / an / the+ element, apparatus, component, means, step, etc. should be openly interpreted as referring to at least one instance of the element, apparatus, component, means, step, etc., unless otherwise specified. The steps of any method and / or procedure disclosed herein do not have to be performed in the order disclosed, unless a step is expressly described as following or preceding another step and / or it is implied that a step must follow or precede another step. Any feature of any embodiment disclosed herein may be applied to any other embodiment, as appropriate. Similarly, the advantages of any embodiment may be applied to other embodiments, and vice versa. Other objects, features, and advantages of the disclosed embodiments will become apparent from the following description.

[0059] Furthermore, the following terminology is used in the following description: Core network node: As used herein, a "core network node" is any type of node in a core network. Some examples of a core network node include, for example, a mobility management entity (MME), a serving gateway (SGW), a packet data network gateway (P-GW), etc. A core network node may also be a node that implements a specific core network function (NF), such as an access and mobility management function (AMF), a session management function (SMF), a user plane function (UPF), a service capability exposure function (SCEF), etc. ● Network Node: As used herein, a "network node" is any node that is part of a core network (e.g., the core network node discussed above) of a telecommunications network. Functionally, a network node is equipment that is capable, configured, arranged, and / or operable to communicate, directly or indirectly, with wireless or wired devices and / or with other network nodes or equipment in the telecommunications network, enable and / or provide wireless or wired access to telecommunications devices, and / or perform other functions (e.g., management) in the telecommunications network. ● Node: As used herein, the term “node” (without any prefix) may be any type of node capable of operating in or with a telecommunications network (including RAN and / or core network), including a radio access node (or its equivalent terminology), a core network node, or a telecommunications device. ● Service: As used herein, the term "service" generally refers to a set of data associated with one or more applications to be transferred over a network, with specific delivery requirements that must be met for the application to be successful. ● Component: As used herein, the term "component" generally refers to any component required for the delivery of a service. Examples of components are the RAN (e.g., E-UTRAN, NG-RAN, or parts thereof such as eNB, gNB, base station (BS)), the CN (e.g., EPC, 5GC, or parts thereof including all types of links between the RAN and CN entities), and a cloud infrastructure with associated resources such as computing, storage, etc. In general, each component may have a "manager," which is an entity that can collect historical information about resource utilization and provide information about the current and predicted future availability of resources associated with that component (e.g., the RAN manager).

[0060] It should be noted that the description given herein focuses on 3GPP® telecommunication systems, and therefore 3GPP® terminology or terminology similar to 3GPP® terminology is generally used. However, the concepts disclosed herein are not limited to 3GPP® systems. Other wireless systems, including but not limited to Wideband Code Division Multiple Access (WCDMA®), Worldwide Interoperability for Microwave Access (WiMax), Ultra Mobile Broadband for Mobile Communications (UMB), and Global System for Mobile Communications (GSM), may also benefit from the concepts, principles, and / or embodiments described herein.

[0061] In addition, functions and / or operations described herein as being performed by a telecommunications device or network node may be distributed across multiple telecommunications devices and / or network nodes.

[0062] 2 illustrates an example non-roaming 5G reference architecture with service-based interfaces and various NFs defined by 3GPP in the control plane (CP), including the following NFs, the most relevant to this disclosure of which will be described in more detail: ● Application Functions (AF, with Naf interface) interact with the 5GC to provide information to the network operator and subscribe to certain events occurring in the operator's network. The AF provides applications where services are delivered at a different layer (i.e. transport layer) than the layer where the service was requested (i.e. signaling layer) and the control of flow resources is according to what has been negotiated with the network. The AF communicates dynamic session information to the PCF (via the N5 interface), including a description of the media delivered by the transport layer. ● The Policy Control Function (PCF, with Npcf interface) supports a unified policy framework that governs the network behavior by providing PCC rules (e.g. regarding the treatment of each service data flow under PCC control) to the SMF over the N7 reference point. The PCF provides policy control decisions and flow-based charging control, including service data flow discovery, gating control, QoS control, and flow-based charging (excluding credit management) to the SMF. The PCF receives session and media related information from the AF and notifies the AF of traffic (or user) plane events. The User Plane Function (UPF) supports the processing of user plane traffic based on rules received from the SMF, including packet inspection and various enforcement actions such as event detection and reporting. The UPF communicates with the RAN (e.g., NG-RAN) via the N3 reference point, with the SMF (described later) via the N4 reference point, and with the external Packet Data Network (PDN) via the N6 reference point. The N9 reference point is for communication between two UPFs. The Session Management Function (SMF, with Nsmf interface) interacts with the separated traffic (or user) plane, including creating, updating and removing Protocol Data Unit (PDU) sessions and managing session context with the User Plane Function (UPF), for example for event reporting. For example, the SMF performs data flow detection (based on filter definitions contained in PCC rules), online and offline charging interactions, and policy enforcement. The Charging Function (CHF, with Nchf interface) is responsible for centralized online and offline charging functionality. It provides quota management (for online charging), re-authorization triggers, rating conditions etc. and is informed about usage reports from the SMF. Quota management involves allowing a certain number of units (e.g. bytes, seconds) for a service. The CHF also interacts with the billing system. ● The Access and Mobility Management Function (AMF, with Namf interface) terminates the RAN CP interface and handles all mobility and connection management for the UE (similar to MME in EPC). The AMF communicates with the UE via the N1 reference point and with the RAN (e.g. NG-RAN) via the N2 reference point. Network Exposure Function (NEF) with Nnef interface - acts as an entry point into the operator's network by securely exposing network capabilities and events offered by 3GPP® NFs to the AF and by providing a way for the AF to securely provide information to the 3GPP® network. For example, the NEF provides a service that allows the AF to provision specific subscription data for various UEs (e.g. expected UE behavior). ● Network Repository Function (NRF) with Nnrf interface - provides service registration and discovery, allowing NFs to identify suitable services available from other NFs. ● Network Slice Selection Function (NSSF) with Nnssf interface - A "network slice" is a logical partition of a 5G network that provides specific network capabilities and characteristics, for example in support of a specific service. A network slice instance is a set of NF instances and the necessary network resources (e.g., compute, storage, communication) that provide the capabilities and characteristics of the network slice. The NSSF enables other NFs (e.g., AMF) to identify a network slice instance appropriate for the service desired by the UE. Authentication server with Nausf interface machine AUSF - User of Based on the home network (HPLMN), the user Approval It performs authentication and computes security keying material for a variety of purposes. • The Network Data Analysis Facility (NWDAF) with Nnwdaf interface is described in more detail above and below. ● A Location Management Function (LMF) with an Nlmf interface supports various functions related to determining the UE location, including determining the location of the UE and obtaining either DL location measurements or location estimates from the UE, UL location measurements from the NG RAN, and non-UE related assistance data from the NG RAN.

[0063] The Unified Data Management (UDM) function supports the generation of 3GPP authentication credentials, user identification processing, access authorization based on subscription data, and other subscriber-related functions. To provide this functionality, the UDM uses subscription data (including authentication data) stored in the 5GC Unified Data Repository (UDR). In addition to the UDM, the UDR supports the storage and retrieval of policy data by the PCF and the storage and retrieval of application data by the NEF.

[0064] The NRF allows all NFs to discover services offered by other NFs, and the Data Storage Function (DSF) allows all NFs to store their context. Additionally, the NEF exposes 5GC capabilities and events to AFs within and outside the 5GC. For example, the NEF provides a service that allows the AF to provision specific subscription data (e.g., expected UE behavior) for various UEs.

[0065] The communication link between the UE and the 5G network (AN and CN) can be grouped into two different hierarchical layers. The UE communicates with the CN via the Non-Access Stratum (NAS) and with the AN via the Access Stratum (AS). All NAS communication takes place between the UE and the AMF via the NAS protocol (N1 interface in Figure 2). Security for communication over this layer is provided by the NAS protocol (for NAS) and the PDCP protocol (for AS).

[0066] 3GPP Release 17 enhances SBA by adding a Data Management Framework including Data Collection Coordination Function (DCCF) and Messaging Framework Adapter Function (MFAF) detailed in 3GPP TR 23700-91 (v17.0.0). The Data Management Framework is backward compatible with the NWDAF functions of Release 16 mentioned above. For Release 17, the baseline of the services provided by the DCCF (e.g., to the NWDAF analysis function) are the Release 16 NF services used to obtain data. For example, the baseline DCCF service used by the NWDAF consumer to obtain UE mobility data is Namf_EventExposure.

[0067] As mentioned above, 3GPP TS 23.288 (v17.2.0) specifies that the NWDAF is the primary network function for computing analysis reports. The 5G system architecture allows any NF to obtain analysis from the NWDAF using the DCCF function and associated DCCF services. The NWDAF can also store and retrieve analysis information from the Analysis Data Repository Function (ADRF).

[0068] 3GPP® TS 23.288 also categorizes the NWDAF into two sub-functions (or logical functions): the NWDAF Analysis Logical Function (NWDAFAnLF), which performs analysis procedures, and the NWDAF Model Training Logical Function (NWDAFMTLF), which performs training and retraining of ML models used by the NWDAFAnLF.

[0069] 3GPP TS 23.288 specifies a subscribe / notify procedure for a consumer NF to retrieve ML models associated with one or more analysis IDs whenever a new ML model is trained and available by the NWDAFMTLF. This is called ML model provisioning and is implemented by the Nnwdaf_MLModelProvision service.

[0070] 3 illustrates an example procedure for an NWDAF service consumer (e.g., NWDAFAnLF) to subscribe to notifications about ML model availability from the NWDAF (MTLF). As described above, the procedure is implemented based on the Nnwdaf_MLModel-Provision_Subscribe and Nnwdaf_MLModelProvision_Notify messages (part of the Nnwdaf_MLModelProvision service). 3GPP TS 23.288 Section 6.2A describes the procedure in more detail.

[0071] 3GPP TS 23.288 also specifies a request / response procedure for a consumer NF (e.g., NWDAFAnLF) to retrieve information about ML models associated with one or more analysis IDs. This procedure is implemented by the Nnwdaf_MLModelInfo service and is shown in FIG. 4.

[0072] As mentioned above, these services used by the two NWDAF logical functions operate on the implicit assumption of a one-to-one relationship between ML models and Analytics IDs. However, in certain scenarios, multiple ML models can be used to generate the analytics reports provided to a particular Analytics ID. For example, these analytics reports can be generated using different ML models depending on location, time of day, network load, or any other information filtering parameters. As another example, these analytics reports can be generated by several cascaded ML models.

[0073] FIG. 5 illustrates one example scenario where one Analytics ID may be provided by several ML models. In this scenario, three different ML models (A-C) generate their respective predictions (A-C) based on the same inputs (a-c). The aggregation / voting function aggregates the predictions to provide a system output Generate.

[0074] FIG. 6 illustrates another example scenario in which one Analytics ID may be provided by several ML models. In this situation, three different ML models (A-C) receive system inputs (a-c), and an ML model (B) provides two predictions (B-C) that are inputs to two other ML models (A, C), which provide the system outputs (A-B), respectively. In some scenarios, one or both of the ML models (A, C) may be used depending on various conditions.

[0075] FIG. 7 illustrates another example scenario in which one Analytics ID may be provided by several ML models. In this situation, the system includes three different ML models (A-C), each receiving different inputs and each generating different predictions than the other two ML models. However, only one of the ML models is used at any given time, based on various conditions related to the respective inputs. Thus, the system output at a particular time depends on which ML model is being used.

[0076] In the case of the Nnwdaf_MLModelProvision service described above, if the NWDAF(MTLF) retrains one of the ML models associated with an Analytics ID and this ML model does not have a one-to-one relationship with an AnalyticsFilter, the notification is not meaningful enough for the NWDAF(AnLF). In other words, the subscription request does not include an Analytics Filter (only an Analytics ID), so the NWDAF(MTLF) returns the location of only the retrained ML model (out of multiple ML models). That information alone is not sufficient for the NWDAF(AnLF) to determine which ML model it has to swap.

[0077] A similar problem arises for the Nnwdaf_MLModelInfo service mentioned above: the standard does not detail how to specify the particular ML models associated with the meta-information carried by the service response. Rather, the service consumer is expected to specify the ML models that it is interested in, which is not possible as mentioned above.

[0078] Additionally, the response to the Nnwdaf_MLModelInfo request includes ML model metadata, but the actual content is not specified. NWDAF (AnLF) typically requires additional information to execute the ML model for inference. Such information may include ML model dependencies, supported execution platforms, instructions to execute additional software components that may be required to execute the ML model, etc. Because AI / ML is a constantly evolving area with new platforms and new technologies emerging, it is not possible to standardize all existing options up front.

[0079] Accordingly, embodiments of the present disclosure address these and other problems, challenges, and / or difficulties by providing improved techniques for provisioning ML models by introducing explicit references to ML models, conditions for training ML models in the NWDAF (MTLF), and / or conditions for swapping ML models in the NWDAF (AnLF). These techniques provide various benefits and / or advantages. For example, they facilitate the NWDAF (MTLF) to identify the ML models it has retrained when notifying the NWDAF (AnLF), thereby enabling the NWDAF (AnLF) to swap the particular ML models it is using for inference. As another example, they facilitate the NWDAF (AnLF) to express conditions under which the NWDAF (MTLF) should seek to retrain ML models and whether it must notify the NWDAF (AnLF) upon retraining an ML model. As another example, they facilitate expressing training performance results so that the NWDAF (MTLF) can decide whether to swap ML models. At a high level, embodiments improve the management of ML models used for analysis in communications networks (e.g., 5GC).

[0080] The abbreviations "MTLF", "NWDAF(MTLF)", and "MTLF of the NWDAF" are used interchangeably herein. Similarly, the abbreviations "AnLF", "NWDAF(AnLF)", and "AnLF of the NWDAF" are used interchangeably herein.

[0081] FIG. 8 illustrates a block diagram of an NWDAF according to various embodiments of the present disclosure. The NWDAF illustrated in FIG. 8 includes an AnLF and an MTLF that interact through Nnwdaf_MLModelProvision and Nnwdaf_MLModelInfo services. The embodiment maintains the existing AnLF / MTLF functional division, but updates the service operations in the Nnwdaf_MLModelProvision and Nnwdaf_MLModelInfo services. Additionally, the MTLF functionality is improved such that upon receiving a Nnwdaf_MLModelProvision_Subscribe request, the MTLF takes into account conditions in the existing subscription before training a new ML model.

[0082] According to various embodiments, a new ML model identifier information element is added to the Nnwdaf_MLModelProvision_Subscribe request and the Nnwdaf_MLModelInfo_Request. This ML model identifier can replace the Analytics ID in the request and can be used to filter the request. Several instances of the ML model identifier can be included in the request, enabling requests for several ML models that affect the same Analytics ID. According to some embodiments, a consumer (e.g., AnLF) can include a list of its supported capabilities as an additional filtering option in the Nnwdaf_MLModel-Provision_Subscribe request, so that the consumer receives notifications only for retrained ML models that match the indicated capabilities.

[0083] According to various embodiments, the Nnwdaf_MLModelProvision_Notify message can retain its existing structure to carry a set of tuples. Each tuple can include the following information: ┌────────────┬───────────┬────────────┐ │IE name │Existence │Existing / New / Modify │ ├────────────┼───────────┼────────────┤ │Analytics ID│Options(ML model│Existing │ │(Analysis ID) │No identifier) ​​│ │ ├────────────┼───────────┼────────────┤ │ML Model Identifier │Options │New │ │ │ │ │ ├────────────┼───────────┼────────────┤ │ML Model Version │Options │New │ │ │ │ │ ├────────────┼───────────┼────────────┤ │ML Model Manifest │Options │Modifications │ │ or artifacts │ │ │ │URL for │ │ │ └────────────┴───────────┴────────────┘

[0084] According to various embodiments, the Nnwdaf_MLModelInfo_Request response message can retain its existing structure for carrying a set of tuples. Each tuple can include the following information: ┌────────────┬───────────┬────────────┐ │IE name │Existence │Existing / New / Modify │ ├────────────┼───────────┼────────────┤ │Analytics ID│Options(ML model│Existing │ │(Analysis ID) │No identifier) ​​│ │ ├────────────┼───────────┼────────────┤ │ML Model Identifier │Options │New │ │ │ │ │ ├────────────┼───────────┼────────────┤ │ML Model Version │Options │New │ │ │ │ │ ├────────────┼───────────┼────────────┤ │URL for ML metadata │Optional │Existing │ │ │ │ │ └────────────┴───────────┴────────────┘

[0085] ML model metadata (also called a "manifest") can include information about the version, location, capabilities, requirements, etc.

[0086] Various embodiments will now be described in more detail, first with respect to the Nnwdaf_MLModelProvision service, and then with respect to the Nnwdaf_MLModelInfo service.

[0087] 3GPP TS 23.288 section 7.5.2 specifies the following data to be included in the Nnwdaf_MLModelProvision_Subscribe request: ● Input, required: (Set of) Analytics IDs defined in Table 7.1-2, "Notification Target Address (+ Notification Correlation ID)". ● Input, optional: Subscription correlation ID (in case of amending an ML model subscription), analysis filter information indicating the conditions that the ML model needs to be analyzed for, and the target of the analysis report indicating the objects that the ML model needs to be analyzed for, an entity such as a specific UE, a group of UEs, or any UE (all UEs), the target period for the ML model, and the expiry date.

[0088] Various embodiments of the present disclosure provide one or more of the following improvements to the existing Nnwdaf_MLModelProvision_Subscribe request: ●Make the Analytics ID(s) optional (i.e., if the ML model identifier is included, it will not be included). Include (a set of) ML model identifiers as an optional parameter. The request must contain either an Analytics ID or a model identifier. If only an Analytics ID is included, this will be used as a filter to select the ML models referenced by the notification. If an ML model identifier is included, the ML model version may also be included. The NWDAF (AnLF), as a consumer, can subscribe to receive notifications about new ML model versions after the first version has been deployed. By including the ML model version in the request, the consumer states that it only wants to be notified if the version is greater (newer) than the version identified by the ML model version. Additional conditions can be included as filters for the NWDAF (MTLF) to notify consumers when an ML model is retrained, e.g., no notification will be issued by the NWDAF (MTLF) unless the new ML model meets the following conditions: ○ The performance metric you are interested in (or an identifier for it, e.g., "mse" for mean squared error, "rmse" for root mean squared error, or "mae" for mean absolute error; this list is not exhaustive and depends on the type of ML model). ○ Performance metric thresholds. o The logical relationship between the performance metric and a threshold (i.e., "gt" for "greater than", "lt" for "less than", etc.). ML / AI framework constraints. For example, a new ML model must be runnable on a specific type and version of an ML / AI framework.

[0089] 3GPP TS 29.520 (v17.3.0) section 5.4.6.2.2 defines the existing Nnwdaf_MLModel-Provision_Subscribe request message. FIG. 9 illustrates an example Nnwdaf_MLModelProvision_Subscribe request message data structure according to various embodiments of the present disclosure. Each arrow leads from an instance of an object in the data structure to the corresponding description (or type) of that object. New fields include mlModelInfo of MLEventSubscription and mlModelConditions of MLAnalyticsFilter, which are defined by the corresponding types shown in FIG. 9.

[0090] 3GPP TS 23.288 Section 6.2A.1 specifies that when a subscription request is received, the NWDAF (MTLF) may determine whether further training for an existing trained ML model is required to fulfill the subscription. The standard states that this determination is made based on implementation logic without specifying such logic.

[0091] According to some embodiments, the NWDAF (MTLF) explicitly checks the conditions provided in the subscription for notifications to determine if retraining is required. Additionally, the NWDAF (MTLF) informs the consumer / subscriber that even if retraining is performed according to the conditions, notifications may be provided conditionally. For example, the NTWDAF (MTLF) may inform the consumer that it will only be notified if the performance metrics of the retrained ML model match the conditions in the subscription.

[0092] 3GPP TS 23.288 section 7.5.4 specifies the following data to be included in the Nnwdaf_MLModelProvision_Notify message: ● Input,required: a set of tuples (analysis ID, address of model file (e.g., URL or FQDN)), and,notification correlation information. ● Input, optional: validity period, spatial validity.

[0093] Various embodiments of the present disclosure provide one or more of the following improvements to the existing Nnwdaf_MLModelProvision_Notify request for each tuple in the message: Adding an ML model identifier: This can be optional if your Analytics ID implementation only includes one ML model. Add ML model version, and ● If a performance metric identifier is present in the corresponding subscription request, add the performance metric identifier and the performance metric value obtained when training the ML model.

[0094] The above mentioned URL can be an ML model URL (allowing to retrieve the ML model) or an ML model manifest URL pointing to the ML model metadata (or a description thereof). For example, the ML model metadata can include additional information for the NWDAF (AnLF) to execute the ML model, as well as the actual URL of the ML model. This allows more flexibility to include further information related to the model execution without the need to code each attribute as part of the service operation.

[0095] 3GPP TS 29.520 (v17.3.0) section 5.4.6.2.5 specifies the existing Nnwdaf_MLModel-Provision_Notify request message. FIG. 10 illustrates an example Nnwdaf_MLModelProvision_Notify request message data structure according to various embodiments of the present disclosure. Each arrow leads from an instance of an object in the data structure to the corresponding description (or type) of that object. New fields include mlModelInfo in MLEventNotify and mlModelConditions in MLAnalyticsFilter, which are defined by the corresponding types. Note that the MLModelPerformanceMetric type is the same as in FIG. 9. The MLModelMetadata type is updated to include several additional fields.

[0096] According to another embodiment, the Nnwdaf_MLModelInfo service described in 3GPP® TS 23.288 section 7.6 can be improved in a similar manner as the Nnwdaf_MLModelProvision service described above, but without specificity for ML model training. 3GPP® TS 23.288 specifies the following data to be included in the Nnwdaf_MLModelInfo_Request request message: ● Input, required: (a set of) analysis IDs as defined in Table 7.1-2. ● Input, optional: analysis filter information indicating the conditions required by the ML model for analysis, and the target of the analysis report indicating the objects required by the ML model for analysis (e.g., a specific UE, a group of UEs, or any UE (i.e., all UEs)), the ML model target period.

[0097] Various embodiments of the present disclosure provide one or more of the following improvements to the existing Nnwdaf_MLModelInfo_Request request message: ●Make the Analytics ID(s) optional (if the ML model identifier is included, this is not included). Include (a set of) ML model identifiers as an optional parameter. The request must contain either an Analytics ID or a model identifier. If only an Analytics ID is included, this will be used as a filter to select the ML models referenced by the notification. If an ML model identifier is included, the ML model version may also be included.

[0098] 11 illustrates an example Nnwdaf_MLModelInfo_Request request message data structure according to various embodiments of the present disclosure. Each arrow leads from an instance of an object in the data structure to the corresponding description (or type) of that object. New fields include mlModelInfo within MLEventSubscription.

[0099] 3GPP TS 23.288 specifies the following data to be included in the Nnwdaf_MLModelInfo_Request response message: Output, required: A set of tuples (Analytics ID, address of the model file). The address can be a URL or a fully qualified domain name (FQDN). ● Output, optional: validity period, spatial validity.

[0100] Various embodiments of the present disclosure provide one or more of the following improvements to the existing Nnwdaf_MLModelInfo_Request response message, particularly for each tuple in the message: Adding an ML model identifier: This is optional if your Analytics ID implementation only includes one ML model. Indicates that the URL references an ML model manifest, which must contain the following required elements: ○ML model identifier. The Analytics ID for which the ML model will be used. ○Updated ML model version. ○ The creation timestamp of the updated version of the ML model. ○ML model URL.

[0101] 12 illustrates a data structure of an example Nnwdaf_MLModelInfo_Request response message, according to various embodiments of the present disclosure. Each arrow leads from an instance of an object in the data structure to the corresponding description (or type) of that object. New fields include mlModelInfo for MLEventSubscription, which is defined by the corresponding type MLModelInfo. The MLModelMetadata type is updated to include several additional fields.

[0102] The above-described embodiments may be further described with reference to FIGS. 13-14, which respectively illustrate example methods (e.g., procedures) for the MTLF and AnLF of the NWDAF. In other words, various features of the operations described below correspond to the various embodiments described above. The example methods illustrated in FIGS. 13-14 may be used in conjunction (e.g., with each other and with other procedures described herein) to provide benefits, advantages, and / or solutions to problems described herein. Although the example methods are illustrated in FIGS. 13-14 by certain blocks in a particular order, the operations corresponding to the blocks may be performed in a different order than shown and may be combined and / or divided into blocks and / or operations having different functionality than shown. Optional blocks and / or operations are indicated by dashed lines.

[0103] Specifically, Figure 13 shows Disclosure13 illustrates an example method (e.g., procedure) for a first network node or function (NNF) configured for ML model management in a communications network (e.g., 5GC) in accordance with various embodiments of the present invention. For example, the example method illustrated in FIG. 13 may be performed by an NWDAF (MTLF) or a network node hosting an NWDAF (MTLF) as described elsewhere herein.

[0104] An example method may include the operations of block 1310, where a first NNF may receive, from a second NNF of the communication network, a first message including one of the following: one or more ML model identifiers corresponding to one or more ML models maintained by the MTLF; or ● An identifier for an analysis based on one or more ML models (for example, if the ML model identifier is not included).

[0105] The example method may also include the operation of block 1340, where the first NNF may send a second message to the AnLF that includes one of the following: A plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a distinct one of the ML model identifiers and one or more information elements associated with the corresponding ML model; or ● A single tuple containing an analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

[0106] According to some embodiments, for each tuple included in the second message, the one or more information elements include a network address that is one of a Universal Resource Locator (URL) or a Fully Qualified Domain Name (FQDN). According to some embodiments, each network address included in the second message is one of an address that allows for obtaining the corresponding ML model or an address that allows for obtaining a manifest or metadata file associated with the corresponding ML model. According to some of these embodiments, the manifest or metadata file associated with each of the ML models includes: ●ML model identifier. ●An identifier for one or more analyses based on the ML model. An updated version of the ML model, where "updated" refers to the latest version of the ML model that has been trained and validated by the MTLF and therefore may be used for inference by AnLF. The creation timestamp of the updated version of the ML model. A network address that allows retrieving the ML model.

[0107] According to some embodiments for each tuple included in the second message, the one or more information elements also include an updated version of the corresponding ML model. According to some embodiments, if the first message includes an analysis identifier (e.g., at block 1310), the example method may also include the operation of block 1320, where the first NNF may use the (received) analysis identifier to determine one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based.

[0108] According to other embodiments, if the first message includes one or more of the ML model identifiers, the first message also indicates a version associated with the corresponding ML model. For example, the version included in the first message is the version of the ML model used for inference by the second NNF when the first message is sent. When a new version of the ML model is trained by the first NNF, it is assigned a larger (newer) version identifier. According to some of these embodiments, sending the second message in block 1340 includes one of the following operations, identified with a corresponding subblock number: ● (1341) selectively sending a second message including the single tuple based on the updated version of the single ML model being newer than the version indicated by the first message; or ● (1342) Selectively including each of the plurality of tuples in a second message based on an updated version of the corresponding ML model being newer than the version indicated by the first message.

[0109] According to some embodiments, the first NNF is an MTLF of the NWDAF and the second NNF is an AnLF of the NWDAF. According to some of these embodiments, the first message is an Nnwdaf_MLModelInfo_Request message (e.g., FIG. 11) and the second message is a response to the Nnwdaf_MLModelInfo_Request message (e.g., FIG. 12). FIG. 4 shows an example signaling diagram of these messages.

[0110] According to these other embodiments, the first message is a Nnwdaf_MLModel-Provision_Subscribe message (e.g., FIG. 9) and the second message is a Nnwdaf_MLModelProvision_Notify message (e.g., FIG. 10). FIG. 3 shows an example of a signaling diagram of these messages. According to some of these embodiments, the first message also includes one or more conditions for receiving the second message, including one or more of the following: ●Performance metrics (or identifiers thereof) of the ML model or models on which the analysis is based. - Thresholds for each of the performance metrics. ● The logical relationship between performance metrics and thresholds, and ●ML / AI framework constraints for running one or more ML models.

[0111] According to some variations, the example method may also include the operation of block 1330, where the first NNF may determine whether to retrain one or more ML models based on whether current values ​​of the performance metrics of the respective models satisfy respective thresholds and logical relationships (e.g., as indicated in the first message).

[0112] According to some variations, sending the second message in block 1340 includes one of the following operations, identified with a corresponding sub-block number: ● (1343) selectively sending a second message including a single tuple based on a single ML model that satisfies one or more conditions included in the first message; or ● (1344) Selectively including each of the plurality of tuples in a second message based on a corresponding ML model that satisfies one or more conditions included in the first message.

[0113] According to some further variations, for each tuple included in the second message, the one or more information elements also include an identifier of a performance metric indicated in the first message and a value of the identified performance metric for the corresponding ML model.

[0114] Further, Figure 14 illustrates an example method (e.g., procedure) for a second NNF configured for ML model management in a communications network (e.g., 5GC) according to various embodiments of the present disclosure. For example, the example method illustrated in Figure 14 may be performed by an NWDAF (AnLF) or a network node hosting an NWDAF (AnLF), as described elsewhere herein.

[0115] An example method may include the operation of block 1420, where the second NNF may send a first message to a first NNF of the communication network, the first message including one of the following: one or more ML model identifiers corresponding to one or more ML models maintained by the MTLF; or ● An identifier for an analysis based on one or more ML models (for example, if the ML model identifier is not included).

[0116] The example method may also include the operation of block 1430, where the second NNF may receive, from the MTLF, a second message including one of the following: A plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a distinct one of the ML model identifiers and one or more information elements associated with the corresponding ML model; or ● A single tuple containing an analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

[0117] According to some embodiments, for each tuple included in the second message, the one or more information elements include a network address that is a URL or a FQDN. According to some embodiments, each network address included in the second message is one of an address that allows to obtain a corresponding ML model or an address that allows to obtain a manifest or metadata file associated with the corresponding ML model.

[0118] According to some of these embodiments, the manifest or metadata file associated with each of the ML models includes: ●ML model identifier. ●An identifier for one or more analyses based on the ML model. ●Updated version of the ML model. The creation timestamp of the updated version of the ML model. A network address that allows retrieving the ML model.

[0119] According to these other embodiments, the example method may also include operations of blocks 1440-1450, where the second NNF may obtain a ML model or models based on the network address in the corresponding tuple or tuples (e.g., included in the second message) and apply the obtained ML model or models to determine an analysis identified by the analysis identifier.

[0120] According to some embodiments, for each tuple included in the second message, the one or more information elements also include an updated version of the corresponding ML model. According to some embodiments, if the first message includes one or more of the ML model identifiers, the first message also indicates the version associated with the corresponding ML model. As described above, the version included in the first message may be the version of the ML model used for inference by the second NNF when the first message is transmitted. When a new version of the ML model is trained by the first NNF, it may be assigned a larger (newer) version identifier.

[0121] In some of these embodiments, one or more of the following apply: A second message containing a single tuple is received only if the updated version of the single ML model is newer than the version indicated by the first message. Each of the multiple tuples is received in the second message only if the updated version of the corresponding ML model is newer than the version indicated by the first message.

[0122] According to some embodiments, the first NNF is an MTLF of the NWDAF and the second NNF is an AnLF of the NWDAF. According to some of these embodiments, the first message is an Nnwdaf_MLModelInfo_Request message (e.g., FIG. 11) and the second message is a response to the Nnwdaf_MLModelInfo_Request message (e.g., FIG. 12). FIG. 4 shows an example signaling diagram of these messages.

[0123] According to these other embodiments, the first message is a Nnwdaf_MLModel-Provision_Subscribe message (e.g., FIG. 9) and the second message is a Nnwdaf_MLModelProvision_Notify message (e.g., FIG. 10). FIG. 3 shows an example of a signaling diagram of these messages. According to some of these embodiments, the first message also includes one or more conditions for receiving the second message, including one or more of the following: ●Performance metrics (or identifiers thereof) of the ML model or models on which the analysis is based. - Thresholds for each of the performance metrics. The logical relationship between performance metrics and thresholds; and ●ML / AI framework constraints for running one or more ML models.

[0124] According to some variants, one or more of the following apply: A second message containing a single tuple is received only if the single ML model satisfies one or more conditions contained in the first message. ● Each of the multiple tuples is received in the second message only if the corresponding ML model satisfies one or more conditions included in the first message.

[0125] According to some further variants, or in each tuple included in the second message, the one or more information elements also include an identifier of the performance metric indicated in the first message and a value of the identified performance metric for the corresponding ML model.

[0126] According to some embodiments, the example method may also include the operation of block 1410, where the second NNF may use the analysis identifier to determine one or more ML model identifiers corresponding to one or more ML models on which the analysis is based. In such a case, the first message includes the determined one or more ML model identifiers.

[0127] Although various embodiments have been described herein with respect to methods, apparatus, devices, computer readable media, and receivers, such methods may be implemented using hardware and software in various systems, communication devices, computing devices, control devices, apparatus, non-transitory computer readable media, and the like. Eh Those skilled in the art will readily appreciate that the present invention can be implemented by various combinations of the above.

[0128] FIG. 15 illustrates an example of a communications system 1500 according to some embodiments. In this example, the communications system 1500 includes a telecommunications network 1502 including an access network 1504, such as a radio access network (RAN), and a core network 1506 including one or more core network nodes 1508. The access network 1504 includes one or more access network nodes, such as network nodes 1510a and 1510b (one or more of which may be generally referred to as network nodes 1510), or any other similar Third Generation Partnership Project (3GPP) access nodes or non-3GPP access points. The network nodes 1510 enable direct or indirect connectivity of user equipment (UE), such as by connecting UEs 1512a, 1512b, 1512c, and 1512d (one or more of which may be generally referred to as UEs 1512) to the core network 1506 via one or more wireless connections.

[0129] Exemplary wireless communications over wireless connections include transmitting and / or receiving radio signals using electromagnetic waves, radio waves, infrared, and / or other types of signals suitable for conveying information without the use of wires, cables, or other data conductors. Further, according to various embodiments, communication system 1500 may include any number of wired or a radio network, a network node, a UE, and / or wired or any other components or systems that may facilitate or be involved in the communication of data and / or signals, whether via wireless connections or not. Communications system 1500 may include and / or interface with any type of communications, telecommunications, data, cellular, wireless networks, and / or other similar types of systems.

[0130] The UE 1512 may be any of a wide variety of communication devices, including wireless devices designed, configured, and / or operable to wirelessly communicate with the network node 1510 and other communication devices. Similarly, the network node 1510 is designed, enabled, configured, and / or operable to communicate directly or indirectly with the UE 1512 and / or other network nodes or apparatus in the telecommunications network 1502 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as management in the telecommunications network 1502.

[0131] According to the illustrated embodiment, the core network 1506 connects the network nodes 1510 to one or more hosts, such as the host 1516. These connections may be direct or indirect through one or more intermediate networks or devices. In other examples, the network nodes may be directly coupled to the hosts. The core network 1506 may include hardware Eh Art and software EhThe core network node may include one or more core network nodes (e.g., core network node 1508) comprised of a UE, a network node, and / or a host. Features of these components may be substantially similar to those described with respect to the UE, the network node, and / or the host, and therefore, those descriptions are generally applicable to the corresponding components of the core network node 1508. An example core network node includes one or more functions of a Mobile Switching Center (MSC), a Mobility Management Entity (MME), a Home Subscriber Server (HSS), an Access and Mobility Management Function (AMF), a Session Management Function (SMF), an Authentication Server Function (AUSF), a Subscription Identifier Unhiding Function (SIDF), a Unified Data Management (UDM), a Security Edge Protection Proxy (SEPP), a Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0132] As a specific example, the one or more core network nodes 1508 may be configured to perform operations attributed to a first NNF and a second NNF in the description of various methods or procedures herein. More specifically, the one or more core network nodes 1508 may be configured to perform operations attributed to an MTLF of the NWDAF and an AnLF of the NWDAF.

[0133] The host 1516 may be under the ownership or control of, and may be operated by or on behalf of, a service provider other than an operator or provider of the access network 1504 and / or the telecommunications network 1502. The host 1516 may host various applications to provide one or more services. Examples of such applications include live and pre-recorded audio / video content, data collection services such as searching and compiling data regarding various ambient conditions detected by multiple UEs, analytics functions, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and monitoring center, or any other such functions performed by a server.

[0134] In general, the communication system 1500 of FIG. 15 enables connectivity between UEs, network nodes, and hosts. In that sense, the communication system may include, but is not limited to: Mobile Global System for Communications (GSM), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standards (e.g., 6G), wireless local area network (WLAN) standards such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard (WiFi), and / or World Wide Web standards for microwave access. interoperability The wireless device may be configured to operate according to predetermined rules or procedures, such as a particular standard, including any other suitable wireless communication standard, such as WiMax, Bluetooth, Z-Wave, Near Field Communication (NFC), ZigBee, LiFi, and / or any Low Power Wide Area Network (LPWAN) standard, such as LoRa and Sigfox.

[0135] In some examples, the telecommunications network 1502 implements features of 3GPP standardization. Thus, the telecommunications network 1502 may support network slicing to provide different logical networks to different devices connected to the telecommunications network 1502. For example, the telecommunications network 1502 may provide Ultra-Reliable Low-Latency Communication (URLLC) services to some UEs, while providing enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive IoT services to additional UEs.

[0136] In some examples, the UE 1512 is configured to transmit and / or receive information without direct human interaction. For example, the UE may be designed to transmit information to the access network 1504 on a predefined schedule, when triggered by an internal or external event, or in response to a request from the access network 1504. Furthermore, the UE may be configured to operate in a single or multi-RAT or multi-standard mode. For example, the UE may operate in any one or combination of Wi-Fi, NR (New Radio), and LTE, i.e., configured with Multi-Radio Dual Connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio-Dual Connectivity (EN-DC).

[0137] In an illustrative example, the hub 1514 communicates with the access network 1504 to facilitate indirect communication between one or more UEs (e.g., UEs 1512c and / or 1512d) and a network node (e.g., network node 1510b). In some examples, the hub 1514 may be a controller, a router, a content source and analysis, or any of the other communication devices described herein with respect to UEs. For example, the hub 1514 may be a broadband router that enables access to the core network 1506 for the UE. As another example, the hub 1514 may be a controller that sends commands or instructions to one or more actuators in the UE. The instructions or instructions may be received from the UE, the network node 1510, or by executable code, scripts, processes, or other instructions in the hub 1514. As another example, the hub 1514 may be a data collector that acts as a temporary storage for UE data and may perform analysis or other processing of the data according to some embodiments. As another example, the hub 1514 may be a content source. For example, in the case of a UE that is a VR headset, display, loudspeaker or other media distribution device, the hub 1514 may retrieve, via a network node, data related to VR assets, video, audio, or other media or sensory information, which the hub 1514 then provides directly to the UE, either after performing local processing and / or adding additional local content. In yet another example, the hub 1514 acts as a proxy server or orchestrator for the UEs, particularly if one or more of the UEs are low energy IoT devices.

[0138] The hub 1514 may have a constant / persistent or intermittent connection to the network node 1510b. The hub 1514 may also enable other communication schemes and / or schedules between the hub 1514 and the UEs (e.g., UEs 1512c and / or 1512d) and between the hub 1514 and the core network 1506. According to other embodiments, the hub 1514 may: wiredThe hub 1514 may be configured to connect to the core network 1506 and / or one or more UEs via a connection. Additionally, the hub 1514 may be configured to connect to an M2M service provider via the access network 1504 and / or to another UE via a direct connection. In some situations, the UE may wired or may establish a wireless connection with the network node 1510 while still connected via the hub 1514 via a wireless connection. According to some embodiments, the hub 1514 may be a dedicated hub, i.e., a hub whose primary function is to route communications to / from the network node 1510b to the UE. According to other embodiments, the hub 1514 may be a non-dedicated hub, i.e., a device that is capable of operating to route communications between the UE and the network node 1510b, but that is also capable of operating as a communication origination and / or termination point for a particular data channel.

[0139] FIG. 16 illustrates a UE 1600 according to some embodiments. As used herein, a UE refers to a device capable of wirelessly communicating with network nodes and / or other UEs. Examples of UEs include smartphones, move Examples include, but are not limited to, telephones, mobile phones, voice-over-IP (VoIP) phones, wireless local loop phones, desktop computers, personal digital assistants (PDAs), wireless cameras, gaming consoles or devices, music storage devices, playback appliances, wearable terminal devices, wireless endpoints, mobile stations, tablets, laptops, laptop embedded equipment (LEE), laptop mounted equipment (LME), smart devices, wireless customer premises equipment (CPE), vehicle mounted or embedded / integrated wireless devices, etc. Other examples include any UE defined by the 3rd Generation Partnership Project (3GPP), including Narrowband Internet of Things (NB-IoT) UE, Machine Type Communication (MTC) UE, and / or enhanced MTC (eMTC) UE.

[0140] A UE may support device-to-device (D2D) communications, for example, by implementing 3GPP standards for sidelink communications, dedicated short-range communications (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates an associated device. Instead, a UE may represent a device that is intended for sale to or operation by a human user, but may or may not initially be associated with a particular human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to or operation by an end user, but may be associated with or operated for a user (e.g., a smart power meter).

[0141] The UE 1600 includes a processing circuit 1602 operatively coupled to an input / output interface 1606, a power source 1608, a memory 1610, a communication interface 1612, and / or any other components, or any combination thereof, via a bus 1604. Some UEs may utilize all or a subset of the components shown in FIG. 16. The level of integration between components may vary from UE to UE. Additionally, some UEs may include multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0142] The processing circuit 1602 is configured to process instructions and data and may be configured to implement any sequential state machine operable to execute instructions stored as a machine-readable computer program in the memory 1610. The processing circuit 1602 may be implemented as one or more hardware-implemented state machines (e.g., discrete logic, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.), programmable logic with appropriate firmware, one or more stored computer programs such as a microprocessor or digital signal processor (DSP) with appropriate software, a general purpose processor, or any combination of the above. For example, the processing circuit 1602 may include multiple central processing units (CPUs).

[0143] In this example, the input / output interface 1606 may be configured to provide an interface to an input device, an output device, or one or more input and / or output devices. Examples of output devices include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smart card, another output device, or any combination thereof. An input device may enable a user to capture information to the UE 1600. Examples of input devices include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a webcam, etc.), a microphone, a sensor, a mouse, a trackball, a directional keypad, a trackpad, a scroll wheel, a smart card, etc. A presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. The sensor may be, for example, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, a light sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0144] According to some embodiments, the power source 1608 is configured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electrical outlet), a photovoltaic device, or a power cell, can be used. The power source 1608 may further include a power circuit for delivering power to various parts of the UE 1600 from the power source 1608 itself and / or from the external power source via an interface, such as an input circuit or a power cable. The delivery of power may be, for example, for charging the power source 1608. The power circuit may perform any formatting, conversion, or other modification of the power from the power source 1608 to make the power suitable for the respective component of the UE 1600 to be powered.

[0145] The memory 1610 may be or be configured to include random access memory (RAM), read only memory (ROM), field programmable gate array read only memory (PROM), erasable field programmable gate array read only memory (EPROM), electrically erasable field programmable gate array read only memory (EEPROM), magnetic disk, optical disk, hard disk, removable cartridge, flash drive, or the like. According to one embodiment, the memory 1610 includes one or more application programs 1614, such as an operating system, a web browser application, widgets, gadget engine, or other applications, and corresponding data 1616. The memory 1610 may store any of a variety of operating systems or combinations of operating systems for use by the UE 1600.

[0146] The memory 1610 may be configured to include several physical drives, such as a redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high density digital versatile disk (HD-DVD) optical disk drive, internal hard disk drive, Blu-ray optical disk drive, Holographic Digital Data Storage (HDDS) optical disk drive, external mini dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smart card memory, such as a tamper-resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), e.g., USIM and / or ISIM, other memory, or any combination thereof. The UICC may be, for example, an embedded UICC (eUICC), an integrated UICC (iUICC), or a removable UICC commonly known as a "SIM card." The memory 1610 may enable the UE 1600 to access, offload data, or upload data, instructions, application programs, and the like stored on a temporary or non-transitory memory medium. An article of manufacture, such as one that utilizes a communications system, may be tangibly embodied as or in memory 1610, which may be or include a device-readable storage medium.

[0147] The processing circuit 1602 may be configured to communicate with an access network or other networks using a communication interface 1612. The communication interface 1612 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 1622. The communication interface 1612 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in the access network). Each transceiver may include a transmitter 1618 and / or a receiver 1620 suitable for providing network communication (e.g., optical, electrical, frequency allocation, etc.). Additionally, the transmitter 1618 and receiver 1620 may be coupled to one or more antennas (e.g., antenna 1622) and may share circuit components, software or firmware or may be implemented separately.

[0148] According to the illustrated embodiment, the communication capabilities of the communication interface 1612 may include cellular communications, Wi-Fi communications, LPWAN communications, data communications, voice communications, multimedia communications, short-range communications such as Bluetooth, close-range communications, location-based communications such as using the Global Positioning System (GPS) to determine location, another similar communication capability, or any combination thereof. Communications may be implemented according to one or more communications protocols and / or standards, such as IEEE 802.11, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Synchronous Optical Networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), etc.

[0149] Regardless of the type of sensor, the UE can provide data output captured by its sensors to a network node over a wireless connection via its communications interface 1612. Data captured by a sensor in a UE can be communicated to a network node over a wireless connection via another UE. The output can be periodic (e.g., once every 15 minutes when reporting sensed temperature), random (e.g., over a load from reports from several sensors), in response to a triggering event (e.g., when moisture is detected, an alert is sent), upon request (e.g., a user-initiated request), or in response to a continuous stream (e.g., a live video feed of a patient). (ring) It could be.

[0150] As another example, the UE may receive wireless signals from a network node via a wireless connection. input The UE may include an actuator, motor, or switch associated with a communications interface configured to receive wireless input. In response to the received wireless input, the state of the actuator, motor, or switch may change. For example, the UE may control a flying robot arm to perform a medical procedure in accordance with the received input. Drone The aircraft may include motors for adjusting the control surfaces or rotors of the aircraft.

[0151] When the UE is in the form of an Internet of Things (IoT) device, it may be a device for use in one or more application areas, including, but not limited to, urban wearable technology, extended industrial applications, and healthcare. Non-limiting examples of such IoT devices are devices such as or mounted on connected refrigerators or freezers, TVs, connected lighting devices, electric meters, robotic vacuum cleaners, voice-controlled smart speakers, home security cameras, motion detectors, thermostats, smoke detectors, door / window sensors, flood / moisture sensors, electric door locks, connected doorbells, air conditioning systems such as heat pumps, autonomous vehicles, surveillance systems, weather monitors, vehicle parking monitors, electric vehicle charging stations, smart watches, fitness trackers, head mounted displays for augmented reality (AR) or virtual reality (VR), wearables for haptic or sensory augmentation, water sprinklers, devices for tracking animals or items, sensors for monitoring flora and fauna, industrial robots, unmanned aerial vehicles (UAVs), and any type of medical device such as a heart rate monitor or a remote operated surgical robot. A UE in the form of an IoT device comprises circuitry and / or software depending on the intended application of the IoT device, in addition to other components as described in relation to UE 1600 shown in FIG. 16 .

[0152] As yet another specific example, in an IoT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements and transmits results of such monitoring and / or measurements to another UE and / or a network node. The UE may be an M2M device, which in this case may be referred to as an MTC device in a 3GPP® context. As one particular example, the UE may implement the 3GPP® NB-IoT standard. In other scenarios, the UE may represent a vehicle, such as a car, a bus, a truck, a ship, and an aircraft, or other equipment that can monitor and / or report its operating status or other functions related to its operation.

[0153] In fact, any number of UEs may be used together for a single use case. For example, a first UE may be a drone or may be integrated into a drone and may provide drone speed information (obtained via a speed sensor) to a second UE that is a remote controller that operates the drone. When a user makes a change from the remote controller, the first UE may adjust a throttle on the drone (e.g., by controlling an actuator) to increase or decrease the drone's speed. The first and / or second UE may also include two or more of the functions described above. For example, a UE may be equipped with a sensor and an actuator and handle communication of data for both the speed sensor and the actuator.

[0154] 17 illustrates a network node 1700 according to some embodiments. As used herein, a network node refers to a configured, arranged, and / or operative device that can communicate directly or indirectly with UEs and / or other network nodes or devices in a telecommunications network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., wireless access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs), and NR Node Bs (gNBs)).

[0155] Base stations may be classified based on the size of the coverage they provide (or, stated differently, their transmission power level) and may therefore be referred to as femto, pico, micro, or macro base stations depending on the size of the coverage provided. A base station may be a relay node or a relay donor node that controls a relay. A network node may also include one or more (or all) parts of a distributed radio base station, such as a centralized digital unit and / or a remote radio unit (RRU), sometimes referred to as a remote radio head (RRH). Such remote radio units may or may not be integrated with an antenna as an antenna-integrated radio. Some of the distributed radio base stations may also be referred to as nodes in a distributed antenna system (DAS).

[0156] Other examples of network nodes include a multi-transmission point (multi-TRP) 5G access node, a multi-standard radio (MSR) equipment such as an MSR BS, a network controller such as a radio network controller (RNC) or a base station controller (BSC), a base transceiver station (BTS), a transmission point, a transmitting node, a multi-cell / multicast coordination entity (MCE), an operation and maintenance (O&M) node, an operation support system (OSS) node, a self-organizing network (SON) node, a positioning node (e.g., an evolved serving mobile location center (E-SMLC)), and / or a minimization of drive testing (MDT).

[0157] The network node 1700 includes a processing circuit 1702, a memory 1704, a communication interface 1706, and a power source 1708. The network node 1700 may be comprised of multiple physically separate components (e.g., a Node B component and an RNC component, or a BTS component and a BSC component, etc.), each of which may have respective components. In certain situations where the network node 1700 includes multiple separate components (e.g., a BTS and a BSC component), one or more separate components may be shared among multiple network nodes. For example, a single RNC may control multiple Node Bs. In such a scenario, each unique Node B and RNC pair may potentially be considered a single individual network node. In some embodiments, the network node 1700 may include multiple Nothing The network node 1700 may be configured to support multiple wireless access technologies (RATs). According to such an embodiment, some components may be duplicated (e.g., separate memory 1704 for different RATs) and some components may be reused (e.g., the same antenna 1710 may be shared by different RATs). The network node 1700 may also include multiple sets of the various illustrated components for the various wireless technologies integrated into the network node 1700, e.g., GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID), or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chips or chipsets and other components within the network node 1700.

[0158] The processing circuitry 1702 may include one or more combinations of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit, field programmable gate array, or any other suitable computing device, resources, or combination of hardware, software and / or coded logic, which combination is operable, alone or in conjunction with other network node 1700 components, such as memory 1704, to provide network node 1700 functionality.

[0159] According to some embodiments, the processing circuit 1702 comprises a system on a chip (SOC). According to some embodiments, the processing circuit 1702 includes one or more of a radio frequency (RF) transceiver circuit 1712 and a baseband processing circuit 1714. In some embodiments, the radio frequency (RF) transceiver circuit 1712 and the baseband processing circuit 1714 may be on separate chips (or chipsets), boards, or units, such as a radio unit and a digital unit. In alternative embodiments, some or all of the RF transceiver circuit 1712 and the baseband processing circuit 1714 may be on the same chip or chipset, board, or unit.

[0160] The memory 1704 may comprise any form of volatile or non-volatile computer-readable memory, including, but not limited to, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (e.g., hard disk), removable storage media (e.g., flash drive, compact disk (CD) or digital video disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory device that stores information, data, and / or instructions that may be used by the processing circuit 1702. The memory 1704 may include computer programs, software, EhThe processing circuit 1702 may store any suitable instructions, data, or information that can be executed by the processing circuit 1702 and utilized by the network node 1700, including applications including one or more of the following: programs, logic, rules, code, tables, and / or other instructions (collectively referred to as computer program product 1704a). The memory 1704 may be used to store any operations performed by the processing circuit 1702 and / or any data received via the communications interface 1706. According to some embodiments, the processing circuit 1702 and the memory 1704 are integrated.

[0161] The communication interface 1706 is used for wired or wireless communication of signaling and / or data between network nodes, access networks, and / or UEs. As shown, the communication interface 1706 includes a port / terminal 1716 for transmitting and receiving data to and from a network, for example, via a wired connection. The communication interface 1706 also includes a radio front-end circuit 1718 that may be coupled to the antenna 1710, or to a portion thereof according to an embodiment. The radio front-end circuit 1718 includes a filter 1720 and an amplifier 1722. The radio front-end circuit 1718 may be connected to the antenna 1710 and the processing circuit 1702. The radio front-end circuit may be configured to condition signals communicated between the antenna 1710 and the processing circuit 1702. The radio front-end circuit 1718 may receive digital data to be sent to other network nodes or UEs via a wireless connection. The radio front-end circuitry 1718 may convert the digital data into a radio signal having appropriate channel and bandwidth parameters using a combination of filters 1720 and / or amplifiers 1722. The radio signal may then be transmitted via the antenna 1710. Similarly, when receiving data, the antenna 1710 may collect a radio signal that is converted into digital data by the radio front-end circuitry 1718. The digital data may be passed to the processing circuitry 1702. In other embodiments, the communication interface may include different components and / or different combinations of components.

[0162] According to certain alternative embodiments, the network node 1700 does not include a separate radio front-end circuit 1718; instead, the processing circuit 1702 includes the radio front-end circuit and is connected to the antenna 1710. Similarly, according to some embodiments, all or some of the RF transceiver circuit 1712 is part of the communications interface 1706. According to yet other embodiments, the communications interface 1706 includes one or more ports or terminals 1716, the radio front-end circuit 1718, and the RF transceiver circuit 1712 as part of a radio unit (not shown), and the communications interface 1706 communicates with a baseband processing circuit 1714 that is part of a digital unit (not shown).

[0163] The antenna 1710 may include one or more antennas, or an antenna array, configured to transmit and / or receive wireless signals. The antenna 1710 may be coupled to the radio front-end circuitry 1718 and may be any type of antenna capable of wirelessly transmitting and receiving data and / or signals. According to an embodiment, the antenna 1710 is separate from the network node 1700 and may be connectable to the network node 1700 through an interface or port.

[0164] The antenna 1710, the communication interface 1706, and / or the processing circuit 1702 may be configured to perform any receiving operations and / or some acquisition operations described herein as being performed by a network node. Any information, data, and / or signals may be transmitted to a UE, another network node, and / or any other Network Devices Similarly, the antenna 1710, the communication interface 1706, and / or the processing circuit 1702 may be configured to perform any transmission operation described herein as being performed by a network node. Any information, data, and / or signals may be transmitted to a UE, another network node, and / or any other Network Devices may be sent to

[0165] The power source 1708 provides power to the various components of the network node 1700 in a form suitable for each component (e.g., at voltage and current levels required by each component). The power source 1708 may further comprise or be coupled to a power management circuit for supplying the components of the network node 1700 with power to perform the functions described herein. For example, the network node 1700 may be connectable to an external power source (e.g., a power grid, an electrical outlet) via an input circuit or interface, such as a wire, whereby the external power source provides power to the power circuit of the power source 1708. As a further example, the power source 1708 may comprise a power source in the form of a battery or battery pack connected to or integrated into the power circuit. In the event of a failure of the external power source, a backup power may be provided from the battery.

[0166] Embodiments of network node 1700 may include additional components beyond those shown in Figure 17 to provide certain aspects of network node functionality, including any of the functionality described herein and / or any functionality essential to support the subject matter described herein. For example, network node 1700 may include user interface devices that enable input of information into network node 1700 and output of information from network node 1700. This allows a user to perform diagnostics, maintenance, repair, and other management functions of network node 1700.

[0167] As a specific example, the one or more network nodes 1700 may be configured to perform operations attributed to a first NNF and a second NNF in the description of various methods or procedures herein. More specifically, the one or more network nodes 1700 may be configured to perform operations attributed to an MTLF of the NWDAF and an AnLF of the NWDAF.

[0168] 18 is a block diagram of a host 1800, which may be an embodiment of the host 1516 of FIG. 15, in accordance with various aspects described herein. As used herein, the host 1800 may be or comprise various combinations of hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, a container, or processing resources in a server farm. The host 1800 may provide one or more services to one or more UEs.

[0169] The host 1800 includes a processing circuit 1802 operably coupled to an input / output interface 1806, a network interface 1808, a power source 1810, and a memory 1812 via a bus 1804. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 16 and 17, such that the descriptions are generally applicable to the corresponding components of the host 1800.

[0170] The memory 1812 may include one or more computer programs, including one or more host application programs 1814 and data 1816, which may include user data, e.g., data generated by the UE for the host 1800 or data generated by the host 1800 for the UE. An embodiment of the host 1800 may utilize only a subset or all of the components shown. The host application programs 1814 may be implemented in a container-based architecture, and may be implemented in a number of different classes, types, or implementations of UE (e.g., handsets, desktop computers, web computers, etc.). EhThe host application program 1814 may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for video, audio, and video streaming (VL, MPEG, HDMI ...

[0171] FIG. 19 is a block diagram illustrating a virtualization environment 1900 in which functionality implemented by some embodiments may be virtualized. In this context, virtualization means creating a virtual version of a device or device, including virtualizing a hardware platform, storage, and networking resources. As used herein, virtualization may apply to any device described herein, or components thereof, and relates to implementations in which at least a portion of functionality is implemented as one or more virtual components. Some or all of the functionality described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1900 hosted by one or more of the hardware nodes, such as a network node, a UE, a core network node, or a hardware computing device acting as a host. Furthermore, in embodiments in which the virtual node does not require wireless connectivity (e.g., a core network node or a host), the node may be fully virtualized.

[0172] An application 1902 (which may alternatively be referred to as a software instance, a virtual appliance, a network function, a virtual node, a virtual network function, etc.) executes in the virtualization environment Q400 to implement some of the features, functions, and / or advantages of some of the embodiments disclosed herein.

[0173] As a particular example, the first NNF and / or the second NNF described herein with respect to other figures may be implemented as a virtual network function 1902 in the virtualization environment 1900. As a more specific example, the MTLF of the NWDAF and / or the AnLF of the NWDAF may be implemented as a virtual network function 1902 in the virtualization environment 1900.

[0174] The hardware 1904 includes processing circuitry, memory that stores software and / or instructions executable by the hardware processing circuitry (collectively referred to as computer program product 1904a), and / or other hardware devices described herein, such as network interfaces, input / output interfaces, etc. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1906 (also referred to as a hypervisor or virtual machine monitor (VMM)), provide VMs 1908a and 1908b (one or more of which may be generally referred to as VMs 1908), and / or perform any of the functions, features and / or benefits described in connection with some embodiments described herein. The virtualization layer 1906 may present a virtual operating platform to the VMs 1908 that appears to be networking hardware.

[0175] The VMs 1908 may comprise virtual processing, virtual memory, virtual networking or interfaces, and virtual storage, and may be executed by a corresponding virtualization layer 1906. Various embodiments of instances of virtual appliances 1902 may be implemented on one or more of the VMs 1908, and the implementation may be done in different ways. Hardware virtualization occurs in some contexts called network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry-standard high-volume server hardware, physical switches, and physical storage that may be located in a data center, as well as customer premises equipment.

[0176] In the context of NFV, a VM 1908 may be a software implementation of a physical machine that executes programs as if they were running on a physical, non-virtualized machine. Each of the VMs 1908, and that portion of the hardware 1904 on which it runs, is hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forming a separate virtual network element. Furthermore, in the context of NFV, a virtual network function runs in one or more VMs 1908 on the hardware 1904 and is responsible for handling a particular network function corresponding to an application 1902.

[0177] The hardware 1904 may be implemented in a standalone network node having generic or concrete components. The hardware 1904 may implement some functions via virtualization. Alternatively, the hardware 1904 may be part of a larger cluster of hardware (e.g., in a data center or CPE) where many hardware nodes work together and are managed via a management and orchestration 1910 that oversees, among other things, the lifecycle management of the applications 1902. According to some embodiments, the hardware 1904 is coupled to one or more radio units, each including one or more transmitters and one or more receivers that may be coupled to one or more antennas. The radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may provide wireless communications to the virtual nodes. access In some embodiments, some signaling may be provided using the control system 1912, which may alternatively be hardware based. Eh It can be used for communication between the anode and the radio unit.

[0178] Figure 20 illustrates a communication diagram of a host 2002 communicating via a network node 2004 with a UE 2006 over a partial wireless connection, according to some embodiments. Exemplary embodiments of the UEs (such as the UE 1512a of Figure 15 and / or the UE 1600 of Figure 16), network nodes (such as the network node 1510a of Figure 15 and / or the network node 1700 of Figure 17), and hosts (such as the host 1516 of Figure 15 and / or the host 1800 of Figure 18) described in the previous paragraphs, according to various embodiments, are described with reference to Figure 20.

[0179] Similar to the host 1800, an embodiment of the host 2002 includes hardware such as a communications interface, processing circuitry, and memory. The host 2002 also includes software stored on or accessible by the host 2002 and executable by the processing circuitry. This software includes a host application that may be operable to provide services to a remote user, such as a UE 2006 that connects via an over-the-top (OTT) connection 2050 extending between the UE 2006 and the host 2002. In providing to the remote user, the host application may provide user data that is transmitted using the OTT connection 2050.

[0180] The network node 2004 includes hardware for communicating with the host 2002 and the UE 2006. Eh The connection 2060 may be direct or pass through a core network (such as core network 1506 in FIG. 15) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, the intermediate network may be a backbone network or the Internet.

[0181] The UE 2006 includes hardware and software that is stored in or accessible by the UE 2006 and executable by the processing circuitry of the UE. The software includes a client application, such as a web browser or an operator specific "app", that may be operable to provide services to a human or non-human user via the UE 2006 with the support of the host 2002. An executing host application in the host 2002 may communicate with an executing client application via an OTT connection 2050 that terminates at the UE 2006 and the host 2002. In providing services to a user, the client application in the UE may receive request data from the host application in the host and provide user data in response to the request data. The OTT connection 2050 may carry both the request data and the user data. The client application in the UE may interact with the user and generate user data to provide to the host application via the OTT connection 2050.

[0182] The OTT connection 2050 may provide a connection between the host 2002 and the UE 2006 via a connection 2060 between the host 2002 and a network node 2004, and via a wireless connection 2070 between the network node 2004 and the UE 2006. The connections 2060 and wireless connections 2070 over which the OTT connection 2050 may be provided are depicted abstractly to illustrate communication between the host 2002 and the UE 2006 via the network node 2004, although any intermediate devices and the precise routing of messages through these devices are not explicitly mentioned.

[0183] As an example of transmitting data over the OTT connection 2050, in step 2008 the host 2002 provides user data that may be executed by executing a host application. According to some embodiments, the user data is associated with a particular human user interacting with the UE 2006. According to other embodiments, the user data is associated with a UE 2006 that shares data with the host 2002 without explicit human interaction. In step 2010, the host 2002 initiates a transmission carrying the user data towards the UE 2006. The host 2002 may initiate the transmission in response to a request sent by the UE 2006. The request may result from a human interaction with the UE 2006 or from the operation of a client application running on the UE 2006. The transmitted signal may include a user data transmission request, a request for the user data transmission, and a request for the user data transmission. Disclosure In accordance with the teachings of the embodiments described throughout, the present invention may be passed through the network node 2004. Thus, in step 2012, the network node 2004 Disclosure In accordance with the teachings of the embodiments described herein, the UE 2006 transmits user data carried in a transmission initiated by the host 2002 to the UE 2006. In step 2014, the UE 2006 receives the user data carried in the transmission, which may be executed by a client application executed on the UE 2006 associated with a host application executed by the host 2002.

[0184] In some examples, the UE 2006 executes a client application that provides user data to the host 2002. The user data may be provided in reaction to or in response to data received from the host 2002. Thus, in step 2016, the UE 2006 may provide the user data, which may be executed by executing the client application. In providing the user data, the client application may further take into account user input received from a user via an input / output interface of the UE 2006. Regardless of the specific manner in which the user data was provided, the UE 2006, in step 2018, initiates transmission of the user data to the host 2002 via the network node 2004. In step 2020, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 2004 receives the user data from the UE 2006 and initiates transmission of the received user data to the host 2002. In step 2022, the host 2002 receives the user data carried in a transmission initiated by the UE 2006.

[0185] One or more of the various embodiments improve the performance of the OTT service provided to the UE 2006 using the OTT connection 2050 in which the wireless connection 2070 forms the last leg. The embodiments disclosed herein can improve the management of ML models used for analysis in a communication network (e.g., 5GC). More precisely, the embodiments can enable the MTLF to identify the ML model it has retrained when informing the AnLF, thereby enabling the AnLF to swap the particular ML model it is using for inference. As another example, the embodiments can enable the AnLF to express conditions under which the MTLF should retrain the ML model. As another example, the embodiments can enable the MTLF to express training performance results so that the MTLF can decide whether to swap the ML model. In this way, the embodiments can facilitate improved analysis (especially based on ML models) in the network, which can result in improved network performance. Furthermore, the improved network performance can increase the value of the OTT services delivered to both service providers and end users over the network.

[0186] In an exemplary scenario, factory status information may be collected and analyzed by the host 2002. As another example, the host 2002 may process audio and video data retrieved from UEs for use in creating maps. As another example, the host 2002 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 2002 may store surveillance video uploaded by UEs. As another example, the host 2002 may store or control access to media content, such as video, audio, VR, or AR, that may be broadcast, multicast, or unicast to UEs. As another example, the host 2002 may be used for remote control of non-time-critical electrical loads, to balance energy pricing, power generation needs, location services, presentation services (such as compiling diagrams, etc. from data collected from remote devices), or any other function that collects, retrieves, stores, analyzes, and / or transmits data.

[0187] In some examples, measurement procedures may be provided for purposes of monitoring data rates, latency, and other factors that one or more embodiments improve upon. Additionally, there may be optional network functionality to reconfigure the OTT connection 2050 between the host 2002 and the UE 2006 in response to variations in the measurement results. The measurement procedures and / or network functionality to reconfigure the OTT connection may be implemented in software or hardware in the host 2002 and / or the UE 2006. According to some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 2050 passes, the sensors providing values ​​of the monitored quantities exemplified above, or in response to software or hardware. Eh by providing values ​​of other physical quantities from which the monitored quantity can be calculated or estimated; Measurement procedureThe host 2002 may participate in the OTT connection 2050. Reconfiguration of the OTT connection 2050 may include message formats, retransmission settings, preferred routing, etc., and the reconfiguration need not directly change the operation of the network node 2004. Such procedures and functionality may be known and practiced in the art. According to some embodiments, the measurements may involve proprietary UE signaling that facilitates measurements by the host 2002 of throughput, propagation time, latency, etc. The measurements may be software implemented such that messages, particularly empty or "dummy" messages, are sent using the OTT connection 2050 while monitoring propagation times, communication errors, etc.

[0188] As described herein, a device and / or an apparatus may be represented by a semiconductor chip, a chipset, or a (hardware) module comprising such a chip or chipset, which may include, but is not limited to, The functionality of the device or apparatus is Instead of being implemented in hardware, it may be implemented as a software module, such as a computer program or computer program product comprising executable software code portions for execution or executed on a processor. The possibility that In addition, the functionality of a device or apparatus may be implemented by any combination of hardware and software. A device or apparatus may also be considered an assembly of multiple devices and / or apparatus, whether functionally cooperating with each other or independent of each other. Furthermore, devices and apparatus may be implemented in a distributed manner throughout a system, so long as the functionality of the device or apparatus is maintained. Such and similar principles are believed to be known to those skilled in the art.

[0189] Additionally, functionality described herein as being performed by a wireless device or network node may be distributed across multiple wireless devices and / or network nodes. In other words, it is contemplated that the functionality of the network nodes and wireless devices described herein is not limited to performance by a single physical device, but may in fact be distributed among several physical devices.

[0190] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which this disclosure belongs. Terms used herein should be interpreted to have a meaning consistent with their meaning in the context of this specification and related art, and it will be further understood that they are not to be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0191] In addition, certain terms used in this disclosure, including the specification, drawings, and exemplary embodiments thereof, may be used synonymously in certain instances, including, but not limited to, for example, data and information. It should be understood that these and / or other words that may be synonymous with each other may be used synonymously herein, but it may be intended that such words are not used synonymously. Furthermore, knowledge of the prior art is expressly incorporated herein in its entirety, unless expressly incorporated above by reference. All publications referenced are incorporated herein in their entirety by reference.

[0192] The above merely illustrates the principles of the present disclosure. Various modifications and changes to the described embodiments will be apparent to those skilled in the art in light of the teachings herein. Thus, it will be appreciated that those skilled in the art can devise numerous systems, configurations, and procedures that are not explicitly shown or described herein, but which embody the principles of the present disclosure and thus fall within the spirit and scope of the present disclosure. As will be appreciated by those skilled in the art, the various exemplary embodiments can be used together and interchangeably with one another.

[0193] Exemplary embodiments of the techniques and apparatus described herein include, but are not limited to, the embodiments listed below:

[0194] A1. A method for a first node of a communication network, the method comprising: receiving a first message from a second node of the communications network, wherein the first message includes one of the following: one or more ML model identifiers corresponding to one or more ML models maintained by the first node; or An identifier for an analysis based on said one or more ML models. and transmitting a second message to the second node, the second message including one of the following: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, where each tuple includes a distinct one of a plurality of ML model identifiers and a network address associated with the corresponding ML model; or A single tuple including the analysis identifier and a network address associated with a single ML model on which the analysis is based.

[0195] A2. The method of embodiment A1, wherein each network address included in the second message is one of a universal resource locator (URL) or a fully qualified domain name (FQDN).

[0196] A3. The method of any of embodiments A1-A2, wherein each network address included in the second message is one of the following: an address allowing to obtain the corresponding ML model, or An address that allows for obtaining a manifest or metadata file associated with the corresponding ML model.

[0197] A4. The method of embodiment A3, wherein the manifest or metadata file associated with each of the ML models includes: An ML model identifier; and an identifier for one or more analyses based on the ML model; an updated version of the ML model; and A creation timestamp of an updated version of the ML model; and A network address through which the ML model can be obtained.

[0198] A5. A method according to any of embodiments A1-A4, wherein each tuple in the second message also includes an updated version of the corresponding ML model.

[0199] A6. A method according to any of embodiments A1 to A5, further comprising, if the first message includes the analysis identifier, using the analysis identifier to determine one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based.

[0200] A7. A method according to any of embodiments A1-A5, wherein if the first message includes one or more of the ML model identifiers, the first message also indicates a version associated with the corresponding ML model.

[0201] A8. The method of embodiment A7, wherein sending the second message includes one of the following: Selectively transmitting the second message including the single tuple based on the updated version of the single ML model being newer than the version indicated by the first message; or Selectively including each of the plurality of tuples in the second message based on the updated version of the corresponding ML model being newer than the version indicated by the first message.

[0202] A8a. A method according to any of embodiments A1 to A8, wherein the first node is a Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) and the second node is an Analysis Logic Function (AnLF) of the NWDAF.

[0203] A9. The method of embodiment A8a, wherein the first message is a Nnwdaf_MLModelInfo_Request message and the second message is a response to the Nnwdaf_MLModelInfo_Request message.

[0204] A10. The method of embodiment A8a, wherein the first message is a Nnwdaf_MLModelProvision_Subscribe message and the second message is a Nnwdaf_MLModelProvision_Notify message.

[0205] A11. The method of embodiment A10, wherein the first message also includes one or more conditions for receiving the second message, including, inter alia, one or more of the following: performance metrics for the one or more ML models on which the analysis is based; and Respective thresholds for said performance metrics; and a logical relationship between the performance metrics and the thresholds; ML / AI framework constraints for executing the one or more ML models.

[0206] A12. The method of embodiment A11, wherein sending the second message includes one of the following: Selectively transmitting the second message including the single tuple based on the single ML model satisfying the one or more conditions included in the first message; or Selectively including each of the plurality of tuples in the second message based on the corresponding ML model satisfying the one or more conditions included in the first message.

[0207] A13. The method of embodiment A12, wherein each tuple included in the second message further includes: an identifier of a performance metric indicated in the first message; and The value of the performance metric determined for the corresponding ML model.

[0208] A14. A method according to any one of embodiments A11 to A13, further comprising determining whether to retrain the one or more ML models based on whether current values ​​of the performance metrics for each of the models satisfy respective thresholds and logical relationships.

[0209] B1. A method for a second node of a communication network, the method comprising: sending a first message to a first node of the communication network, wherein the first message includes one of the following: one or more ML model identifiers corresponding to one or more ML models maintained by the first node; or An identifier for an analysis based on said one or more ML models. receiving a second message from the first node, the second message including one of the following: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, where each tuple includes a distinct one of the plurality of ML model identifiers and a network address associated with the corresponding ML model; or A single tuple including the analysis identifier and a network address associated with a single ML model on which the analysis is based.

[0210] B2. The method of embodiment B1, wherein each network address included in the second message is one of a universal resource locator (URL) or a fully qualified domain name (FQDN).

[0211] B3. The method of any of embodiments B1-B2, wherein each network address included in the second message is one of the following: an address allowing to obtain the corresponding ML model, or An address that allows for obtaining a manifest or metadata file associated with the corresponding ML model.

[0212] B4. The method of embodiment B3, wherein the manifest or the metadata file associated with each of the ML models includes: An ML model identifier; and an identifier for one or more analyses based on the ML model; an updated version of the ML model; and A creation timestamp of an updated version of the ML model; and A network address through which the ML model can be obtained.

[0213] B5. The method of any of embodiments B1-B4, wherein each tuple in the second message also includes an updated version of the corresponding ML model.

[0214] B6. A method according to any of embodiments B1 to B5, wherein if the first message includes the one or more ML model identifiers, the first message further indicates a version associated with the corresponding ML model.

[0215] B7. The method of embodiment B6, wherein one or more of the following is applied: The second message including the single tuple is received only if the updated version of the single ML model is newer than the version indicated by the first message. Each of the plurality of tuples is received in the second message only if the updated version of the corresponding ML model is newer than the version indicated by the first message.

[0216] B7a. A method according to any of embodiments B1 to B7, wherein the first node is a Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) and the second node is an Analysis Logic Function (AnLF) of the NWDAF.

[0217] B8. The method of embodiment B7a, wherein the first message is a Nnwdaf_MLModelInfo_Request message and the second message is a response to the Nnwdaf_MLModelInfo_Request message.

[0218] B9. The method of embodiment B7a, wherein the first message is a Nnwdaf_MLModelProvision_Subscribe message and the second message is a Nnwdaf_MLModelProvision_Notify message.

[0219] B10. The method of embodiment B9, wherein the first message further includes one or more conditions for receiving the second message, including, inter alia, one or more of the following: performance metrics of the one or more ML models on which the analysis is based; and a threshold for each of said performance metrics; and a logical relationship between the performance metrics and the thresholds; and ML / AI framework constraints for executing the one or more ML models.

[0220] B11. The method of embodiment B10, wherein one or more of the following is applied: The second message including the single tuple is received only if the single ML model satisfies the one or more conditions included in the first message. Each of the plurality of tuples is received in the second message only if the corresponding ML model satisfies the one or more conditions included in the first message.

[0221] B12. The method of embodiment B11, wherein each tuple included in the second message also includes: an identifier of a performance metric indicated in the first message; and The value of the identified performance metric for the corresponding ML model.

[0222] B13. The method according to any one of embodiments B1 to B12, further comprising: obtaining the ML model or the plurality of ML models based on the network address in the corresponding tuple or plurality of tuples; applying the obtained ML model or models to determine the analysis identified by the analysis identifier; Includes.

[0223] B14. A method according to any of embodiments B1 to B13, further comprising using the analysis identifier to determine one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based, wherein the first message includes the determined one or more ML model identifiers.

[0224] C1. A Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) of a communications network, comprising: the MTLF is implemented by an operably coupled communications interface circuit and a processing circuit; The processing circuitry and the communications interface circuitry are configured to perform operations corresponding to any of the methods of embodiments A1-A14.

[0225] C2. A Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) of a communications network, the MTLF configured to perform operations corresponding to any of the methods of embodiments A1-A14.

[0226] C3. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by a processing circuit associated with a Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) of a communications network, configure the MTLF to perform operations corresponding to any of the methods of embodiments A1-A14.

[0227] C4. A computer program product comprising computer-executable instructions that, when executed by a processing circuit associated with a Model Training Logic Function (MTLF) of a Network Data Analysis Function (NWDAF) of a communications network, configure the MTLF to perform operations corresponding to any of the methods of embodiments A1-A14.

[0228] D1. An Analysis Logic Function (AnLF) of a Network Data Analysis Function (NWDAF) of a communications network, comprising: the AnLF being implemented by an operably coupled communications interface circuit and processing circuit; The processing circuitry and the communications interface circuitry are configured to perform operations corresponding to any of the methods of embodiments B1-B12.

[0229] D2. An Analysis Logic Function (AnLF) of a Network Data Analysis Function (NWDAF) of a communications network, said AnLF configured to perform operations corresponding to the method of any of embodiments B1-B12.

[0230] D3. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by a processing circuit associated with an Analysis Logic Function (AnLF) of a Network Data Analysis Function (NWDAF) of a communications network, configure the AnLF to perform operations corresponding to any of the methods of embodiments B1-B12.

[0231] D4. A computer program product comprising computer-executable instructions that, when executed by a processing circuit associated with an Analysis Logic Function (AnLF) of a Network Data Analysis Function (NWDAF) of a communications network, configure the AnLF to perform operations corresponding to any of the methods of embodiments B1-B12.

Claims

1. 1. A method for a first network node or function (NNF) configured for machine learning (ML) model management in a communication network, the method comprising: Receiving a first message from a second NNF of the communication network (1310); the first message comprising: one or more machine learning (ML) model identifiers corresponding to one or more ML models maintained by the first NNF; and an identifier of the one or more ML model-based analyses; and transmitting a second message to the second NNF (1340), the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; and a single tuple including the analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

2. 2. The method of claim 1, wherein for each tuple included in the second message, the one or more information elements include a network address that is one of a Universal Resource Locator (URL) or a Fully Qualified Domain Name (FQDN).

3. 3. The method according to claim 1, wherein each network address included in the second message is an address allowing to obtain said corresponding ML model; an address allowing for obtaining a manifest or metadata file associated with the corresponding ML model; The method is one of the above.

4. 4. The method of claim 3, wherein the manifest or metadata file associated with each of the ML models comprises: an ML model identifier; an identifier for one or more analyses based on the ML model; an updated version of the ML model; and a creation timestamp of the updated version of the ML model; and a network address allowing the ML model to be obtained; A method comprising:

5. 5. The method of claim 1, wherein for each tuple included in the second message, the one or more information elements further include an updated version of the corresponding ML model.

6. 6. The method of claim 1, further comprising: if the first message includes the analysis identifier, determining (1320) using the analysis identifier to determine one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based.

7. 6. The method of claim 1, wherein if the first message includes the one or more of the ML model identifiers, the first message further indicates a version associated with the corresponding ML model.

8. 8. The method of claim 7, wherein the step of transmitting (1340) the second message further comprises: Selectively sending the second message including the single tuple based on the updated version of the single ML model being newer than the version indicated by the first message (1341); Selectively including each of the plurality of tuples in the second message based on the updated version of the corresponding ML model being newer than the version indicated by the first message (1342); The method includes one of the following:

9. 9. The method according to any one of claims 1 to 8, wherein the first NNF is a model training logic function (MTLF) of a network data analysis function (NWDAF), and the second NNF is an analysis logic function (AnLF) of the NWDAF.

10. 10. The method of claim 9, The first message is a Nnwdaf_MLModelInfo_Request message; The method, wherein the second message is a response to a Nnwdaf_MLModelInfo_Request message.

11. 10. The method of claim 9, The first message is a Nnwdaf_MLModelProvision_Subscribe message; The method, wherein the second message is a Nnwdaf_MLModelProvision_Notify message.

12. 12. The method of claim 11, The first message includes one or more conditions for receiving the second message, the conditions including: performance metrics of the one or more ML models on which the analysis is based; and a threshold for each of said performance metrics; and a logical relationship between the performance metrics and the thresholds; and ML / AI framework constraints for executing the one or more ML models; and The method includes one or more of:

13. 13. The method of claim 12, wherein the step of transmitting (1340) the second message comprises: Selectively transmitting (1343) the second message including the single tuple based on the single ML model satisfying the one or more conditions included in the first message; Selectively including each of a plurality of tuples in the second message based on the corresponding ML model satisfying the one or more conditions included in the first message (1344); The method includes one of the following:

14. 14. The method of claim 13, wherein the one or more information elements for each tuple included in the second message further comprise: an identifier of a performance metric indicated in the first message; and the value of the performance metric determined for the corresponding ML model; and A method comprising:

15. 15. The method of claim 12, further comprising: determining (1330) whether to retrain the one or more ML models based on whether current values ​​of the performance metrics for the corresponding ML models satisfy the corresponding thresholds and logical relationships.

16. 1. A method for a second network node or function (NNF) configured for machine learning (ML) model management in a communications network, the method comprising: Sending a first message to a first NNF of the communication network (1420); the first message comprising: one or more ML model identifiers corresponding to one or more ML models maintained by the first NNF; an identifier of the one or more ML model-based analyses; and receiving a second message from the first NNF (1430), the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; and a single tuple including the analysis identifier and one or more information elements associated with a single ML model on which the analysis is based.

17. 17. The method of claim 16, wherein the one or more information elements for each tuple included in the second message include a network address that is one of a universal resource locator (URL) or a fully qualified domain name (FQDN).

18. 18. The method according to claim 16, wherein each network address included in the second message is an address allowing to obtain said corresponding ML model; an address allowing for obtaining a manifest or metadata file associated with the corresponding ML model; The method includes one of the following:

19. 20. The method of claim 18, wherein the manifest or metadata file associated with each of the ML models comprises: an ML model identifier; an identifier for one or more analyses based on the ML model; an updated version of the ML model; and a creation timestamp of the updated version of the ML model; and a network address allowing the ML model to be obtained; A method comprising:

20. The method according to any one of claims 17 to 18, further comprising: Obtaining the ML model or the ML models based on a network address in the corresponding tuple or tuples (1440); applying 1450 the retrieved ML model or models to determine the analysis identified by the analysis identifier; A method comprising:

21. A method according to any one of claims 16 to 20, wherein for each tuple included in the second message, the one or more information elements further include an updated version of the corresponding ML model.

22. The method according to any one of claims 16 to 21, wherein if the first message includes the one or more ML model identifiers, the first message also indicates a version associated with the corresponding ML model.

23. 23. The method of claim 22, the second message including the single tuple is received only if the updated version of the single ML model is newer than the version indicated by the first message; each of the plurality of tuples is received in the second message only if the updated version of the corresponding ML model is newer than the version indicated by the first message; The method according to claim 1, wherein one or more of the following is applied:

24. 24. A method according to any one of claims 16 to 23, wherein the first NNF is a model training logic function (MTLF) of a network data analysis function (NWDAF), and the second NNF is an analysis logic function (AnLF) of the NWDAF.

25. 25. The method of claim 24, The first message is a Nnwdaf_MLModelInfo_Request message; The method, wherein the second message is a response to a Nnwdaf_MLModelInfo_Request message.

26. 25. The method of claim 24, The first message is a Nnwdaf_MLModelProvision_Subscribe message; The method, wherein the second message is a Nnwdaf_MLModelProvision_Notify message.

27. 27. The method of claim 26, wherein the first message further includes one or more conditions for receiving the second message, the conditions including: performance metrics of the one or more ML models on which the analysis is based; and a threshold for each of said performance metrics; and a logical relationship between the performance metrics and the thresholds; and ML / AI framework constraints for executing the one or more ML models; and The method includes one or more of:

28. 28. The method of claim 27, the second message containing the single tuple is received only if the single ML model satisfies the one or more conditions contained in the first message; each of the plurality of tuples is received in the second message if and only if the corresponding ML model satisfies the one or more conditions included in the first message; The method according to claim 1, wherein one or more of the following is applied:

29. 29. The method of claim 28, wherein the one or more information elements for each tuple included in the second message further comprise: an identifier of a performance metric indicated in the first message; and the value of the performance metric determined for the corresponding ML model; and A method comprising:

30. 30. The method of claim 16, further comprising: using the analysis identifier to determine (1410) one or more ML model identifiers corresponding to the one or more ML models on which the analysis is based, and wherein the first message includes the determined one or more ML model identifiers.

31. A first network node or function (NNF) (810, 1508, 1700, 1902) configured for machine learning (ML) model management in a communication network (199, 200, 1506), comprising: The first NNF is implemented by an operably coupled communication interface circuit (1706, 1904) and a processing circuit (1702, 1904); The processing circuit and the communication interface circuit include: receiving a first message from a second NNF (820, 1508, 1700, 1902) of the communication network, the first message comprising: one or more machine learning (ML) model identifiers corresponding to one or more ML models maintained by the first NNF; and an identifier of the one or more ML model-based analyses; and transmitting a second message to the second NNF, the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including said analysis identifier and one or more information elements associated with a single ML model on which said analysis is based; A first NNF including one of:

32. A first NNF according to claim 31, wherein the processing circuit and the communication interface circuit are further configured to perform operations corresponding to any of the methods described in claims 2 to 15.

33. A first network node or function (NNF) (810, 1508, 1700, 1902) configured for machine learning (ML) model management in a communication network (199, 200, 1506), said first NNF comprising: receiving a first message from a second NNF (820, 1508, 1700, 1902) of the communication network, the first message comprising: one or more machine learning (ML) model identifiers corresponding to one or more ML models maintained by the first NNF; and an identifier of the one or more ML model-based analyses; and transmitting a second message to the second NNF, the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including said analysis identifier and one or more information elements associated with a single ML model on which said analysis is based; A first NNF including one of:

34. A first NNF according to claim 33, further configured to perform operations corresponding to the method according to any of claims 2 to 15.

35. A non-transitory computer readable medium (1704, 1904) storing computer executable instructions that, when executed by a processing circuit (1702, 1904) associated with a first network node or function (NNF) (810, 1508, 1700, 1902) configured for machine learning (ML) model management in a communications network (199, 200, 1506), configures the first NNF to perform operations corresponding to a method according to any of claims 1 to 15.

36. A computer program product (1704a, 1904a) comprising computer executable instructions that, when executed by a processing circuit (1702, 1904) associated with a first network node or function (NNF) (810, 1508, 1700, 1902) configured for machine learning (ML) model management in a communications network (199, 200, 1506), configures the first NNF to perform operations corresponding to the method according to any of claims 1 to 15.

37. A second network node or function (NNF) (820, 1508, 1700, 1902) configured for machine learning (ML) model management in a communication network (199, 200, 1506), comprising: The second NNF is implemented by an operably coupled communication interface circuit (1706, 1904) and a processing circuit (1702, 1904); The processing circuit and the communication interface circuit include: sending a first message to a first NNF (810, 1508, 1700, 1902) of the communication network, the first message comprising: one or more ML model identifiers corresponding to one or more ML models maintained by the first NNF; an identifier of the one or more ML model-based analyses; and receiving a second message from the first NNF, the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including said analysis identifier and one or more information elements associated with a single ML model on which said analysis is based; A second NNF including one of:

38. A second NNF according to claim 37, wherein the processing circuitry and the communications interface circuitry are further configured to perform operations corresponding to the methods of any of claims 17 to 30.

39. A second network node or function (NNF) (820, 1508, 1700, 1902) configured for machine learning (ML) model management in a communication network (199, 200, 1506), comprising: The second NFF further transmits a first message to a first NNF (810, 1508, 1700, 1902) of the communication network, the first message comprising: one or more ML model identifiers corresponding to one or more ML models maintained by the first NNF; an identifier of the one or more ML model-based analyses; and receiving a second message from the first NNF, the second message comprising: a plurality of tuples corresponding to a plurality of ML models on which the analysis is based, each tuple including a different one of the ML model identifiers and one or more information elements associated with the corresponding ML model; a single tuple including said analysis identifier and one or more information elements associated with a single ML model on which said analysis is based; A second NNF including one of:

40. A second NNF according to claim 39, further configured to perform operations corresponding to the method according to any of claims 17 to 30.

41. A non-transitory computer readable medium (1704, 1904) storing computer executable instructions that, when executed by a processing circuit (1702, 1904) associated with a second network node or function (NNF) (820, 1508, 1700, 1902) configured for machine learning (ML) model management in a communications network (199, 200, 1506), configures the second NNF to perform operations corresponding to the method of any one of claims 16 to 30.

42. A computer program product (1704a, 1904a) comprising computer executable instructions which, when executed by a processing circuit (1702, 1904) associated with a second network node or function (NNF) (820, 1508, 1700, 1902) configured for machine learning (ML) model management in a communications network (199, 200, 1506), configures the second NNF to perform operations corresponding to the method according to any of claims 16 to 30.

Images