First network function node, second network function node, and methods therein for handling a machine learning model

Abstract

A method performed by a second network function node for handling a Machine Learning (ML) model in a wireless communications network is provided. The second network function node receives (301) from a first network function node, the ML model and an ML profile for the ML model. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The second network function node obtains (302) verification data for verifying the ML model. The verification data comprises training data used for training the ML model and / or ground truth data for the ML model. The second network function node verifies (305) the ML model based on the ML profile and the verification data. The second network function node provides (306) the ML model and an indication of the verification of the ML model to a consumer network node.

Description

[0001] FIRST NETWORK FUNCTION NODE, SECOND NETWORK FUNCTION NODE, AND METHODS THEREIN FOR HANDLING A MACHINE LEARNING MODEL

[0002] TECHNICAL FIELD

[0003] The present disclosure relates generally to a first network function node, a second network function node, and methods therein for handling a Machine Learning (ML) model in a wireless communications network. The present disclosure may further relate to a system, a computer program product and a computer-readable storage medium.

[0004] BACKGROUND

[0005] In a typical wireless communications network, wireless devices, also known as wireless communication devices, mobile stations, stations (STA) and / or User Equipment (UE), communicate via a Wide Area Network or a Local Area Network such as a Wi-Fi network or a cellular network comprising a Radio Access Network (RAN) part and a Core Network (CN) part. The RAN covers a geographical area which is divided into service areas or cell areas, which may also be referred to as a beam or a beam group, with each service area or cell area being served by a radio network node such as a radio access node e.g., a Wi-Fi access point, a Base Station (BS) or a radio base station (RBS), which in some networks may also be denoted, for example, a Base Station (BS), a NodeB, eNodeB (eNB), or gNodeB (gNB) as denoted in Fifth Generation (5G) telecommunications. A service area or cell area is a geographical area where radio coverage is provided by the radio network node. The radio network node communicates over an air interface operating on a radio frequency with the wireless devices within the range of the radio network node.

[0006] 3rd Generation Partnership Project (3GPP) is the standardization body for specifying the standards for the cellular system evolution, e.g., including 3G, 4G, 5G and the future evolutions. Specifications for Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Packet System (EPS) have been completed within the 3GPP. In 4G also called a Fourth Generation (4G) network, EPS is core network and E-UTRA is radio access network. In 5G, 5GC is core network, NR is radio access network. As a continued network evolution, the new release of 3GPP specifies a 5G network also referred to as 5G New Radio (NR) and 5G Core (5GC).

[0007] Frequency bands for 5G NR are being separated into two different frequency ranges, Frequency Range 1 (FR1) and Frequency Range 2 (FR2). FR1 comprises sub-6 GHz frequency bands. Some of these bands are bands traditionally used by legacy standards but have been extended to cover potential new spectrum offerings from 410 MHz to 7125 MHz.FR2 comprises frequency bands from 24.25 GHz to 52.6 GHz. Bands in this millimeter wave range have shorter range but higher available bandwidth than bands in the FR1.

[0008] Multi-antenna techniques may significantly increase the data rates and reliability of a wireless communication system. For a wireless connection between a single user, such as UE, and a base station (BS), the performance is in particular improved if both the transmitter and the receiver are equipped with multiple antennas, which results in a Multiple-Input Multiple-Output (MIMO) communication channel. This may be referred to as Single-User (SU)-MIMO. In the scenario where MIMO techniques are used for the wireless connection between multiple users and the base station, MIMO enables the users to communicate with the base station simultaneously using the same time-frequency resources by spatially separating the users, which increases further the cell capacity. This may be referred to as Multi-User (MU)-MIMO. Note that MU-MIMO may benefit when each UE only has one antenna. The cell capacity can be increased linearly with respect to the number of antennas at the BS side. Due to that, more and more antennas are employed in BS. Such systems and / or related techniques are commonly referred to as massive MIMO.

[0009] In wireless communication systems, in particular relating to telecommunications, many aspects may need to be handled which may typically be suitable for ML models such as handling large amounts of data and / or procedures which may need to dynamically adapt to new scenarios.

[0010] With the advent of 5G and with regards to future developments such as with regards to 6G, networks are not only faster but also more complex, catering to a wider range of services for various verticals and devices. This complexity necessitates a more sophisticated approach to network management. A Network Data Analytics Function (NWDAF) is a Network Function (NF) integrated within the 5G core (5GC) network. An NWDAF enables more intelligent and efficient network operation by generating reports called Analytics Reports in response to requests from other NFs in the network. An Analytics Report typically comprises information about network performance, user behavior, and service quality in the form of statistics such as provided from past data or predictions to estimate the future state of the network. An NWDAF typically supports a wide range of analytics types, e.g., User Data Congestion, UE mobility, Network Performance, and NF Load.

[0011] To be able to generate Analytics Reports, an NWDAF needs to collect data from other 5GC NFs via an Ndf interface. The Ndf interface permits to subscribe or simply request the necessary monitoring information. Then, the NWDAF may train and maintain ML models and provision the ML models for inference to an ML model consumer. These tasks are handled by Logical Functions (LFs) within the NWDAF i.e. , Model Training Logical Function (MTLF) to train and maintain ML models and Analytics Logical Function (AnLF) to use ML models for inference. NWDAF can be implemented and deployed in 5GC as instances from differentvendors. In 3GPP standards, NWDAF instances may be arranged to share trained ML models with each other. As an example, 3GPP S2-2501444 discusses how ML models can be tested, how the MTLF and AnLF can co-operate for handling ML models, as well as how ML models can be selected based on a requested ML profile. While some solutions to ML model management are provided, issues in handling ML models with high security and efficiency remain.

[0012] SUMMARY

[0013] An object of embodiments herein is to improve efficiency and security of handling an ML model.

[0014] According to a first aspect of embodiments herein the object is achieved by a method performed by a second network function node for handling an ML model in a wireless communications network. The second network function node receives from a first network function node the ML model and an ML profile for the ML model. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The second network function node obtains verification data for verifying the ML model. The verification data comprises at least part of training data used for training the ML model and / or at least part of ground truth data for the ML model. Based on the ML profile and the verification data, the second network function node verifies the ML model. The second network function node provides to a consumer network node the ML model and an indication of the verification of the ML model.

[0015] According to a second aspect of embodiments herein, the object is achieved by a method performed by a first network function node for handling an ML model in a wireless communications network. The first network function node obtains the ML model and an ML profile for the ML model. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The first network function node transmits to a second network function node, the ML model and the ML profile.

[0016] According to a third aspect of embodiments herein, the object is achieved by a second network function node configured to handle an ML model in a wireless communications network. The second network function node is configured to receive from a first network function node, the ML model and an ML profile for the ML model. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The second network function node is configured to obtain verification data for verifying the ML model. The verification data comprises training data used for training the ML model and / or ground truth data for the ML model. The second network function node is configured to, basedon the ML profile and the verification data, verify the ML model. The second network function node is configured to provide to a consumer network node, the ML model and an indication of the verification of the ML model.

[0017] According to a fourth aspect of embodiments herein, the object is achieved by a first network function node configured to handle an ML model in a wireless communications network. The first network function node is configured to obtain the ML model and an ML profile for the ML model. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The first network function node is configured to transmit to a second network function node, the ML model and the ML profile.

[0018] According to a fifth aspect of embodiments herein, the object is achieved by a system arranged for handling an ML model in a wireless communications network. The system comprises a first network function node according to the third aspect, a second network function node according to the fourth aspect, and a consumer network node.

[0019] According to another aspect of embodiments herein the object is achieved by a computer program product comprising instructions, which, when executed on at least one processor, cause the at least one processor to carry out the method according to the first aspect or the second aspect.

[0020] According to another aspect of embodiments herein the object is achieved by a computer-readable storage medium, having stored thereon a computer program product comprising instructions which, when executed on at least one processor, cause the at least one processor to carry out the method according to the first aspect or the second aspect.

[0021] Since the ML model and the ML profile are transmitted from the first network function node to the second network function node, the second network function node can verify the ML model based on the ML profile. This allows that when the ML model is provided to the consumer network node, the indication of the verification of the ML model can also be provided. This means that the first network function node enables that the ML model is verified with respect to the ML profile. The second network function node performs said verification and enables the consumer network node to be able to receive information of the verification of the ML model. A consequence of the verification of the ML model is a more secure and efficient wireless communications network. This is because the consumer network node can obtain information of the verification, and its subsequent actions may be based on the indication of the verification of the ML model.

[0022] BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Examples of embodiments herein are described in more detail with reference to the accompanying drawings, according to the following description.Fig. 1 illustrates a schematic diagram illustrating a non-limiting example of a wireless communications network, according to some embodiments herein.

[0024] Fig. 2 illustrates a schematic diagram illustrating a non-limiting example of an NWDAF as part of a network function node, according to some embodiments herein. Fig. 3 illustrates a flowchart of a method performed by a second network function node, according to some embodiments herein.

[0025] Fig. 4 illustrates a flowchart of a method performed by a first network function node, according to some embodiments herein.

[0026] Fig. 5 illustrates a combined sequence diagram and flow chart depicting scenarios according to some embodiments herein.

[0027] Fig. 6 illustrates a schematic diagram depicting an example scenario according to some embodiments herein.

[0028] Fig. 7 illustrates a schematic diagram depicting an example scenario according to some embodiments herein.

[0029] Fig. 8 illustrates a schematic diagram depicting an example scenario according to some embodiments herein.

[0030] Fig. 9 illustrates a schematic diagram depicting an example scenario according to some embodiments herein.

[0031] Fig. 10 illustrates a schematic block diagram illustrating a non-limiting example of a second network function node, according to some embodiments herein.

[0032] Fig. 11 illustrates a schematic block diagram illustrating a non-limiting example of a first network function node, according to some embodiments herein.

[0033] Fig. 12 illustrates a schematic block diagram illustrating a non-limiting example of a system, according to some embodiments herein.

[0034] DETAILED DESCRIPTION

[0035] As part of the development of embodiments herein, one or more issues with existing technology will first be identified and discussed.

[0036] Horizontal Federated Learning (HFL) to train ML models by different NWDAF clients is standardized and is performed in various context. In general, the same may be true for any context of training ML in a federated and / or horizontal manner. During HFL, clients and server, where both are typically NWDAF instances, in particular MTLF instances, may share ML models. In this context, the server may measure the quality such as accuracy of local ML models and will inform the client if further training is needed. Local ML models can be aggregated by the server in each iteration and sent back to the clients for further training until a global ML model i.e., the model that is aggregated at the server side, converges. Since clients and server can be instances from different vendors, it may be crucial to ensure that anML model that is shared, or at least transmitted between different entities, fulfils certain requirements, e.g., security and integrity, and that consistency is achieved. However, as part of developing embodiments herein, it has been identified that requirements or consistencies of ML models that are shared or transmitted between entities in a network are not necessarily indicated to an entity receiving the model and / or cannot always be trusted, either due to unintentional issues and / or due to malicious behaviour. This may for example relate to a use case where an NWDAF comprising an MTLF sends a trained ML model to another NWDAF comprising an AnLF.

[0037] ML model quality monitoring may be performed with regards to NWDAF, in particular relating to accuracy, e.g., as described in 3GPP TS 23.288. The quality information can for example be provided to a consumer node and / or can be used for an internal process of an NWDAF. The quality may be computed by using ground truth data, e.g. correct label or output used to train and evaluate a model. In addition to an NWDAF comprising an MTLF, the quality computation can also be performed by an NWDAF containing AnLF. The NWDAF comprising the AnLF can use quality feedback from consumes of an ML mode. However, while some monitoring of models can be performed, ML models which are shared between entities, e.g., different NWDAF, it has been identified as part of developing embodiments herein that such ML models are not verified, meaning that there may be a claim to their respective properties which may be incorrect. Unverified ML models can cause security and / or performance issues. Some examples of the issues relating to verification of ML models have been identified and are listed below.

[0038] Incorrect input-output relation. There is no verification of that a shared ML model accepts correct input parameters and / or generates analytics reports in a correct format.

[0039] Incorrect weight of input parameters. There is no verification of importance of input parameters in an ML model. The importance of input may be weights of the input. This may be referred to as an explainability analysis and wherein a verification may relate to measuring weights of inputs and to verify that none of the inputs are ignored nor have very low effect on the results.

[0040] Inaccurate confidence level value. There is no verification of confidence level of an ML model. When NWDAF generates Analytics Reports, a confidence level value may need to be set which reflect how accurate the prediction was.

[0041] Vulnerability against possible model inversion attacks against an ML model

[0042] There are no verifications that the ML model is not vulnerable and / or part of an inversion attack. An ML model inversion attack may then be performed by an adversary which can be either an NF or a 3rd party Application Function (AF) that contacts NWDAF through a Network Exposure Function (NEF). The purpose of an attacker may be to get information about a training data set that is used to train an ML model. Such information can comprise for exampleUE data that are subject to privacy and shall not be exposed to any NF or AP. A verification can be made in cooperation with an NWDAF, typically comprising an AnLF to make sure that privacy of the ML model is not compromised during the inference and that no sensitive information about the training data is exposed.

[0043] Furthermore, without ML model verification, ML models can cause degradation in network performance in the sense that decisions may be made on inaccurate estimations or predictions. In particular, a consumer such as an NF may trigger actions based on an inaccurate or unreliable prediction made by an NWDAF.

[0044] An aim of some embodiments herein is to ensure the ML models are verified such that decisions can be made accordingly in a safe and efficient manner. In particular embodiments, this may relate to ensure that the NWDAF in 5GC can be able to verify ML models, e.g., with respect to functionality, correctness, and performance, in particular relating to ML models which have been shared among different NWDAF instances. The process of verification may be performed by a newly introduce logical function of the NWDAF referred to as a Verification Logical Function (VLF), e.g., which may act based on requests from an MTLF.

[0045] As mentioned above, an object of embodiments herein is to improve security and efficiency of handling an ML model. Examples herein may according to some embodiments bring some benefits to a communication network, such as:

[0046] - Increasing security when ML models are shared, in particular among NWDAF instances and / or in particular in a multi-vendor environment.

[0047] - Ensuring a possibility and flexibility, e.g., for operators and / or vendors, to specify expectations and requirements over the ML models, in particular when the ML models are shared and a verification process is to be performed. Hence, only ML models that fulfill the requirements may be used for prediction, or at least a consumer will be aware of any unverified ML models and can act accordingly. This aspect may be valuable for different use cases such as when one or more NWDAF instances are running in critical environments which may need stricter requirements than instances running in other environments.

[0048] - Ensuring a possibility and flexibility, e.g., for an ML model consumer to have access to a latest assessment report of an ML model before using it. The assessment report may indicate the verification of the ML model.

[0049] - Preventing or at least informing of a usage of incorrect, inconsistent, and inaccurate ML models using the verification of some embodiments herein, in particular comprising verifying ML model integrity.

[0050] - Ensuring that relevant and correct actions can be triggered by consumers of ML models by providing an indication of the verification of the ML models which thereby can avoid network degradation and / or service outages.Embodiments herein relate to wireless communications networks in general, and are particularly applicable for telecommunications networks. Fig.1 is a schematic overview depicting a wireless communications network 100. While some particular aspects are discussed below, the wireless communications network 100 may be any suitable network, even a network using wired technology. The wireless communications network 100 may comprise one or more RANs and one or more CNs. The wireless communications network 100 may be a 5G system, or a newer system supporting similar and / or different functionality, such as, for example, a Sixth Generation (6G) system. In some examples, the wireless communications network may support, additionally or alternatively, a Long-Term Evolution (LTE) network and may support other technologies such as a for example, LTE Frequency Division Duplex (FDD), LTE Time Division Duplex (TDD), LTE Half-Duplex Frequency Division Duplex (HD-FDD), and LTE operating in an unlicensed band. The telecommunications system may also support other technologies, such as Wideband Code Division Multiple Access (WCDMA), Universal Mobile Telecommunications System Terrestrial Radio Access (UTRA) TDD, Global System for Mobile communications (GSM) network, GSM / Enhanced Data Rate for GSM Evolution (EDGE) Radio Access Network (GERAN) network, Ultra-Mobile Broadband (UMB), EDGE network, network comprising any combination of Radio Access Technologies (RATs) such as e.g. Multi-Standard Radio (MSR) base stations, multi-RAT base stations etc., any 3rd Generation Partnership Project (3GPP) cellular network, Wireless Local Area Network / s (WLAN) or WiFi network / s, Worldwide Interoperability for Microwave Access (WiMax), IEEE 802.15.4-based low-power short-range networks such as IPv6 over Low-Power Wireless Personal Area Networks (6LowPAN), Zigbee, Z-Wave, Bluetooth Low Energy (BLE), or any cellular network or system. The telecommunications system may, for example support a Low Power Wide Area Network (LPWAN). LPWAN technologies may comprise Long Range physical layer protocol (LoRa), Haystack, SigFox, LTE-M, and Narrow-Band loT (NB-loT).

[0051] A number of radio network nodes may operate in the wireless communications network 100. Radio network nodes may provide radio coverage in a number of cells which may also be referred to as a beam or a beam group of beams. Radio network nodes as used herein may be any of a NG-RAN node, a transmission and reception point e.g. a base station, a radio access network node such as a Wireless Local Area Network (WLAN) access point or an Access Point Station (AP STA), an access controller, a base station, e.g. a radio base station such as a NodeB, an evolved Node B (eNB, eNode B), a gNB, a base transceiver station, a radio remote unit, an Access Point Base Station, a base station router, a transmission arrangement of a radio base station, a stand-alone access point, a network controlled repeater or any other network unit capable of communicating with a wireless device within the service area served by the radio network node depending e.g. on the first radio access technology and terminology used. Radio network nodes as used herein may be referred to as serving radionetwork nodes and communicates with one or more UEs with Downlink (DL) transmissions and Uplink (UL) transmissions from the one or more UEs.

[0052] In some examples, the wireless communications network 100 may comprise an access network, such as a radio access network (RAN), and / or a core network (CN), which may include one or more core network nodes. The access network may include one or more access network nodes, e.g., which may be radio network nodes as described above, or any other similar 3rd Generation Partnership Project (3GPP) access nodes or non-3GPP access points.

[0053] Moreover, as will be appreciated by those of skill in the art, a radio network node or a core network node is not necessarily limited to an implementation in which a radio portion and a baseband portion are supplied and integrated by a single vendor. Thus, it will be understood that radio network nodes and / or core network nodes may include disaggregated implementations or portions thereof.

[0054] In some embodiments, the wireless communications network may include one or more Open-RAN (ORAN) network nodes. An ORAN network node may be understood as a node in the telecommunications network that may support an ORAN specification, e.g., a specification published by the O-RAN Alliance, or any similar organization, and may operate alone or together with other nodes to implement one or more functionalities of any node in the telecommunications network, including one or more network nodes and / or core network nodes.

[0055] Examples of an ORAN network node include an open radio unit (O-RU), an open distributed unit (O-DU), an open central unit (O-CU), including an O-CU control plane (O-CU-CP) or an O-CU user plane (O-CU-UP), a RAN intelligent controller, near-real time or non-real time, hosting software or software plug-ins, such as a near-real time control application, e.g., xApp, or a non-real time control application, e.g., rApp, or any combination thereof, the adjective “open” designating support of an ORAN specification. ORAN network nodes may support a specification by, for example, supporting an interface defined by the ORAN specification, such as an A1, F1, W1, E1, E2, X2, Xn interface, an open fronthaul user plane interface, or an open fronthaul management plane interface. Moreover, an ORAN network node may be a logical node in a physical node. Furthermore, an ORAN network node may be implemented in a virtualization environment, in which one or more network functions may be virtualized. For example, the virtualization environment may include an O-Cloud computing platform orchestrated by a Service Management and Orchestration Framework via an O-2 interface defined by the O-RAN Alliance or comparable technologies.

[0056] Radio network nodes as used herein may facilitate direct or indirect connection of one or more UEs such as by connecting the one or more UEs to the CN over one or more wireless connections or wired connections.In the wireless communications network 100, one or more wireless devices operate, such as one or more UEs. The one or more UEs may respectively also be known as a e.g., device, wireless device, mobile terminal, wireless terminal and / or mobile station, mobile telephone, cellular telephone, or laptop with wireless capability, an Internet of Things (loT) device, or a Customer Premises Equipment (CPE), just to mention some further examples. The one or more UEs may for example be, portable, pocket-storable, hand-held, computer-comprised, or a vehicle-mounted mobile device, enabled to communicate voice and / or data, via a RAN, with another entity, such as a server, a laptop, a Personal Digital Assistant (PDA), or a tablet, a Machine-to-Machine (M2M) device, an Internet of Things (loT) device, e.g., a sensor or a camera, a device equipped with a wireless interface, such as a printer or a file storage device, modem, Laptop Embedded Equipped (LEE), Laptop Mounted Equipment (LME), USB dongles, CPE or any other radio network unit capable of communicating over a radio link in the wireless communications network 100. The one or more UEs may be wireless, i.e. , it may be enabled to communicate wirelessly in the wireless communications network 100 and, in some particular examples, may be able support transmission using beamforming. The communication may be performed e.g., between two devices, between a device and a radio network node, and / or between a device and a server. The communication may be performed e.g., via a RAN and possibly one or more CNs, comprised, respectively, within the wireless communications network 100.

[0057] In the wireless communications network 100 one or more network function nodes operate such as the first network function node 111 and the second network function node 112 which respectively may be any suitable node such as a CN node, but may also be a RAN node, ORAN node, or even a UE.

[0058] The first network function node 111 and / or the second network function node 112 may be configured to handle an ML model, e.g., wherein the ML model is arranged to be transmitted by the first network function node 111 to the second network function node 112 such that the ML model is shared between the first network function node 111 and the second network function node 112. The first network function node 111 and / or the second network function node 112 may respectively comprise one or more NF configured to perform at least part of embodiments herein. In particular, the first network function node 111 and / or the second network function node 112 may respectively comprise an NDWAF, e.g., which may be configured to perform parts or all of some embodiments herein.

[0059] Embodiments herein may particularly relate to verifying the ML model based on an ML profile. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The ML profile may be generated by the first network function node 111.In the wireless communications network 100 one or more consumer network nodes may operate such as the consumer network node 113. The consumer network node 113 may be any suitable node arranged to receive the ML model from the second network function node 112, e.g., upon request. The consumer network node 113 may be any suitable node such as a CN node, but may also be a RAN node, ORAN node, or even a UE.

[0060] Methods herein may be performed by a network node, e.g. the first or second network function node 111, 112. As an alternative, a Distributed Node (DN) and functionality, e.g. comprised in a cloud resource 190 such as a server as shown in Fig. 1, may be used for performing or partly performing the methods herein.

[0061] The above-described problem is addressed in a number of embodiments, some of which may be seen as alternatives, while some may be used in combination. Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject matter disclosed herein, the disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

[0062] Fig. 2 illustrates an example embodiment of an NWDAF, e.g., an NWDAF as part of the first network function node 111 and / or as part of the second network function node 112. The NWDAF of the first network function node 111 may comprise any one or more of: a first MTLF 211 , a first AnLF 221 , a first VLF 231. The NWDAF of the second network function node 112 may comprise any one or more of: a second MTLF 212, a second AnLF 222, a second VLF 232.

[0063] The NWDAF of any of the first network function node 111 and / or the second network function node 112 may receive a request 201 for an ML model, e.g., by the consumer network node 113 and / or any suitable consumer such as an NF. The ML model may be transmitted by the NWDAF of the first network function node 111 to the NWDAF of the second network function node 112. In some embodiments herein, the second VLF 232 may verify the ML model, optionally by one or more policies as provided from a policy providing entity 240 such as a vendor or operator. The verification is based on the ML profile, e.g., transmitted by the NWDAF of the first network function node 111 to the NWDAF of the second network function node 112. The NWDAF of the second network function node 112 may provide 202 the verified ML model and an indication of the verification of the ML model to the consumer network node 113 and / or any to entity requesting the ML model.

[0064] Some embodiments herein may be summarized by comprising any one or more of the following features, and may be performed in any suitable order, such as in the order presented below:Signaling among VLF, AnLF and MTLF and a procedure of triggering verification of the ML model in VLF, e.g., the second VLF 232.

[0065] Generating the ML profile of the ML model. The ML profile may also be referred to as an ML Performance Profile (MPP). The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The ML profile may be generated by an instance of NWDAF, e.g., the first MTLF 211, which may be the ML model owner of the ML model.

[0066] Delivering of the ML model to another instance of an NWDAF on the second network function node 112, e.g., to the second MTLF 212, when the ML model is shared between the first network function node 111 and the second network function node 112.

[0067] - A procedure of providing and possibly generating an indication of a verification of the ML model, e.g., as part of a Model Assessment Report (MAR) when a receiver of an ML model, e.g., the second network function node 112 or the NWDAF therein, provides the ML model to a consumer such as the consumer network node 113. Training an ML profile classifier for estimating a level of trust of the ML profile, e.g., such that the verification of the ML mode can be based on whether or not to trust the ML profile or if other verification may be more suitable, e.g., by performing a separate ML model accuracy measurement.

[0068] Training an ML model for policy generation, which may thereby eliminate or reduce a need for a user, operator or vendor to manually submit policies or heuristics for the verification and / or for providing the indication of the verification.

[0069] Embodiments herein may relate to verification of the ML model, in particular when the ML model is shared between instances of NWDAF, e.g., between the first network function node 111 and the second network function node 112. Embodiments herein may utilize a separate and dedicated VFL inside NWDAF for ML model verification. The process of verification may comprise any of:

[0070] • To verify that a correct confidence level is reported in the ML profile.

[0071] • Ensure that a correct input / output relation exists between parameters of the ML model and generated analytics reports.

[0072] • To verify that the ML model is not vulnerable against model inversion attacks and that no confidential information is exposed.

[0073] Embodiments herein increase robustness in the network by minimizing the risk that a network entity might trigger actions based on incorrect or inaccurate prediction of the network state.Without ML model verification, ML models can cause degradation in network’s performance in the sense that an NF would trigger actions based on an inaccurate or unreliable prediction made by NWDAF.

[0074] Advantageously, ML model verification ensures that a shared ML model will operate as expected by an ML model consumer and predictions are according to specifications and standards.

[0075] The above-mentioned verification options are merely non-limiting and non-exhaustive examples of verification procedures.

[0076] Fig. 3 illustrates example embodiments of a method performed by the second network function node 112 for handling the ML model in the wireless communications network 100. The method may be a computer-implemented method. The method may comprise any one or more of the following Actions, which may be taken in any suitable order. The method may repeat any of the following Actions in any suitable order. Optional Actions may be indicated by dashed boxes in Fig. 3. In some embodiments, some or all of the following Actions may be performed by an NWDAF of the second network function node 112, in particular using the second MTLF 212 and / or the second VLF 232.

[0077] Action 300. The second network function node 112 may receive from the consumer network node 113, a request for providing an ML model and the indication of the verification of the ML model. The request may be indicative of the ML model. The request may be indicative of one or more requested properties of the ML model.

[0078] Action 301. The second network function node 112 receives from the first network function node 111, the ML model and the ML profile for the ML model. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model.

[0079] In some embodiments, the one or more properties associated with the one or more conditions of the ML profile may comprise at least one of:

[0080] - expected or required accuracy of the ML model,

[0081] - expected or required one or more algorithms of the ML model,

[0082] - expected or required type of training used by the ML model,

[0083] - expected or required confidence of the ML model,

[0084] - one or more constraints of the ML model,

[0085] - one or more protocols and / or rules expected or required to be used by the ML model, and

[0086] - one or more weights of the ML model.

[0087] The one or more weights of the ML model may relate to one or more internal weights of a structure of the ML model, such as weights of a neural network of the ML model and / or weights applied to one or more layers, one or more inputs, one or more outputs, and / or one or more intermediate values, of the ML model.The ML profile may additionally or alternatively be indicative of one or more tests for verifying at least one of the one or more properties indicated by the ML profile.

[0088] Additionally or alternatively, the ML profile may indicate an initial accuracy measurement of the ML model and / or information about data sources for the training data used for training the ML model.

[0089] The Action may in some embodiments further comprise or be performed subsequent to, requesting e.g., from the first network function node 111 to provide an ML model, e.g., as a response to Action 300.

[0090] Action 302. The second network function node 112 obtains verification data for verifying the ML model, wherein the verification data comprises training data used for training the ML model and / or ground truth data for the ML model.

[0091] Action 303. The second network function node 112 may obtain a trust indication indicating a level of trust of the ML profile.

[0092] In some embodiments, the second network function node 112 may obtain the trust indication indicating the level of trust by estimating the trust indication based on the ML profile and based on at least one of:

[0093] - an identifier of an entity providing and / or generating the ML profile and / or ML model, - information indicative of one or more preset properties associated with the ML model, and

[0094] - an indication of one or more use cases of the ML model.

[0095] In some embodiments, the second network function node 112 may estimate the trust indication based on any suitable classifier, static or trained.

[0096] In some embodiments, the trust indication indicating the level of trust is estimated using an ML profile classifier trained based on training data comprising one or more training ML profiles of one or more training ML models. The ML profile classifier may further be trained on at least one of:

[0097] one or more training instance identifiers, e.g., an identity of an entity associated with the respective ML model or ML profile,

[0098] - one or more preset properties associated with the one or more training ML models, and - one or more use cases of the one or more training ML models.

[0099] Action 304. The second network function node 112 may obtain a policy indication indicative of one or more verification procedures and / or heuristics for performing the verification of the ML model.

[0100] Action 305. The second network function node 112 verifies the ML model based on the ML profile and the verification data. The second network function node 112 verifies the ML model further based on the obtained policy indication.In some embodiments, the second network function node 112 may verify the ML model based on the level of trust of the ML profile.

[0101] In some embodiments, when the level of trust indicates that the ML profile can be trusted, the second network function node 112 may verify the ML model by using the verification data to verify whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile.

[0102] In some embodiments, when the level of trust indicates that the ML profile cannot be trusted, the second network function node 112 may verify the ML model by using the verification data to verify whether or not the ML model adheres to one or more properties indicated by a policy indication obtained for verifying the ML model. The policy indication may be indicative of one or more verification procedures and / or heuristics for performing the verification of the ML model.

[0103] The verification may be performed by the second VLF 232 and / or the second MTLF 212. For example, the policy indication may indicate that the ML model shall be required to reach one or more metrics such as one or more level of accuracy metrics. As an example, the one or more metrics may comprise any of: accuracy, recall, F1 score, or a combination thereof. Many other metrics may also be suitable.

[0104] The policy indication may be generated based on the ML profile, the level of trust associated with the ML profile, and a predefined policy model. Alternatively, the policy indication may be the policy indication obtained in Action 304. In some embodiments herein, the verification of the ML model may be based on a policy indication as obtained in Action 304 and another policy indication as generated based on the ML profile, the level of trust associated with the ML profile, and a predefined policy model. The predefined policy model may be a static model, but may also be a trained ML model. When being a trained policy model, the predefined policy model may have been trained on at least one of:

[0105] - one or more training policy templates,

[0106] - one or more training ML profiles, and

[0107] - level of trust of the one or more training ML profiles.

[0108] As an example, in some embodiments, if the level of trust indicates that the ML profile cannot be trusted, the verification may not use the ML profile for the verification. If the level of trust indicates that the ML profile can partially be trusted, the verification may partially use the ML profile for the verification, e.g., certain one or more predefined properties of the ML profile and / or one or more properties of the ML profile which are established as trusted. If the level of trust indicates that the ML profile can be trusted, e.g., fully trusted, the verification may use the ML profile for the verification, e.g., by verifying that the ML model adheres to the one or more properties of the ML profile.In some embodiments, the second network function node 112 may verify the ML model by verifying, using the verification data, whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile and fulfils the heuristics of the obtained or generated policy indication. In other words, the verification may be a combination of verification against the ML profile which may be at least partly trusted and any suitable policy indication, e.g., generated or obtained such as supplied by a user, vendor, operator, or predefined.

[0109] According to some embodiments, the second network function node 112 may verify the ML model by verifying whether the ML model adheres to the one or more requested properties of the ML model, e.g., as indicated by the request of the consumer network node 113 as discussed in Action 300.

[0110] In some embodiments, the second network function node 112 may verify the ML model by at least one of:

[0111] - testing the ML model according to tests indicated by the ML profile,

[0112] - verifying whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile,

[0113] - verifying whether or not a confidence value and / or an accuracy of the one or more properties associated with the one or more conditions of the ML profile is adhered to by the ML model when verified with the verification data,

[0114] - verifying whether or not the ML model adheres to one or more properties indicated by one or more obtained policies,

[0115] - verifying an input-output relationship of the ML model,

[0116] - verifying one or more weights of one or more input parameters of the ML model, and - verifying whether or not the ML model is vulnerable to an inversion attack.

[0117] In other words, the verification of the ML model may comprise any suitable verification procedures and may be based on information in the ML profile to perform said verification.

[0118] It shall be noted that all verification options mentioned above may be combined in any suitable manner, or be performed in isolation.

[0119] Action 306. The second network function node 112 provides to the consumer network node 113, the ML model and an indication of the verification of the ML model. The indication of the verification of the ML model may be provided as part of an assessment report, in particular a MAR. The indication of the verification of the ML model may indicate whether or not the verification of the ML model was successful and / or which of the one or more properties could be verified. The indication of the verification of the ML model may comprise which verification procedures were conducted as part of the verification procedure, e.g., Action 305.

[0120] In some embodiments, providing the ML model and the indication of the verification of the ML model is performed based on, and / or in response to receiving the request.For example, the request may indicate to provide an ML model which verifies with respect to the one or more requested properties of Action 300. The method herein and / or Actions herein may thereby be performed for multiple ML models, and the ML model may be provided if it is verified as in Action 305, additionally or alternatively, multiple models may be provided if they all are verified. In some embodiments, the ML model is always provided, even if unverified. In some embodiments, if a plurality of ML models verifies, i.e. , such as adheres to the one or more properties of the ML profile and / or the policy indication and / or the one or more requested properties, then a selection of the ML model may be made based on any suitable heuristics and / or a Use Case Context (UCC), e.g., provided by the request in Action 300, such as the highest quality e.g. the highest accuracy and / or the best suited ML model for the use case. The UCC may describe the use case of the ML model to help an NWDAF or MTLF to choose the best ML model when it has multiple options for an analytics report by matching UCC to the ML models that are available.

[0121] The indication of the verification of the ML model provided to the consumer network node 113 may further indicate whether or not the ML model adheres to the one or more requested properties.

[0122] Fig. 4 illustrates example embodiments of a method performed by the first network function node 111 for handling the ML model in the wireless communications network 100. The method may be a computer-implemented method. The method may comprise any one or more of the following Actions, which may be taken in any suitable order. The method may repeat any of the following Actions in any suitable order. Optional Actions may be indicated by dashed boxes in Fig. 4. In some embodiments, some or all of the following Actions may be performed by an NWDAF of the first network function node 111, in particular using the first MTLF 211.

[0123] Action 400. The first network function node 111 obtains the ML model and the ML profile for the ML model. Obtaining the ML model may comprise training the ML model.

[0124] Obtaining the ML profile may comprise generating the ML profile. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model.

[0125] The Action may be performed in response to a request e.g., from the second network function node 112 to provide the ML model.

[0126] Action 401. The first network function node 111 transmits to the second network function node 112, the ML model and the ML profile. The transmission may correspond to the receival described in Action 301.

[0127] Action 402. The first network function node 111 may trigger the second network function node 112 to obtain verification data, and to verify the ML model based on the ML profile and the verification data. For example, the transmission of the ML model and the ML profile may trigger the verification of the ML model, e.g., as discussed with respect to Fig. 3. Alternativelytriggering the verification of the ML model may comprise transmitting a triggering message to the second network function node 112 to verify the ML model based on the ML profile and the verification data.

[0128] The triggering of Action 402 may in particular trigger the second network function node 112 to perform any one or more of Actions 301-306.

[0129] Fig. 5 illustrates a combined sequence diagram and flow chart depicting scenarios according to some embodiments herein. The Actions of Fig. 5 may relate to and / or be combined with the Actions of Figs 3-4 in any suitable manner. The Actions of Fig. 5 may be performed by different LFs of the NWDAF of the first network function node 111 and the second network function node 112, respectively. Furthermore, communication may also be performed with the consumer network node 113 e.g., for requesting and receiving the ML model, a training data node 500, e.g., an Analytics Data Repository Function (ADRF) or Data Collection Coordination Function (DCCF), and any suitable entity of the wireless communications network 100, e.g., for providing ground truth data.

[0130] Action 501. The first network function node 111 may collect data to train an M L model for first analytics. The first analytics may comprise one or more predictions of a future network state of the wireless communications network 100 and / or statistics comprising information about of a historical network state of the wireless communications network 100.

[0131] The Action may in particular be performed by the NWDAF of the first network function node 111, in particular the first MTLF 211 , e.g., as part of Action 400.

[0132] Action 502. The first network function node 111 may trigger a training process to prepare the ML model for first analytics, e.g., as part of Action 400.

[0133] The Action may in particular be performed by the NWDAF of the first network function node 111, in particular the first MTLF 211.

[0134] Action 503. The first network function node 111 may generate the ML profile for the trained ML model, e.g., as part of Action 400.

[0135] The Action may in particular be performed by the NWDAF of the first network function node 111, in particular the first MTLF 211.

[0136] The ML profile may comprise or be accompanied by some testing information and / or accreditation, e.g., indicating one or more properties of the ML model and / or how to verify them.

[0137] The ML profile may indicate a confidence for the ML model. The ML profile may indicate how to test for proving said confidence.

[0138] Action 504. The first network function node 111 may share the prepared ML model, e.g., as part of Actions 301, 401, for the first analytics with the second network function node 112, e.g., the NWDAF of the second network function node 112, in particular the second MTLF 212. The Action may in particular be performed by the NWDAF of the first network functionnode 111, in particular the first MTLF 211. The ML profile may also be shared with or transmitted to the second network function node 112. The ML profile may indicate one or more properties, e.g., accuracy parameters and information about accuracy assessment of the ML model. The assessment of the one or more properties may be performed by first network function node 111 e.g., the NWDAF of the first network function node 111, in particular the first MTLF 211, e.g., as a self-assessment of the ML model.

[0139] Action 505. The second network function node 112 may obtain, e.g., fetch, some or all training data used by the first network function node 111 to train the ML model, e.g., as part of Action 302. The Action may in particular be performed by the NWDAF of the second network function node 111, in particular the second MTLF 212. The training data may be obtained from the training data node 500 and / or from any other suitable entity.

[0140] Action 506. The second network function node 112 may obtain, e.g., collect, some or all relevant ground truth data from the wireless network communications network 100, e.g., as part of Action 302. The ground truth data may be obtained from any one or more suitable entities, e.g., network nodes, UEs, sensors, etc. The Action may in particular be performed by the NWDAF of the second network function node 111, in particular the second MTLF 212.

[0141] Action 507. The second network function node 112 may trigger a verification process according to any submitted or generated policy / rules, e.g., as indicated by the ML profile and / or as indicated by the obtained or generated policy indication. The Action may correspond to Action 305. The Action may in particular be performed by the NWDAF of the second network function node 111, in particular the second MTLF 212 and / or the second VLF 232.

[0142] The verification may comprise any suitable verification or test as mentioned by embodiments herein, e.g., including any one or more of:

[0143] - Input-output relation may be verified which means that the ML model may accept one or more expected input parameters and / or may generate an analytics report, i.e. , the indication of the verification as Action 306, with expected data.

[0144] - An explainability analysis may be performed to verify that no input parameter is excluded when the analytics report i.e., the indication of the verification as in Action 306, is generated.

[0145] - The NWDAF of the second network function node 111, in particular the second MTLF 212 may be arranged to define a sorted list of input parameters of the ML model based on the importance of each parameter and compare the list with a result of explainability analysis over the ML model. The NWDAF of the second network function node 111, in particular the second MTLF 212 may then decide whether or not to use or provide the ML model, based on whether the order of the input parameters differs from the reference list using an internal logic.- A reported confidence level value of the ML model may be evaluated to ensure that the confidence level value correctly reflects the accuracy of the generated analytics report, i.e., the indication of the verification as in Action 306.

[0146] The ML model may be analyzed for reverse attack vulnerability.

[0147] Action 508. The second network function node 112 may generate an ML model Assessment Report (MAR). The MAR may be the indication of the verification as in Action 306. Various parameters e.g., UCC, Interoperability, etc., may be used to generate MAR. The Action may in particular be performed by the NWDAF of the second network function node 111, in particular the second MTLF 212.

[0148] Action 509. The consumer network node 113 may subscribe to receive an ML model, e.g., as part of Action 300. Alternatively, the consumer network node 113 may subscribe to receive any ML model comprising certain properties, e.g., as part of a request such as in Action 300.

[0149] Action 510. The second network function node 112 may provide the ML model and optionally the MAR to the consumer network node 113. The MAR can be used to select among several requested models, e.g., if the consumer network node 113 has subscribed to multiple models.

[0150] In the Actions above, the verification, e.g., as performed by the second VLF 232, such as in Action 305 or 507 may use the ML profile as a baseline for verification. However, the verification may depend on an implementation of the VLF to trust the ML profile, to partially trust the ML profile, or to completely discard the information in the ML profile. An ML model Interoperability Information may comprise vendor-specific information e.g., ML model format, execution environment, algorithm, etc., and may be sent to an ML model consumer such as the second network function node 112 during an ML model provisioning. Such information or other information as presented in embodiments, e.g., Action 305, herein can be used to estimate the level of trust e.g., by the second VLF 232. In some embodiments herein and for some of the Actions above, ML model interoperability is optionally sent to the ML model owner, e.g., the first function network node 111 or the second function network node 112, by an ML model consumer, e.g., the second network function node 112 or the consumer network node 113, as a request to provision the ML model by indicating a need for an ML model which supports a specific algorithm, format, etc.

[0151] Dedicated Logical Function -VLF

[0152] The ML model verification of embodiments herein inside the NWDAF may be based on the VLF, in particular the second VLF 232. The second VLF 232 may be responsible to perform verification, e.g., as in Actions 305 or 507, based on a request from the second MTLF 212. The result may be sent to the second MTLF 212. The second MTLF 212 may send a request to the VLF either when the M L model is shared by the first network function node 111or to postpone the process until when the ML model is being provisioned, e.g., requested by the consumer network node 113. Policies may be set as the ML model requirements and expectations and stored in the second VLF 232, e.g., as obtained or generated by the policy indication with respect to the Actions illustrated by Fig. 3. An operator, user, or vendor may submit policies which indicate the requirements and expectations for an ML model.

[0153] The second VLF 232 may receive the ML profile and optionally use it as a reference for the ML model verification, e.g., as part of Action 305. Moreover, the second VLF 232 may optionally refer to the ML profile when the MAR is generated, e.g., the indication of the verification as in Action 306, or the information of the ML profile may be discarded, e.g., if not trustworthy.

[0154] Fig. 6 illustrates an example architecture for internal components in a VLF, e.g. the second VLF 232. Each illustrated component may be seen as an optional non-limiting example component which can be varied in any suitable manner. In the presented architecture, a database 601 may be used to store collected data for verification purposes as well as submitted policies / rules.

[0155] Policies may be submitted 603 by an operator, user, or vendor and may comprise of requirements of the ML model such as one or more verification procedures and / or heuristics for performing the verification of the ML model. The policies may be indicated by the policy indication discussed in previous embodiments. The policies may indicate an ML model age, an accuracy threshold, an average deviation confidence level threshold, etc.

[0156] The ML profile may be received 602 as values and may be stored in the database 601 and optionally used in confidence level and policy matching units. For instance, if the ML profile describes accuracy measurement values, the policy matching unit may trust these values or perform its own measurements. An internal ML classifier i.e. , an ML profile classifier 604, may be used to evaluate the ML profile. The ML profile classifier 604 may be trained using previous patterns and may decide whether the values in ML profile are trustable, e.g., as previously discussed in Action 305, by using the ML profile classifier 604 for estimating a trust indication indicative of a level of trust of the ML profile.

[0157] The trained data may be associated to an ML model for a specific analytics identifier, An NWDAF or MTLF instance identifier, and / or a vendor identifier. The second VLF 232 may generate a MAR, i.e., the indication of the verification of the ML model, as the baseline for the ML model consumer and result for ML model verification, e.g., as part of Action 306.

[0158] Upon the receipt of an ML model verification request, e.g., as in Action 300, the ML model verification entity in the second VLF 232 may perform a set of tests to verify that the ML model conforms to the specifications shared by the operator, user, or vendor as well as, or alternatively verifying that the ML model conforms to the ML profile, e.g., as in Action 305. These verification procedures may include but are not limited to:• Testing the ML model on baseline test data, e.g., provided by the operator and / or the ML profile, and optionally verifying that the model accuracy or another property is greater or equal than as specified in the ML profile and / or that model requirements and expectations obtained or generated by the policy indication, e.g., as shared by the operator, is adhered to by the ML model.

[0159] • Re-training the ML model using noisy data and re-testing it on baseline data to verify whether the ML model is robust against adversarial attacks.

[0160] • Verifying whether the ML model is responsive in a timely manner, e.g., whether the ML model can provide expected results within a certain time limit, e.g., as provided by the operator.

[0161] • Additional tests, specific for each use case addressed by the ML model may be performed and specified by the operator. For example, if the model aims at detecting anomalies, then specific test on zero-day attacks may be performed and assessed.

[0162] ML profile classifier

[0163] The ML profile classifier 604 may as part of the one or more properties, comprise information about a performance of the ML model. When the ML model is shared, i.e. , transmitted from the first network function node 111 to the network function node 112, the ML profile may also be shared, e.g., as in Action 301. The second network function node 112 may need to evaluate whether or not to trust the ML profile, e.g., as part of Action 305. This is important as it may decide whether the ML profile is to be used or discarded or partially discarded with regards to verifying the ML model, e.g., as in Action 305.

[0164] One way to perform such trustworthiness analysis to estimate the trust indication of Action 305 is to train a separate ML model or a recommendation system, e.g., via contextual bandit, that may provide recommendations as output 703 e.g., via three levels: trustable, partially trustable, non-trustable. The ML profile Classifier 604 may be such a trained model or recommendation system and is illustrated in Fig. 7. The ML classifier 604, also referred to as 702 in Fig. 7, may be trained with the following training data:

[0165] one or more training ML profiles or reports of one or more training ML models, e.g., indicative of accuracy measurements, time measurements, algorithms,

[0166] one or more training instance identifiers, e.g., an identity of an entity associated with the respective ML model or ML profile such as a supplier of a respective training ML profile, e.g., the NWDAF or the second MTLF 212 may query e.g., from an Network Repository Function (NRF), to find out about the one or more training instance identifiers such as information of NF instance properties and capabilities of the one or more training instance identifiers,one or more preset properties associated with the one or more training ML models, e.g., Interoperability information of the ML model, such as any one or more of: encryption, size, vendor, etc., and

[0167] one or more use cases of the one or more training ML models, e.g. UCC that described the use case that the ML model is used to generate analytics reports. Accordingly, an input 701 of the ML profile classifier 604 may comprise the ML profile or a report of performance of the ML model, an identifier of the instance providing the ML model, e.g., an identifier of the first network function node 111, information from a vendor or operator, e.g., static information of the ML model, use case of the ML model.

[0168] The output 703 of ML profile classifier 604 may be the trust indication of Action 305. The output 703 of ML profile classifier 604 may be for example a decision class that indicates whether the ML profile is trustable, partially trustable, or non-trustable. Further actions by the second network function node 112 or the NWDAF therein may be taken in case the number of non-trustable ML profiles exceeds a certain threshold, such as quarantining the ML model owner, sending feedback to the ML model owner, etc.

[0169] One way to implement the ML profile classifier 604 is to use a Random Forest algorithm for classification. The ML profile classifier 604 may be trained using training data and classifies the output based on several decision trees and a voter that generates conclusion from all the trees. It should be noted that the solution of using a classifier is one of the possible ways to define the level of trust for an ML model. Other solutions may also apply when suitable e.g., regressor to output level of trust as a scalar value.

[0170] Policy Generation - ML Approach

[0171] The second VLF 232 of embodiments herein may be arranged to receive and operate based on policies and / or rules, e.g., as discussed with respect to Fig. 2, e.g., as a baseline for verifying the ML model, e.g., as part of Action 305 such as in combination with verification using the one or more properties indicated by the ML profile or instead of using the one or more properties indicated by the ML profile if the ML profile is indicated to not to be trusted.

[0172] Policies and / or rules may be fed to the second VLF 232 by a user, operator and / or vendor e.g., to describe how and / or to what level of strictness, a verification of the ML model shall be done. The policies and / or rules may further comprise any suitable heuristics or properties for the verification of the ML model. In some embodiments the second VLF 232, may remove or not utilize the interface between the second VLF 232 and user, operator and / or vendor and let the second VLF 232 to internally generate such policies as illustrated in Fig. 8.

[0173] A predefined policy model 803 may be used to generate a policy indication 804 indicative of one or more verification procedures and / or heuristics for performing the verification of the ML model, i.e., rules and / or policies. The generated policy indication 804may be used to verify the ML model, e.g., as part of Action 305. The predefined policy model 803 may be a static model or may be a trained ML model trained on at least one of:

[0174] - one or more training policy templates,

[0175] o the one or more training policy templates may describe the ML model properties and corresponding priority for an analytics identifier, e.g., for a specific analytics identifier or identity, etc.

[0176] o The one or more training policy templates may be different between differing analytics identifiers e.g., user data congestion, UE mobility, etc. The predefined policy model 803 may use a relevant policy template as a guideline to generate a policy indication for verification of the ML model.

[0177] - one or more training ML profiles, e.g., indicative of performance or properties of one or more respective ML models and

[0178] - level of trust of the one or more training ML profiles.

[0179] Additionally or as an alternative, the predefined policy model 803 may be trained using a Large Language Model (LLM).

[0180] To generate the policy indication 804, the predefined policy model 803 may use the ML profile and optionally the level of trust of the ML profile as input 801. A set of policy templates 802 may also be used by the predefined policy model 803 to generate the policy indication 804. The set of policy templates 802 may comprise different ML model parameters which may be considered in the generated policy indication 804.

[0181] Use Cases

[0182] Two example use cases are discussed in this section and illustrated by Fig. 9. In a first use case, the ML model is shared between the first MTLF 211 and the second MTLF 212, and may have been trained using the data sources from an Area of Interest (Aol). In a second use case, the ML model may have been trained from data sources within a specific context. The use cases may both follow the sequence as discussed with respect to Fig. 5.

[0183] In the first example use case, the first MTLF 211 may train the ML model, using collected data from data sources which have been associated to a specific Aol e.g., as part of Action 400 such as a first Aol 901 comprising UE-related information, and a second Aol 902 comprising session information, e.g., of UEs. The first MTLF 211 may generate the ML profile, e.g., as part of Action 400 and transmit the ML profile to the second MTLF 212, e.g., as part of Actions 401-402, and 301. In this way, the ML model may be shared between the first MTLF 211 and the second MTLF 212. Information about the Aol may be stated in the ML profile, e.g., UE mobility, UE dispersion, etc., so the second MTLF 212 may be informed or triggered how to collect data, from which data sources, e.g., as part of Action 302, and to verify the ML model, e.g., as part of Actions 303-305. When an ML model is provisioned to an ML modelconsumer, e.g., the consumer network node 113, a MAR is generated and delivered, i.e., the indication of the verification of the ML model, e.g., as in Action 306.

[0184] The ML profile may describe the ML model properties such as algorithm, type of training, initial accuracy measurement, and information about the data sources. An example of the ML profile is indicated by an MPP report below. An example of the indication of the verification of the ML model, e.g., as in Action 306, is indicated as an MAR below. MPP describes the ML model properties such as algorithms, type of training, initial accuracy measurement, and information about the data sources. MPP is created by the ML model owner e.g.

[0185] NWDAF(MTLFI) and MAR is created by NWDAF(MTLF2) which provisions the ML model to an ML model consumer. Both are expressed in JavaScript Object Notation (JSON) but other schemes e.g., Extensible Markup Language (XML) can also be used.

[0186] {

[0187] "MPP" : "ML Model provided over AoI-id-223454" ,

[0188] "date" : "2023-06-15 " ,

[0189] "vendorld" : "339843AG" ,

[0190] "dataSources" : [

[0191] {

[0192] "type" : "UE" ,

[0193] "id" : "2228854A43KEW334 " ,

[0194] "timestamp" : "2023-06-15 , 00 : 23 : 11" ,

[0195] "uri" : "https : / / someaddrl" ,

[0196] "auth" : "true" ,

[0197] "credentials" : "NRF" ,

[0198] "Aol" : "AoI-id-223454 "

[0199] } ,

[0200] {

[0201] "type" : "PDU" ,

[0202] "id" : " 4501154X43DEE177" ,

[0203] "timestamp" : "2023-06-15 , 04 : 12 : 29" ,

[0204] "uri" : "https : / / someaddr2" ,

[0205] "auth" : "true" ,

[0206] "credentials" : "NRF" ,

[0207] "Aol" : "AoI-id-223454 "

[0208] }

[0209] ] ,

[0210] "model" : [

[0211] "alg" : "linReg" ,

[0212] "train" : "sup" ,

[0213] "fl" : " 0 . 799" ,

[0214] "acc" : "89" ,

[0215] "prec" : " 90" ,

[0216] "rec" : " 92" ,

[0217] "traint" : " 992" ,

[0218] "inft" : "542" ,

[0219] "MAR" : "ML Model provided over AoI-id-223454" ,"date" : "2023-06-20",

[0220] "vendorld" : "235493XE",

[0221] "verif" : [

[0222] "conf LevRepl " : "0. 92",

[0223] "conf LevMeasl " : "0. 91",

[0224] "conf LevRep2 " : "0.88",

[0225] "conf LevMeas2 " : "0. 9",

[0226] "conf LevRep3 " : "0. 9",

[0227] "conf LevMeas3 " : "0. 9",

[0228] "inpOut" : "TRUE",

[0229] "revAttackSaf e" : "TRUE",

[0230] "policy" : "TRUE",

[0231] "acc" : "91" ,

[0232] "prec" : "90",

[0233] "rec" : "87"

[0234] ]

[0235] }

[0236] Context Specific ML Model Verification

[0237] In the second example use case, the NWDAF of the first network function node 111, in particular the first MTLF 211 may be the ML model owner and the NWDAF of the second network function node 112, in particular the second MTLF 212 may be an ML model consumer which receives the shared ML model. It may be assumed that the NWDAF of the first network function node 111, in particular the first MTLF 211 trains the ML model for a specific purpose. For instance, collected data may be related to UEs which have been running specific application, Ultra Reliable Low Latency Communication (URLLC) applications, or the context may be limited to certain period in a day. The ML model consumer can optionally send one or more UCC parameters when subscribing for an ML model provisioning, where the UCC may indicate information about how and in what use cases the ML model is supposed to be used. In this scenario, the NWDAF of the first network function node 111, in particular the first MTLF 211 receives a request to share the ML model for an analytics identifier with the UCC from the second network function node 112, in particular the second MTLF 212 which describes the context that the ML model will be used for e.g. generating analytics reports. The NWDAF of the first network function node 111, in particular the first MTLF 211 may select the ML model out of a number of candidate ML models based on the UCC, generate the MP profile for the ML model and provision to the NWDAF of the second network function node 112, in particular the second MTLF 212. Upon receiving the ML model, the NWDAF of the second network function node 112, in particular the second MTLF 212 may initiate a verification process, e.g., as part of Action 305, by sending a request to the second VLF 232. The verification process, e.g., as described in Action 305 may additionally or alternatively comprise any one or more of:

[0238] Measuring and verifying the accuracy of the ML model when input data is collected from the UCC. Ground truth data may be collected from an ADRF.Measuring and verifying the performance, e.g., inference time, when inference data is collected from the UCC.

[0239] Measuring and verifying the ML model security and identifying possible vulnerabilities.

[0240] Measuring and verifying the ML model integrity and consistency e.g., that the reported confidence level for the prediction for UCC, correctly reflects the accuracy of generated analytics report.

[0241] When the verification is performed, a MAR is generated e.g., as part of Action 306. The NWDAF of the second network function node 112, in particular the second MTLF 212 may decide whether to further train the ML model with training data from UCC and if such training is done, the MAR may be updated before the ML model is provisioned to an ML model consumer such as the consumer network node 113. In this way, the NWDAF of the second network function node 112, in particular the second MTLF 212 may be able to trigger a training process whenever needed as well as informing the ML model consumer about an accuracy of the ML model. Without verification of the ML Model, the NWDAF of the second network function node 112, in particular the second MTLF 212 may never find out if the ML model shall be trained further.

[0242] Fig. 10 is a block diagram depicting the second network function node 112 for handling the ML model in the wireless communications network 100.

[0243] The second network function node 112 may comprise processing circuitry 1001, e.g., at least one processor, configured to perform the methods herein.

[0244] The second network function node 112 may comprise an NWDAF 1002, e.g., with reference to Fig. 2, arranged to perform at least part of embodiments herein.

[0245] The second network function node 112 is configured to handle the model in the wireless communications network 100. The second network function node 112 is configured to receive from the first network function node 111, the ML model and the ML profile for the ML model. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The second network function node 112 is configured to obtain verification data for verifying the ML model. The verification data comprises training data used for training the ML model and / or ground truth data for the ML model. The second network function node 112 is configured to, based on the ML profile and the verification data, verify the ML model. The second network function node 112 is configured to provide to a consumer network node 113, the ML model and an indication of the verification of the ML model.

[0246] In some embodiments, the second network function node 112 may be configured to obtain a policy indication indicative of one or more verification procedures and / or heuristics for performing the verification of the ML model. In some of these embodiments, the secondnetwork function node 112 is configured to verify the ML model further based on the obtained policy indication.

[0247] In some embodiments, the second network function node 112 may be configured to obtain a trust indication indicating a level of trust of the ML profile. In some of these embodiments, the second network function node 112 is configured to verify the ML model based on the level of trust of the ML profile.

[0248] In some embodiments, the second network function node 112 may be configured to verify the ML model by: when the level of trust indicates that the ML profile can be trusted, using the verification data to verify whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile.

[0249] In some embodiments, the second network function node 112 may be configured to verify the ML model by: when the level of trust indicates that the ML profile cannot be trusted, using the verification data to verify whether or not the ML model adheres to one or more properties indicated by a policy indication obtained for verifying the ML model, wherein the policy indication is indicative of one or more verification procedures and / or heuristics for performing the verification of the ML model.

[0250] In some embodiments, the policy indication may be generated based on the ML profile, the level of trust associated with the ML profile, and the predefined policy model 803.

[0251] In some embodiments, the predefined policy model 803 may be a trained ML model trained on at least one of: one or more training policy templates, one or more training ML profiles, and level of trust of the one or more training ML profiles.

[0252] In some embodiments, the second network function node 112 may be configured to verify the ML model by: verifying, using the verification data, whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile and fulfils the heuristics of the obtained or generated policy indication.

[0253] In some embodiments, the second network function node 112 may be configured to obtaining the trust indication indicating the level of trust by estimating the trust indication based on the ML profile and based on at least one of: an identifier of an entity providing and / or generating the ML profile and / or ML model, information indicative of one or more preset properties associated with the ML model, and an indication of one or more use cases of the ML model.

[0254] In some embodiments, the trust indication indicating the level of trust may be estimated using an ML profile classifier 604, 702 trained based on training data comprising one or more training ML profiles of one or more training ML models, and comprising at least one of: one or more training instance identifiers, one or more preset properties associated with the one or more training ML models, and one or more use cases of the one or more training ML models.In some embodiments, the second network function node 112 may be configured to verify the ML model by at least one of: testing the ML model according to tests indicated by the ML profile, verifying whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile, verifying whether or not a confidence value and / or an accuracy of the one or more properties associated with the one or more conditions of the ML profile is adhered to by the ML model when verified with the verification data, verifying whether or not the ML model adheres to one or more properties indicated by one or more obtained policies, verifying an input-output relationship of the ML model, verifying one or more weights of one or more input parameters of the ML model, and verifying whether or not the ML model is vulnerable to an inversion attack.

[0255] In some embodiments, the one or more properties associated with the one or more conditions of the ML profile may comprise at least one of: expected or required accuracy of the ML model, expected or required confidence of the ML model, one or more constraints of the ML model, one or more protocols and / or rules expected or required to be used by the ML model, and one or more weights of the ML model.

[0256] In some embodiments, the second network function node 112 may be configured to receive from the consumer network node 113, a request for providing the ML model and the indication of the verification of the ML model. In some of these embodiments, the second network function node 112 may be configured to provide the ML model and the indication of the verification of the ML model based on, and / or in response to receiving the request.

[0257] In some embodiments, the request may be indicative of one or more requested properties of the ML model. In some of these embodiments, the second network function node 112 may be configured to verify the ML model by verifying whether the ML model adheres to the one or more requested properties of the ML model. In some of these embodiments, the indication of the verification of the ML model provided to the consumer network node 113 may further indicate whether or not the ML model adheres to the one or more requested properties.

[0258] The second network function node 112 may comprise a memory 1005. The memory 1005 comprises one or more units to be used to store data on, such as data packets, indications, SSB information, common signal / channel information, cell information, candidate cells, reference signal information, assistance information, application information, messages, measurement, the ML Model, the ML profile, verification data, events and applications to perform the methods disclosed herein when being executed, and similar. Furthermore, the second network function node 112 may comprise a communication interface 1006 such as comprising a transmitter, a receiver, a transceiver and / or one or more antennas.

[0259] The methods according to the embodiments described herein for the second network function node 112 are respectively implemented by means of e.g. a computer program product 1007 or a computer program, comprising instructions, i.e. , software code portions,which, when executed on at least one processor, cause the at least one processor to carry out the actions described herein, e.g., as performed by the second network function node 112. The computer program product 1007 may be stored on a computer-readable storage medium 1008, e.g., a disc, a universal serial bus USB stick or similar. The computer-readable storage medium 1008, having stored thereon the computer program product, may comprise the instructions which, when executed on at least one processor, cause the at least one processor to carry out the actions described herein, as performed by the second network function node 112. In some embodiments, the computer-readable storage medium may be a transitory or a non-transitory computer-readable storage medium. Thus, embodiments herein may disclose the radio network node for handling communication in a wireless communication network, wherein radio network node comprises processing circuitry and a memory, said memory comprising instructions executable by said processing circuitry whereby said radio network node is operative to perform any of the methods herein.

[0260] Fig. 11 is a block diagram depicting the first network function node 111 for handling the ML model in the wireless communications network 100.

[0261] The first network function node 111 may comprise processing circuitry 1101, e.g., at least one processor, configured to perform the methods herein.

[0262] The first network function node 111 may comprise an NWDAF 1102, e.g., with reference to Fig. 2, arranged to perform at least part of embodiments herein.

[0263] The first network function node 111 is configured to handle the ML model in the wireless communications network 100. The first network function node 111 is configured to obtain the ML model and the ML profile for the ML model. The ML profile is indicative of one or more properties associated with one or more conditions of the ML model. The first network function node 111 is configured to transmit to the second network function node 112, the ML model and the ML profile.

[0264] In some embodiments, the first network function node 111 may be further configured to trigger the second network function node 112 to obtain verification data, and to verify the ML model based on the ML profile and the verification data.

[0265] In some embodiments, the first network function node 111 may be further configured to obtain the ML profile by generating the ML profile.

[0266] The first network function node 111 may comprise a memory 1105. The memory 1105 comprises one or more units to be used to store data on, such as data packets, indications, SSB information, common signal / channel information, cell information, candidate cells, reference signal information, assistance information, application information, messages, measurement, the ML Model, the ML profile, verification data, events and applications to perform the methods disclosed herein when being executed, and similar. Furthermore, the firstnetwork function node 111 may comprise a communication interface 1106 such as comprising a transmitter, a receiver, a transceiver and / or one or more antennas.

[0267] The methods according to the embodiments described herein for the first network function node 111 are respectively implemented by means of e.g. a computer program product 1107 or a computer program, comprising instructions, i.e., software code portions, which, when executed on at least one processor, cause the at least one processor to carry out the actions described herein, e.g., as performed by the first network function node 111. The computer program product 1107 may be stored on a computer-readable storage medium 1108, e.g., a disc, a universal serial bus (USB) stick or similar. The computer-readable storage medium 1108, having stored thereon the computer program product, may comprise the instructions which, when executed on at least one processor, cause the at least one processor to carry out the actions described herein, as performed by the first network function node 111. In some embodiments, the computer-readable storage medium may be a transitory or a non-transitory computer-readable storage medium. Thus, embodiments herein may disclose the radio network node for handling communication in a wireless communication network, wherein radio network node comprises processing circuitry and a memory, said memory comprising instructions executable by said processing circuitry whereby said radio network node is operative to perform any of the methods herein.

[0268] Fig. 12 is a block diagram depicting a system 114. The system 114 is arranged for handling the ML model in the wireless communications network 100. The system 114 comprises the first network function node 111 according to any of embodiments herein. The system 114 comprises the second network function node 112 according to any of embodiments herein. The system 114 comprises the consumer network node 113 according to any of embodiments herein.

[0269] In some embodiments of the system 114, the first network function node 111 may be configured to provide the ML model and the ML profile to the second network function node 112. The second network function node 112 may be configured to verify the ML model based on the ML profile and obtained verification data. The second network function node 112 may be configured to provide the ML model and a report indicative of the verification of the ML model to the consumer network node 113.

[0270] When using the word "comprise" or “comprising”, it shall be interpreted as non- limiting, i.e., meaning "consist at least of".

[0271] The embodiments herein are not limited to the above-described preferred embodiments. Various alternatives, modifications and equivalents may be used. Therefore, the above embodiments should not be taken as limiting the scope of the invention.Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and / or is implied from the context in which it is used. All references to a / an / the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and / or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following description.

[0272] As used herein, the expression “at least one of:” followed by a list of alternatives separated by commas, and wherein the last alternative is preceded by the “and” term, may be understood to mean that only one of the list of alternatives may apply, more than one of the list of alternatives may apply or all of the list of alternatives may apply. This expression may be understood to be equivalent to the expression “at least one of:” followed by a list of alternatives separated by commas, and wherein the last alternative is preceded by the “or” term.

[0273] Any of the terms processor and circuitry may be understood herein as a hardware component. As used herein, the expression “in some embodiments” has been used to indicate that the features of the embodiment described may be combined with any other embodiment or example disclosed herein.

[0274] As used herein, the expression “in some examples” has been used to indicate that the features of the example described may be combined with any other embodiment or example disclosed herein.

Claims

33CLAIMS:

1. A method performed by a second network function node (112) for handling a Machine Learning, ML, model in a wireless communications network (100), the method comprising:receiving (301) from a first network function node (111), the ML model and an ML profile for the ML model, wherein the ML profile is indicative of one or more properties associated with one or more conditions of the ML model;obtaining (302) verification data for verifying the ML model, wherein the verification data comprises training data used for training the ML model and / or ground truth data for the ML model;based on the ML profile and the verification data, verifying (305) the ML model; and providing (306) to a consumer network node (113), the ML model and an indication of the verification of the ML model.

2. The method of any of the preceding claims, further comprising:- obtaining (304) a policy indication indicative of one or more verification procedures and / or heuristics for performing the verification of the ML model, and- wherein verifying (305) the ML model is further based on the obtained policy indication.

3. The method of any of the preceding claims, further comprising:- obtaining (303) a trust indication indicating a level of trust of the ML profile, and- wherein verifying (305) the ML model is based on the level of trust of the ML profile.

4. The method of claim 3, wherein verifying (305) the ML model comprises:- when the level of trust indicates that the ML profile can be trusted, using the verification data to verify whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile.

5. The method of claim 3 or 4, wherein verifying (305) the ML model comprises:- when the level of trust indicates that the ML profile cannot be trusted, using the verification data to verify whether or not the ML model adheres to one or more properties indicated by a policy indication obtained for verifying the ML model, wherein the policy indication is indicative of one or more verification procedures and / or heuristics for performing the verification of the ML model.

346. The method of claim 5, wherein the policy indication is generated based on the ML profile, the level of trust associated with the ML profile, and a predefined policy model (803).

7. The method of claim 6 wherein the predefined policy model (803) is a trained ML model trained on at least one of:one or more training policy templates,one or more training ML profiles, andlevel of trust of the one or more training ML profiles.

8. The method of any of claims 5-7, wherein verifying (305) the ML model comprises: verifying, using the verification data, whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile and fulfils the heuristics of the obtained or generated policy indication.

9. The method of any of claims 3-8, wherein obtaining (303) the trust indication indicating the level of trust comprises estimating the trust indication based on the ML profile and based on at least one of:an identifier of an entity providing and / or generating the ML profile and / or ML model, information indicative of one or more preset properties associated with the ML model, andan indication of one or more use cases of the ML model.

10. The method of any of claims 3-9, wherein the trust indication indicating the level of trust is estimated using an ML profile classifier (604, 702) trained based on training data comprising one or more training ML profiles of one or more training ML models, and comprising at least one of:one or more training instance identifiers,one or more preset properties associated with the one or more training ML models, andone or more use cases of the one or more training ML models.

11. The method of any of the preceding claims, wherein verifying (305) the ML model comprises at least one of:- testing the ML model according to tests indicated by the ML profile,- verifying whether or not the ML model adheres to the one or more properties associated with the one or more conditions of the ML profile,- verifying whether or not a confidence value and / or an accuracy of the one or more properties associated with the one or more conditions of the ML profile is adhered to by the ML model when verified with the verification data,- verifying whether or not the ML model adheres to one or more properties indicated by one or more obtained policies,- verifying an input-output relationship of the ML model,- verifying one or more weights of one or more input parameters of the ML model, and - verifying whether or not the ML model is vulnerable to an inversion attack.

12. The method of any of the preceding claims, wherein the one or more properties associated with the one or more conditions of the ML profile comprises at least one of: expected or required accuracy of the ML model,expected or required confidence of the ML model,one or more constraints of the ML model,one or more protocols and / or rules expected or required to be used by the ML model, andone or more weights of the ML model.

13. The method of any of the preceding claims, wherein the method further comprises: receiving (300) from the consumer network node (113), a request for providing the ML model and the indication of the verification of the ML model, and wherein providing (306) the ML model and the indication of the verification of the ML model is performed based on, and / or in response to receiving the request.

14. The method of claim 13 wherein the request is indicative of one or more requested properties of the ML model, wherein verifying (305) the ML model further comprises verifying whether the ML model adheres to the one or more requested properties of the ML model, and wherein the indication of the verification of the ML model provided to the consumer network node (113) further indicates whether or not the ML model adheres to the one or more requested properties.

15. A method performed by a first network function node (111) for handling a Machine Learning, ML, model in a wireless communications network (100), the method comprising:obtaining (400) the ML model and an ML profile for the ML model, the ML profile is indicative of one or more properties associated with one or more conditions of the ML model; andtransmitting (401) to a second network function node (112), the ML model and the ML profile.

16. The method according to claim 15 wherein the method further comprises triggering (402) the second network function node (112) to obtain verification data, and to verify the ML model based on the ML profile and the verification data.

17. The method of claim 15 or 16 wherein obtaining (400) the ML profile comprises generating the ML profile.

18. A second network function node (112) configured to handle a Machine Learning, ML, model in a wireless communications network (100), wherein the second network function node (112) is configured toreceive from a first network function node (111), the ML model and an ML profile for the ML model, wherein the ML profile is indicative of one or more properties associated with one or more conditions of the ML model;obtain verification data for verifying the ML model, wherein the verification data comprises training data used for training the ML model and / or ground truth data for the ML model;based on the ML profile and the verification data, verify the ML model; and provide to a consumer network node (113), the ML model and an indication of the verification of the ML model.

19. The second network function node (112) of claim 18, further being configured to perform the method according to any one of claims 2-14.

20. A first network function node (111) configured to handle a Machine Learning, ML, model in a wireless communications network (100), wherein the first network function node (111) is configured to:obtain the ML model and an ML profile for the ML model, the ML profile is indicative of one or more properties associated with one or more conditions of the ML model; and- transmit to a second network function node (112), the ML model and the ML profile.

21. The first network function node (111) according to claim 20, wherein the first network function node (111) further is configured to perform the method according to any one of claims 16-17.3722. A system (114) arranged for handling a Machine Learning, ML, model in a wireless communications network (100), wherein the system comprises a first network function node (111) according to any of claims 18-19, a second network function node (112) according to any of claims 20-21, and a consumer network node (113).

23. The system (114) of claim 22, wherein the first network function node (111) is configured to provide the ML model and an ML profile to the second network function node (112), wherein the second network function node (112) is configured to verify the ML model based on the ML profile and obtained verification data, and wherein the second network function node (112) is configured to provide the ML model and a report indicative of the verification of the ML model to the consumer network node (113).

24. A computer program product (1007) comprising instructions, which, when executed on at least one processor, cause the at least one processor to carry out the method according to any of claims 1-14.

25. A computer-readable storage medium (1008), having stored thereon a computer program product comprising instructions which, when executed on at least one processor, cause the at least one processor to carry out the method according to any of claims 1-14.

26. A computer program product (1107) comprising instructions, which, when executed on at least one processor, cause the at least one processor to carry out the method according to any of claims 15-17.

27. A computer-readable storage medium (1108), having stored thereon a computer program product comprising instructions which, when executed on at least one processor, cause the at least one processor to carry out the method according to any of claims 15-17.

Images