Apparatus, method and computer program

Abstract

There is provided an apparatus comprising means for receiving a request for analysis information from a first machine learning model from an analysis consumer, obtaining the first machine learning model, obtaining a second machine learning model, the second machine learning model being trained prior to the first machine learning model, the first machine learning model and the second machine learning model have the same analysis identifier; in one embodiment, a method includes obtaining a first inference output from a first machine learning model and a second inference output from a second machine learning model, determining that the first machine learning model has been attacked based on the first inference output and the second inference output, and providing an indication to a network entity that the first machine learning model has been attacked.

Description

Technical Field

[0001] The present application relates to a method, apparatus, system and computer program, particularly but not limited to the detection and mitigation of data poisoning attacks. Background Art

[0002] A communication system can be viewed as a facility that enables a communication session between two or more entities (such as user terminals, base stations and / or other nodes) by providing a carrier between the various entities involved in the communication path. A communication system can be provided, for example, by a communication network and one or more compatible communication devices. A communication session may include, for example, a communication for carrying data for communication, such as voice, video, electronic mail (email), text message, multimedia and / or content data, etc. Non-limiting examples of the services provided include two-way or multi-way calls, data communications or multimedia services, and access to a data network system (such as the Internet).

[0003] In a wireless communication system, at least a part of a communication session between at least two stations is conducted over a wireless link. Examples of wireless systems include public land mobile networks (PLMNs), satellite-based communication systems, and different wireless local networks, such as wireless local area networks (WLANs). Some wireless systems can be divided into cells and are therefore often referred to as cellular systems.

[0004] A user may access a communication system through an appropriate communication device or terminal. The user's communication device may be referred to as a user equipment (UE) or user device. The communication device is provided with appropriate signal receiving and sending means for enabling communication, for example, enabling access to a communication network or direct communication with other users. The communication device may access a carrier provided by a station (e.g., a base station of a cell) and send and / or receive communications on the carrier.

[0005] A communication system and associated equipment typically operate in accordance with a given standard or specification that specifies what the various entities associated with the system are allowed to do and how it should be implemented. The communication protocols and / or parameters used for the connection are also typically defined. An example of a communication system is the Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (UTRAN) (3G radio). Other examples of communication systems are the Long Term Evolution (LTE) of the Universal Mobile Telecommunications System (UMTS) radio access technology and the so-called 5G or New Radio (NR) networks. NR is being standardized by the Third Generation Partnership Project (3GPP). Other examples of communication systems include 5G-Advanced (NR Rel-18 and above) and 6G. Summary of the invention

[0006] In a first aspect, an apparatus is provided, comprising components for receiving a request for analytical information from a first machine learning model from an analytics consumer, obtaining the first machine learning model, obtaining a second machine learning model, the second machine learning model being trained before the first machine learning model, the first machine learning model and the second machine learning model having the same analytics identifier, obtaining a first inference output from the first machine learning model, and a second inference output from the second machine learning model, determining based on the first inference output and the second inference output that the first machine learning model has been attacked, and providing an indication to a network entity that the first machine learning model has been attacked.

[0007] The apparatus may include means for determining a concept drift between the first inference output and the second inference output, and determining that the first machine learning model has been attacked based on the determined concept drift.

[0008] The apparatus may include means for determining, based on a concept drift threshold, that the first machine learning model has been attacked.

[0009] The apparatus may include components for: providing a first reasoning output to an analysis consumer, obtaining feedback information of the first reasoning output from the analysis consumer; and further determining that the first machine learning model has been attacked based on the feedback information.

[0010] The device may include components for: obtaining first training data used to train a first machine learning model from at least one network function; obtaining second training data used to train a second machine learning model from at least one network function; and determining, from at least one network function, a network function on which the first machine learning model is attacked based on the first training data and the second training data.

[0011] The apparatus may include means for determining a data distribution drift based on the first training data and the second training data.

[0012] The apparatus may include means for determining a network function from a plurality of network functions based on a data distribution drift threshold.

[0013] The apparatus may include means for providing an indication of the determined network functionality to a network entity.

[0014] The apparatus may include means for excluding the determined network functionality when requesting data for subsequent training of the first machine learning model.

[0015] The network entity may include: a network analysis function, an operation and management function, or a user equipment.

[0016] In a second aspect, an apparatus is provided that includes components for: receiving a request for a first machine learning model from a network entity, providing the first machine learning model, and receiving an indication from the network entity that the first machine learning model has been attacked.

[0017] The apparatus may include components for: receiving a request for first training data from at least one network function from a network entity, the first training data being used to train a first machine learning model; providing the first training data to the network entity; and receiving from the network entity: an indication of a network function in which the first machine learning model determined from at least one network function is attacked, the indication being based on the first training data.

[0018] The network entity may include: an analysis network function, a radio access network node, or an operation and management function.

[0019] In a third aspect, an apparatus is provided, comprising components for receiving a request for a machine learning model associated with an analysis identifier from a network entity and providing the machine learning model associated with the analysis identifier to the network entity.

[0020] The apparatus may include means for receiving, from a network entity, a request for training data from at least one network function, the training data being used to train a machine learning model associated with an analysis identifier; and providing the training data to the network entity.

[0021] The apparatus may include: an analytical data repository functionality, or a data repository including a machine learning model and training data.

[0022] The network entity may include: an analysis network function, or a radio access network node.

[0023] In a fourth aspect, an apparatus is provided, comprising components for providing a request from an analytics consumer to a network entity, the request for analytics information from a first machine learning model, the first machine learning model being associated with an analytics identifier, receiving a first inference output from the first machine learning model from the network entity, determining feedback information based on the first inference output, and providing the feedback information to the network entity.

[0024] When the feedback information is negative, the apparatus may include means for providing a request to the network entity for additional analysis information associated with the analysis identifier.

[0025] The network entity may include: an analysis network function, or a radio access node.

[0026] In a fifth aspect, a method is provided, comprising: receiving a request for analytical information from a first machine learning model from an analysis consumer, obtaining the first machine learning model, obtaining a second machine learning model, the second machine learning model being trained before the first machine learning model, the first machine learning model and the second machine learning model having the same analysis identifier, obtaining a first inference output from the first machine learning model, and a second inference output from the second machine learning model, determining based on the first inference output and the second inference output that the first machine learning model has been attacked, and providing an indication to a network entity that the first machine learning model has been attacked.

[0027] The method may include determining a concept drift between the first inference output and the second inference output, and determining that the first machine learning model has been attacked based on the determined concept drift.

[0028] The method may include determining that the first machine learning model has been attacked based on a concept drift threshold.

[0029] The method may include providing a first inference output to an analysis consumer, obtaining feedback information based on the first inference output from the analysis consumer, and further determining that the first machine learning model has been attacked based on the feedback information.

[0030] The method may include: obtaining first training data used to train a first machine learning model from at least one network function, obtaining second training data used to train a second machine learning model from at least one network function, and determining a network function in which the first machine learning model is attacked from at least one network function based on the first training data and the second training data.

[0031] The method may include determining a data distribution drift based on the first training data and the second training data.

[0032] The method may include determining a network function from a plurality of network functions based on a data distribution drift threshold.

[0033] The method may include providing an indication of the determined network functionality to a network entity.

[0034] The method may include excluding the determined network functionality when requesting data for subsequent training of the first machine learning model.

[0035] The network entity may include: a network analysis function, an operation and management function, or a user equipment.

[0036] In a sixth aspect, a method is provided, comprising: receiving a request for a first machine learning model from a network entity, providing the first machine learning model, and receiving an indication from the network entity that the first machine learning model has been attacked.

[0037] The method may include: receiving a request for first training data from at least one network function from a network entity, the first training data being used to train a first machine learning model; providing the first training data to the network entity; and receiving from the network entity: an indication of a network function in which the first machine learning model determined from at least one network function is attacked, the indication being based on the first training data.

[0038] The network entity may include: an analysis network function, a radio access network node, or an operation and management function.

[0039] In a seventh aspect, a method is provided, comprising: receiving a request for receiving a request for a machine learning model associated with an analysis identifier from a network entity, and providing the machine learning model associated with the analysis identifier to the network entity.

[0040] The method may include receiving a request from a network entity for training data from at least one network function, the training data being used to train a machine learning model associated with an analysis identifier; and providing the training data to the network entity.

[0041] The method may be performed at an analytics data repository function, or a data repository including a machine learning model and training data.

[0042] The network entity may comprise an analysis network function or a radio access network node.

[0043] In an eighth aspect, a method is provided, comprising providing a request from an analytics consumer to a network entity, the request being for analytics information from a first machine learning model, the first machine learning model being associated with an analytics identifier, receiving a first inference output from the first machine learning model from the network entity, determining feedback information based on the first inference output, and providing the feedback information to the network entity.

[0044] When the feedback information is negative, the method may include providing a request to the network entity for additional analysis information associated with the analysis identifier.

[0045] The network entity may comprise an analyzing network function or a radio access node.

[0046] In a ninth aspect, a device is provided, comprising at least one processor and at least one memory, the at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured, together with the at least one processor, to cause the device to at least: receive a request for analysis information from a first machine learning model from an analysis consumer, obtain the first machine learning model, obtain a second machine learning model, the second machine learning model being trained before the first machine learning model, the first machine learning model and the second machine learning model having the same analysis identifier, obtain a first inference output from the first machine learning model, and a second inference output from the second machine learning model, determine based on the first inference output and the second inference output that the first machine learning model has been attacked, and provide an indication to a network entity that the first machine learning model has been attacked.

[0047] The apparatus may be configured to determine a concept drift between the first inference output and the second inference output and determine that the first machine learning model has been attacked based on the determined concept drift.

[0048] The apparatus may be configured to determine that the first machine learning model has been attacked based on a concept drift threshold.

[0049] The apparatus may be configured to provide a first inference output to an analysis consumer, obtain feedback information based on the first inference output from the analysis consumer, and further determine that the first machine learning model has been attacked based on the feedback information.

[0050] The device can be enabled to obtain first training data used to train a first machine learning model from at least one network function, obtain second training data used to train a second machine learning model from at least one network function, and determine a network function in which the first machine learning model is attacked from at least one network function based on the first training data and the second training data.

[0051] The apparatus may be configured to determine a data distribution drift based on the first training data and the second training data.

[0052] The apparatus may be caused to determine the network function from among the plurality of network functions based on a data distribution drift threshold.

[0053] The apparatus may be caused to provide an indication of the determined network functionality to a network entity.

[0054] The apparatus may be enabled to exclude the determined network functionality when requesting data for subsequent training of the first machine learning model.

[0055] The network entity may include a network analysis function, an operation and management function, or a user equipment.

[0056] In a tenth aspect, a device is provided, comprising at least one processor and at least one memory, the at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured, together with the at least one processor, to cause the device to at least: receive a request for a first machine learning model from a network entity; provide the first machine learning model, and receive an indication from the network entity that the first machine learning model has been attacked.

[0057] The apparatus may be configured to receive from a network entity a request for first training data from at least one network function, the first training data being used to train a first machine learning model; provide the first training data to the network entity; and receive from the network entity: an indication of a network function in which the first machine learning model determined from at least one network function is attacked, the indication being based on the first training data.

[0058] The network entity may include: an analysis network function, a radio access network node, or an operation and management function.

[0059] In an eleventh aspect, a device is provided, comprising at least one processor and at least one memory, the at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured, together with the at least one processor, to cause the device to at least: receive a request for receiving a request for a machine learning model associated with an analysis identifier from a network entity, and provide the machine learning model associated with the analysis identifier to the network entity.

[0060] The apparatus may be configured to receive a request from a network entity for training data from at least one network function, the training data being used to train a machine learning model associated with an analysis identifier; and provide the training data to the network entity.

[0061] The apparatus may include: an analytical data repository functionality, or a data repository including a machine learning model and training data.

[0062] The network entity may comprise an analysis network function, or a radio access network node.

[0063] In a twelfth aspect, a device is provided, comprising at least one processor and at least one memory, the at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured, together with the at least one processor, to cause the device to at least: provide a request from an analysis consumer to a network entity, the request being for analysis information from a first machine learning model associated with an analysis identifier, receive a first inference output from the first machine learning model from the network entity, determine feedback information based on the first inference output, and provide the feedback information to the network entity.

[0064] When the feedback information is negative, the apparatus may be caused to provide a request to the network entity for further analysis information associated with the analysis identifier.

[0065] The network entity may comprise an analyzing network function or a radio access node.

[0066] In a thirteenth aspect, a computer-readable medium is provided, comprising instructions, which when executed by an apparatus causes the apparatus to perform at least the following items: receive a request for analysis information from a first machine learning model from an analysis consumer, obtain the first machine learning model, obtain a second machine learning model, the second machine learning model being trained before the first machine learning model, the first machine learning model and the second machine learning model having the same analysis identifier, obtain a first inference output from the first machine learning model, and a second inference output from the second machine learning model, determine based on the first inference output and the second inference output that the first machine learning model has been attacked, and provide an indication to a network entity that the first machine learning model has been attacked.

[0067] The apparatus may be configured to perform determining a concept drift between the first inference output and the second inference output, and determining that the first machine learning model has been attacked based on the determined concept drift.

[0068] The apparatus may be configured to perform a determination based on a concept drift threshold that the first machine learning model has been attacked.

[0069] The apparatus may be configured to perform providing a first reasoning output to an analysis consumer; obtaining feedback information based on the first reasoning output from the analysis consumer; and further determining, based on the feedback information, that the first machine learning model has been attacked.

[0070] The device can be configured to obtain first training data used to train a first machine learning model from at least one network function, obtain second training data used to train a second machine learning model from at least one network function, and determine a network function in which the first machine learning model is attacked from at least one network function based on the first training data and the second training data.

[0071] The apparatus may be configured to perform determining a data distribution drift based on the first training data and the second training data.

[0072] The apparatus may be configured to perform determining a network function from a plurality of network functions based on a data distribution drift threshold.

[0073] The apparatus may be caused to perform providing an indication of the determined network function to a network entity.

[0074] The apparatus may be configured to perform operations that exclude the determined network functionality when requesting data for subsequent training of the first machine learning model.

[0075] The network entity may include: a network analysis function, an operation and management function, or a user equipment.

[0076] In a fourteenth aspect, a computer-readable medium is provided, comprising instructions which, when executed by an apparatus, cause the apparatus to perform at least the following: receive a request for a first machine learning model from a network entity; provide the first machine learning model and receive an indication from the network entity that the first machine learning model has been attacked.

[0077] The apparatus may be configured to perform receiving a request from a network entity for first training data from at least one network function, the first training data being used to train a first machine learning model; providing the first training data to the network entity; and receiving from the network entity: an indication of a network function in which the first machine learning model determined from at least one network function is attacked, the indication being based on the first training data.

[0078] The network entity may include: an analysis network function, a radio access network node, or an operation and management function.

[0079] In a fifteenth aspect, a computer-readable medium is provided, comprising instructions which, when executed by an apparatus, cause the apparatus to at least perform the following: receive a request for receiving a request for a machine learning model associated with an analysis identifier from a network entity, and provide the machine learning model associated with the analysis identifier to the network entity.

[0080] The apparatus may be configured to perform receiving a request from a network entity for training data from at least one network function, the training data being used to train a machine learning model associated with an analysis identifier; and providing the training data to the network entity.

[0081] The apparatus may include: an analytical data repository functionality, or a data repository including a machine learning model and training data.

[0082] The network entity may include: an analysis network function, or a radio access network node.

[0083] In a sixteenth aspect, a computer-readable medium is provided, comprising instructions, which, when executed by an apparatus, cause the apparatus to perform at least the following: provide a request from an analysis consumer to a network entity, the request being for analysis information from a first machine learning model, the first machine learning model being associated with an analysis identifier, receiving a first inference output from the first machine learning model from the network entity, determining feedback information based on the first inference output, and providing the feedback information to the network entity.

[0084] When the feedback information is negative, the apparatus may be caused to perform providing a request to the network entity for further analysis information associated with the analysis identifier.

[0085] The network entity may include: an analysis network function, or a radio access node.

[0086] In a seventeenth aspect, a non-transitory computer-readable medium is provided, comprising program instructions for causing an apparatus to execute a method according to at least the third aspect or the fourth aspect.

[0087] In the above, many different embodiments have been described. It should be understood that additional embodiments can be provided by combining any two or more of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Embodiments will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0089] Figure 1 A schematic diagram of an example 5GS communication system is shown;

[0090] Figure 2 A schematic diagram of an example mobile communication device is shown;

[0091] Figure 3 A schematic diagram of an example control device is shown;

[0092] Figure 4 A flow chart showing a method according to an example embodiment is shown;

[0093] Figure 4 A flow chart showing a method according to an example embodiment is shown;

[0094] Figure 5 A flow chart showing a method according to an example embodiment is shown;

[0095] Figure 6 A flow chart showing a method according to an example embodiment is shown;

[0096] Figure 7 shows a signaling flow according to an example embodiment;

[0097] Figure 8 A signaling flow according to an example embodiment is shown. Fig. 9 An example call flow is shown where the analysis is performed at a RAN node. DETAILED DESCRIPTION

[0098] Before explaining the examples in detail, certain general principles of wireless communication systems and mobile communication devices are first referred to. Figure 1 , Figure 2 and Figure 3 are briefly explained to help understand the techniques underlying the described examples.

[0099] An example of a suitable communication system is the 5G or NR concept. The network architecture in NR may be similar to that in LTE-advanced. The base station of the NR system may be referred to as the next generation NodeB (gNB). Changes to the network architecture may depend on the need for various radio technologies and more refined quality of service (QoS) support, as well as some on-demand requirements, such as QoS levels for supporting user quality of experience (QoE). In addition, network-aware services and applications, and service- and application-aware networks may bring changes to the architecture. These are related to the information-centric network (ICN) and user-centric content delivery network (UC-CDN) approaches. NR may use multiple-input multiple-output (MIMO) antennas, many more base stations or nodes than LTE (the so-called small cell concept), including macro sites operating in cooperation with smaller sites, and may also adopt various radio technologies to achieve better coverage and higher data rates.

[0100] Future networks may utilize network function virtualization (NFV), a network architecture concept that proposes virtualizing network node functions into "building blocks" or entities that can be operatively connected or linked together to provide services. A virtualized network function (VNF) may include one or more virtual machines that run computer program code using standard or generic types of servers rather than custom hardware. Cloud computing or data storage may also be utilized. In radio communications, this may mean that node operations are performed at least in part in a server, host, or node that is coupled to remote radio head operations. Node operations may also be distributed among multiple servers, nodes, or hosts. It should also be understood that the distribution of labor between core network operations and base station operations may be different from LTE, or even non-existent.

[0101] Figure 1 A schematic diagram of a 5G system (5GS) 100 is shown. The 5GS may include a user equipment (UE) 102 (also referred to as a communication device or terminal), a 5G radio access network (5GRAN) 104, a 5G core network (5GCN) 106, one or more internal or external application functions (AFs) 108, and one or more data networks (DNs) 110.

[0102] An example 5G core network (CN) includes functional entities. The 5GCN 106 may include one or more access and mobility management functions (AMFs) 112, one or more session management functions (SMFs) 114, an authentication server function (AUSF) 116, a unified data management (UDM) 118, one or more user plane functions (UPFs) 120, a unified data repository (UDR) 122, and / or a network exposure function (NEF) 124. The UPF is controlled by the SMF (session management function) which receives policies from the PCF (policy control function).

[0103] The CN is connected to the UE via a radio access network (RAN). The 5G RAN may include one or more gNodeB (gNB) distributed unit (DU) functions, which are connected to one or more gNodeB (gNB) centralized unit (CU) functions. The RAN may include one or more access nodes.

[0104] A user plane function (UPF) known as a PDU Session Anchor (PSA) may be responsible for forwarding frames back and forth between the DN and the tunnel established over 5G towards the UE(s) exchanging traffic with the DN.

[0105] Possible mobile communication devices will now refer to Figure 2 Described in more detail, Figure 2 A schematic partial cross-sectional view of a communication device 200 is shown. Such a communication device is generally referred to as a user equipment (UE) or terminal. Suitable mobile communication devices can be provided by any device capable of sending and receiving radio signals. Non-limiting examples include a mobile station (MS) or mobile device, such as a mobile phone or a so-called "smart phone", a computer provided with a wireless interface card or other wireless interface facilities (e.g., a USB dongle), a personal data assistant (PDA) or a tablet provided with a wireless communication function, an IP voice (VoIP) phone, a portable computer, a desktop computer, an image capture terminal device, such as a digital camera, a game terminal device, a music storage and playback device, a vehicle-mounted wireless terminal device, a wireless endpoint, a mobile station, a laptop embedded device (LEE), a laptop mounted device (LME), a smart device, a wireless client device (CPE), or any combination of these, etc. The mobile communication device can provide, for example, a communication for carrying data such as voice, email (email), text messages, multimedia, etc. Therefore, users can be supplied and provided with a variety of services via their communication devices. Non-limiting examples of these services include two-way or multi-way calls, data communications or multimedia services or simply access to a data communications network system, such as the Internet. Users may also be provided with broadcast or multicast data. Non-limiting examples of content include downloads, television and radio programs, videos, advertisements, various alerts and other information.

[0106] The mobile device is typically provided with at least one data processing entity 201, at least one memory 202 and other possible components 203 for software and hardware assisted execution of the tasks it is designed to perform, including controlling access to and communication with access systems and other communication devices. Data processing, storage and other related components may be provided in appropriate circuit boards and / or chipsets. This feature is represented by reference numeral 204. The user can control the operation of the mobile device through a suitable user interface (such as a keyboard 205, voice commands, a touch screen or touchpad, a combination thereof, etc.). A display 208, a speaker and a microphone may also be provided. In addition, the mobile communication device may include appropriate connectors (wired or wireless) to other devices and / or for connecting external accessories (e.g., a hands-free device) to the mobile communication device.

[0107] The mobile device 200 may receive signals over the air or radio interface 207 via appropriate means for receiving, and may transmit signals via appropriate means for transmitting radio signals. Figure 2 In the mobile device, the transceiver arrangement is schematically represented by block 206. The transceiver arrangement 206 may be provided, for example, by a radio component and an associated antenna arrangement. The antenna arrangement may be arranged internally or externally to the mobile device.

[0108] Figure 3 An example of a control device 300 for a communication system is shown, for example, the control device 300 will be coupled to and / or used to control a station of the access system, such as a RAN node, such as a base station, eNB or gNB, a relay node, or a core network node, such as an MME or a serving gateway (S-GW) or a packet data network gateway (P-GW), or a core network function, such as an AMF / SMF, or a server or a host. The method can be implemented in a single control device or across more than one control device. The control device can be integrated with a node or module of the core network or RAN or located outside it. In some embodiments, the base station includes a separate control device unit or module. In other embodiments, the control device can be another network element, such as a radio network controller or a spectrum controller. In some embodiments, each base station can have such a control device and a control device provided in the radio network controller. The control device 300 can be arranged to provide control of communications in a service area of ​​the system. The control device 300 includes at least one memory 301, at least one data processing unit 302, 303 and an input / output interface 304. Via the interface, the control device may be coupled to a receiver and a transmitter of the base station.The receiver and / or the transmitter may be implemented as a radio front end or a remote radio head.

[0109] Proposals for artificial intelligence / machine learning (AI / ML) enhancements are currently being discussed. One option is to enable cross-domain machine learning so that entities (e.g., (multiple) UEs, RAN, management, 5G core, and (multiple) AFs) all participate in AI / ML operations. Cross-domain AI / ML may cause an increase in the attack surface, thereby increasing the possibility of attacking AI / ML models (e.g., data poisoning attacks), for example, during the training and / or inference phases. For example, a data poisoning attack may involve a malicious entity (e.g., those entities involved in AI / ML operations as described above) injecting tampered data to affect the behavior of a trained AI / ML model or the training of an AI / ML model.

[0110] In such an attack, it would be useful to detect whether the system has been compromised. It would also be useful to determine at which stage of the AI / ML model lifecycle the system was compromised and / or identify the attacker.

[0111] Data drift refers to the changed data distribution to which the AI / ML model is applied, while concept drift refers to the changed underlying goal or purpose for the AI / ML model. The main difference between data drift and concept drift is that data drift refers to the change in the input data used for modeling, while concept drift refers to the change in the relationship between the input features and the target variable that the model is trying to predict. Both data drift and concept drift can cause a degradation in the performance of machine learning models.

[0112] Figure 4 A flow chart of a method according to an example embodiment is shown.The method may be performed at a network analysis function (eg, AnLF), a RAN node, or an OAM.

[0113] At 401, the method includes: receiving a request from an analytics consumer for analytics information from a first machine learning model,

[0114] At 402, the method includes obtaining a first machine learning model.

[0115] In 403, the method includes: obtaining a second machine learning model, the second machine learning model is trained before the first machine learning model, and the first machine learning model and the second machine learning model have the same analysis identifier.

[0116] At 404, the method includes obtaining a first inference output from a first machine learning model and a second inference output from a second machine learning model.

[0117] In 405, the method includes: determining that the first machine learning model has been attacked based on the first inference output and the second inference output.

[0118] At 406, the method includes providing an indication to a network entity that the first machine learning model has been attacked.

[0119] The method may include determining a concept drift between the first inference output and the second inference output, and determining that the first machine learning model has been attacked based on the determined concept drift. The first machine learning model may be determined to have been attacked based on a concept drift threshold.

[0120] Figure 5 A flow chart of a method according to an example embodiment is shown. The method may be performed at a network analysis function (e.g., MTLF), a 5GC NF (e.g., OAM) (e.g., in a cross-domain model sharing concept), or a UE (e.g., in a UE model sharing concept).

[0121] In 501, the method includes: receiving a request for a first machine learning model from a network entity.

[0122] At 502, the method includes providing a first machine learning model to a network entity.

[0123] At 503, the method includes receiving an indication from a network entity that the first machine learning model has been attacked.

[0124] As reference Figure 5 The described method may include: receiving a request for first training data from at least one network function from a network entity, the first training data being used to train a first machine learning model; providing the first training data to the network entity; and receiving from the network entity: an indication of a network function in which the first machine learning model determined from at least one network function is attacked, the indication being based on the first training data.

[0125] As reference Figure 5 The described method may include training the first machine learning model using first training data before providing the first machine learning model to the network entity.

[0126] Figure 6 A flow chart of a method according to an example embodiment is shown. The method may be performed at an analytics data repository function, or any other suitable other data repository storing AI / ML models and sample training data.

[0127] In 601, the method includes receiving a request from a network entity for a machine learning model associated with an analysis identifier.

[0128] At 602, the method includes providing a machine learning model associated with an analysis identifier to a network entity.

[0129] The method may include receiving a request from a network entity for training data from at least one network function, the training data being used to train a machine learning model associated with an analysis identifier; and providing the training data to the network entity.

[0130] As reference Figure 6 The described methods may include training a machine learning model using training data before providing the machine learning model to a network entity. Figure 6 The machine learning models mentioned in the method can include Figure 4 The second machine learning model mentioned in the method.

[0131] The method may include providing a first inference output to an analysis consumer, obtaining feedback information based on the first inference output from the analysis consumer, and further determining that the first machine learning model has been attacked based on the feedback information.

[0132] Figure 7 An apparatus according to an example embodiment is shown. The method may be performed at an analysis consumer. The analysis consumer may be a UE, an OAM or other NF.

[0133] In 701, the method includes providing a request from an analytics consumer to a network entity, the requesting for analytics information from a first machine learning model, the first machine learning model being associated with an analytics identifier.

[0134] At 702, the method includes receiving, from a network entity, a first inference output from a first machine learning model.

[0135] In 703 , the method includes determining feedback information based on the first inference output.

[0136] At 704, the method includes providing feedback information to a network entity.

[0137] When the feedback is negative, such as reference Figure 7 The described method may include providing a request to a network entity for additional analytics information associated with the analytics identifier.

[0138] Feedback information from an analysis consumer may mean whether the requested analysis helped the analysis consumer obtain positive results in the process for which the analysis was requested (e.g., the feedback information is positive), or whether the received analysis caused a negative deterioration in the performance of its process (e.g., the feedback information is negative).

[0139] The method can provide data poisoning attack detection and malicious entity identification mechanisms. The method can use concept drift in the inferences generated from the AI / ML model and / or feedback mechanisms from analytical consumers requesting data to detect whether there is an attack, or a real data distribution change.

[0140] Furthermore, identification of the attacker (in the case of a positive attack) may be performed using training data analysis on past and current data.

[0141] The method may include: obtaining first training data used to train a first machine learning model from at least one NF, obtaining second training data used to train a second machine learning model from at least one NF, and determining a network function in which the first machine learning model is attacked from at least one network function based on the first training data and the second training data.

[0142] The method may include determining a data distribution drift based on the first training data and the second training data. The network function may be determined from a plurality of network functions based on a data distribution drift threshold.

[0143] Some example measures for determining data / concept drift may include: Jenson-Shannon divergence, Wasserstein distance, Kullback-Leibler divergence, population stability index, Euclidean distance, etc.

[0144] The example method may include providing an indication of the determined network function to a network entity. Alternatively or additionally, the example method may include locally marking the determined NF as suspicious. The example method may include excluding the determined network function when requesting data for subsequent training of the first machine learning model.

[0145] As reference Figure 4 The method may be performed at an ML model reasoning entity (eg, NWDAF ANLF). The network entity may include: NWDAF, such as an ML model training entity (eg, NWDAF MTLF).

[0146] Alternatively or additionally, in the example cross-domain model sharing concept, as referenced Figure 4 The method may be performed at a RAN node. The network entity may include an OAM / Management domain.

[0147] Alternatively or additionally, in the example UE model sharing concept, the UE may train the model (i.e. act as a network entity). In this case, as referenced Figure 4 The method may be performed at RAN / OAM.

[0148] The second machine learning model can be obtained from ADRF (SA2 architecture) or any other suitable other data repository that stores AI / ML models and sample training data in the cross-domain model sharing concept or UE model sharing concept.

[0149] Figure 8 A signaling diagram for NWDAF to NWDAF model sharing according to an example embodiment is shown.

[0150] In this example embodiment, the ADRF contains ML model versions corresponding to the analysis IDs for which they were trained (these are examples of second machine learning models). The ADRF also contains sample data / data distributions for each NF that contribute to providing training data during the training of a particular ML model (this is an example of second training data used to train a second machine learning model from multiple network functions).

[0151] In step 1, the analysis consumer (NFc) sends a request containing an AnLF for a specific analysis ID to the NWDAF. This is an example of receiving a request for analysis information from a machine learning model from an analysis consumer. In this example embodiment, the UE includes an analysis consumer.

[0152] In step 2, AnLF requests the ML model corresponding to the requested analysis ID, and the data distribution / sample data from each NF that will be used to train the ML model for the given analysis ID.

[0153] In steps 3, 4, and 5, the NWDAF MTLF sends requests for data collection to other NFs directly or via the DCCF, and may receive maliciously tampered data (in the case where a malicious NF wants to perform a data poisoning attack). The NWDAF MTLF trains an ML model (e.g., model "X") and sends it to the AnLF along with the sample data / data distribution requested in step 2 for each NF that contributed to the training data.

[0154] Steps 2 to 4 are examples of obtaining a machine learning model and obtaining first training data used to train the first machine learning model from at least one network function.

[0155] In step 6, AnLF sends a request to ADRF for a trained model version for the same analysis ID "abc" (if one exists), and receives the same response (ie, ADRF sends model "Y").

[0156] In step 7 , AnLF obtains analytical reasoning output using model “X” received from MTLF in step 5 , and also using the model version (model “Y”) received from ADRF in step 6 .

[0157] This is an example of obtaining a second machine learning model (e.g., model “Y”) that is trained before a first machine learning model (e.g., model “X”), the first machine learning model and the second machine learning model having the same analysis identifier.

[0158] In step 8, AnLF calculates the drift between the analytical inference outputs obtained from two ML models (X and Y). If the drift is more than a certain threshold, the new model "X" obtained from MTLF is marked as suspicious.

[0159] Step 8 is an example of determining a concept drift between the first inference output and the second inference output, and determining that the first machine learning model has been attacked based on the determined concept drift and the concept drift threshold.

[0160] In step 9, AnLF sends the analysis generated via the new model received from MTLF to the analysis consumer and also requests its feedback.

[0161] The analysis consumer provides positive / negative feedback in step 10. In case of negative feedback, the analysis is also rejected and a new analysis is requested from the AnLF.

[0162] Steps 9 and 10 are examples of providing a first inference output to an analysis consumer, obtaining feedback information based on the first inference output from the analysis consumer, and determining that the first machine learning model has been additionally attacked based on the feedback information.

[0163] In step 11, in case of negative feedback, AnLF initiates detection of data poisoning attack.

[0164] In step 12, the AnLF sends a request to the ADRF for historical data samples / data distributions for the NF(s) whose information was obtained in step 5 and which may have provided poisoned data in steps 4a / 4b.

[0165] In step 13, the ADRF sends data for the NF(s) providing training data to train the previous model version for the same analysis ID. This is an example of obtaining second training data used to train a second machine learning model from at least one network function.

[0166] In step 14 , the AnLF analysis compares the sample data / data distribution for each NF received in step 5 with the data received in step 13 for the same NF.

[0167] In step 15, in case the deviation between the data distributions for a particular NF exceeds a threshold, the AnLF locally marks the NF as suspicious and does not request data for the next model training. The AnLF can then notify the MTLF about the suspicious NF and the attack.

[0168] This is an example of determining, from at least one network function, a network function to which a first machine learning model is attacked based on first training data and second training data based on a data distribution drift between the first training data and the second training data, and a data distribution drift threshold.

[0169] In step 8, if the concept drift seems very sudden and is more likely to be caused by an attack, the process can go directly to step 12 and continue detection. This can help avoid unnecessary rollbacks after negative feedback. If step 8 does not give a clear indication, the method can continue to step 9.

[0170] In an alternative version after step 11, AnLF may instruct MTLF to initiate detection of a poisoning attack so that steps 12, 13, 14 and 15 may be performed at MTLF, and then MTLF may provide a response to AnLF in a new step 16 regarding the confirmation of the attack.

[0171] This approach can be applied to any other domain or cross-domain model sharing and analysis concepts. In an example embodiment, when the RAN receives the model from the OAM / Management domain, analysis is performed at the RAN node and then similar solutions as described above will be applied to detect issues.

[0172] Fig. 9 An example call flow in which the analysis is performed at a RAN node is shown. The steps of the method are as shown in reference Figure 8 As described, the RAN node replaces the AnLF, the 5GC (e.g., OAM) replaces the MTLF, and the ML repository replaces the ADRF.

[0173] In UE model sharing concept, UE can train the model and provide it to RAN / OAM (e.g. beamforming use case, etc.). UE should use local data to train the model. UE should also push the model and local data to the server / OAM (repository). Similar solutions can be applied to detect data poisoning attacks when the network detects the attack.

[0174] An apparatus may include components for receiving a request for analytical information from a first machine learning model from an analytics consumer, obtaining the first machine learning model, obtaining a second machine learning model, the second machine learning model being trained before the first machine learning model, the first machine learning model and the second machine learning model having the same analytics identifier, obtaining a first inference output from the first machine learning model, and a second inference output from the second machine learning model, determining based on the first inference output and the second inference output that the first machine learning model has been attacked, and providing an indication to a network entity that the first machine learning model has been attacked.

[0175] An apparatus may include components for receiving a request for a first machine learning model from a network entity, providing the first machine learning model, and receiving an indication from the network entity that the first machine learning model has been attacked.

[0176] The apparatus may include components for providing a request from an analytics consumer to a network entity for analytics information from a first machine learning model associated with an analytics identifier, receiving a first inference output from the first machine learning model from the network entity, determining feedback information based on the first inference output, and providing feedback information to the network entity.

[0177] The apparatus may include components for receiving a request for receiving a request for a machine learning model associated with an analysis identifier from a network entity, and providing the machine learning model associated with the analysis identifier to the network entity.

[0178] The apparatus may comprise a network entity, such as user equipment.

[0179] The apparatus may include a network entity, such as a user equipment (e.g., a mobile phone) or a network function (e.g., an analysis network function (e.g., AnLF or MTLF), a 5GC NF, an OAM or a repository function (e.g., ADRF)) or a RAN node, and may be a network entity, or included in a network entity, or a chipset for performing at least some actions of / for a network entity.

[0180] It should be understood that these devices may include or be coupled to other units or modules used for transmission and / or reception, such as radio components or radio heads. Although these devices are described as one entity, different modules and memories may be implemented in one or more physical or logical entities.

[0181] It should be noted that although some embodiments have been described with respect to 5G networks, similar principles may be applied to other networks and communication systems, such as 6G networks or 5G-Advanced networks. Therefore, although certain embodiments are described above by way of example with reference to certain example architectures for wireless networks, technologies and standards, the embodiments may be applied to any other suitable form of communication system in addition to the communication system shown and described herein.

[0182] It is also noted herein that while example embodiments are described above, there are numerous variations and modifications which may be made to the disclosed solution without departing from the scope of the present invention.

[0183] As used herein, “at least one of: ” and “at least one of ” and similar expressions, where a list of two or more elements is connected by “and” or “or”, mean at least any one element, or at least two or more elements, or at least all the elements.

[0184] In general, various embodiments may be implemented in hardware or dedicated circuit systems, software, logic, or any combination thereof. Some aspects of the present disclosure may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the present disclosure is not limited thereto. Although various aspects of the present disclosure may be illustrated. And described as a block diagram, flow chart, or using some other graphical representation, it is fully understood that, as non-limiting examples, these boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general hardware or controllers or other computing devices, or some combination thereof.

[0185] As used in this application, the term "circuitry" may refer to one, more, or all of the following:

[0186] (a) hardware circuit implementation only (e.g., implementation using only analog and / or digital circuitry) and

[0187] (b) a combination of hardware circuitry and software such as (where applicable):

[0188] (c) a combination of analog and / or digital hardware circuits and software / firmware and

[0189] (d) any portion of hardware processor(s) (including digital signal processors) with software, software and memory(s) that work together to enable a device (such as a mobile phone or server) to perform various functions) and

[0190] I(multiple) hardware circuits and / or(multiple) processors (such as(multiple) microprocessors or portions of(multiple) microprocessors)) require software (e.g., firmware) to operate, but when the software is not required for operation, the software may not be present.

[0191] This definition of circuitry applies to all uses of the term in this application, including in any claims. As a further example, as used in this application, the term "circuitry" also covers an implementation of only a hardware circuit or processor (or multiple processors) or a portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. For example, if applicable to a particular claim element, the term "circuitry" also covers an integrated circuit, such as a baseband integrated circuit or processor integrated circuit for a mobile device, or a similar integrated circuit in a server, cellular network device, or other computing or network device.

[0192] The embodiments of the present disclosure may be implemented by computer software executable by a data processor of a mobile device (such as a processor entity), or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) include software routines, applets and / or macros that can be stored in a data storage medium readable by any device, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof.

[0193] In this regard, it should also be noted that any box of the logic flow shown in the figure may represent a program step, or an interconnected logic circuit, box and function, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media, such as memory chips or memory blocks implemented in processors, magnetic media (such as hard disks or floppy disks), and optical media (such as, for example, DVDs and their data variants CDs). Physical media are non-transient media. The term "non-transient" used herein is a limitation of the medium itself (i.e., tangible, not a signal), not a limitation of data storage persistence (e.g., RAM vs. ROM).

[0194] The memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The data processor may be of any type suitable for the local technical environment and may include, as non-limiting examples, one or more of the following: a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an FPGA, a gate-level circuit, and a processor based on a multi-core processor architecture.

[0195] Embodiments of the present disclosure may be practiced in various components, such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available to convert logic level designs into semiconductor circuit designs for etching and formation on semiconductor substrates.

[0196] The scope of protection sought by various embodiments of the present disclosure is defined by the independent claims. Embodiments and features described in this specification that do not belong to the scope of the independent claims (if any) should be interpreted as examples that help understand the various embodiments of the present disclosure.

[0197] The foregoing description has provided, by way of non-limiting examples, a complete and informative description of exemplary embodiments of the present disclosure. However, various modifications and adaptations may become apparent to those skilled in the relevant art when the foregoing description is read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of the present disclosure will still fall within the scope of the present invention as defined in the appended claims. Indeed, there are additional embodiments that include a combination of one or more embodiments with any other embodiments previously discussed.

Claims

1. An apparatus for a communication network, comprising means for: receiving a request from an analytics consumer for analytics information from the first machine learning model; Obtaining the first machine learning model; Obtain a second machine learning model, the second machine learning model being trained before the first machine learning model, the first machine learning model and the second machine learning model having the same analysis identifier; Obtaining a first inference output from the first machine learning model and a second inference output from the second machine learning model; Based on the first reasoning output and the second reasoning output, determining that the first machine learning model has been attacked; as well as Providing an indication to a network entity that the first machine learning model has been attacked.

2. The apparatus of claim 1 , comprising a component for determining a concept drift between the first inference output and the second inference output, and determining that the first machine learning model has been attacked based on the determined concept drift.

3. The apparatus of claim 2, comprising a component for determining that the first machine learning model has been attacked based on a concept drift threshold.

4. The device according to any one of claims 1 to 3 comprises components for: providing the first inference output to the analysis consumer; obtaining feedback information based on the first inference output from the analysis consumer; and further determining that the first machine learning model has been attacked based on the feedback information.

5. The apparatus according to any one of claims 1 to 4, comprising means for: obtaining first training data used to train the first machine learning model from at least one network function; obtaining second training data from the at least one network function for training the second machine learning model; And based on the first training data and the second training data, determining the network function in which the first machine learning model is attacked from the at least one network function. 6 . The apparatus of claim 5 , comprising means for determining a data distribution drift based on the first training data and the second training data. 7 . The apparatus of claim 6 , comprising means for determining the network function from the plurality of network functions based on a data distribution drift threshold.

8. An apparatus according to any of claims 5 to 7, comprising means for providing an indication of the determined network functionality to the network entity.

9. An apparatus according to any one of claims 1 to 8, comprising a component for excluding the determined network function when requesting data for subsequent training of the first machine learning model.

10. The apparatus according to any one of claims 1 to 9, wherein the network entity comprises: Network analysis functions, operations and management functions, or user equipment.

11. An apparatus for a communication network, comprising means for: receiving a request for a first machine learning model from a network entity; providing the first machine learning model; and Receiving an indication from the network entity that the first machine learning model has been attacked.

12. The apparatus of claim 11, comprising means for: receiving a request from the network entity for first training data from at least one network function, the first training data being used to train the first machine learning model; Providing the first training data to the network entity; and receiving from the network entity: an indication of a network function determined from the at least one network function in which the first machine learning model is attacked, the indication being based on the first training data.

13. The apparatus according to claim 11 or claim 12, wherein the network entity comprises: Analyze network functions, radio access network nodes, or operations and management functions.

14. An apparatus for a communication network, comprising means for: receiving a request for receiving, from a network entity, a request for a machine learning model associated with an analysis identifier; and The machine learning model associated with the analysis identifier is provided to the network entity.

15. The apparatus of claim 14, comprising components for: receiving a request from the network entity for training data from at least one network function, the training data being used to train the machine learning model associated with the analysis identifier; and providing the training data to the network entity.

16. An apparatus according to claim 14 or claim 15, wherein the apparatus comprises: An analysis data repository function, or a data repository including the machine learning model and the training data.

17. The apparatus according to any one of claims 14 to 16, wherein the network entity comprises: Analyze network functions, or radio access network nodes.

18. An apparatus for a communication network, comprising means for: providing a request from an analytics consumer to a network entity, the request for analytics information from a first machine learning model, the first machine learning model being associated with an analytics identifier; receiving, from the network entity, a first inference output from the first machine learning model; determining feedback information based on the first reasoning output; as well as The feedback information is provided to the network entity.

19. The apparatus of claim 18, wherein the feedback information is negative and comprises means for providing a request to the network entity for additional analysis information associated with the analysis identifier.

20. The apparatus according to any one of claims 18 or 19, wherein the network entity comprises: Analyze network functions, or radio access nodes.

21. A method for use in a communication network, comprising: receiving a request from an analytics consumer for analytics information from the first machine learning model; Obtaining the first machine learning model; Obtaining a second machine learning model, wherein the second machine learning model is trained before the first machine learning model, and the first machine learning model and the second machine learning model have the same analysis identifier; Obtaining a first inference output from the first machine learning model and a second inference output from the second machine learning model; Based on the first reasoning output and the second reasoning output, determining that the first machine learning model has been attacked; as well as Providing an indication to a network entity that the first machine learning model has been attacked.

22. A method for use in a communication network, comprising: receiving a request for a first machine learning model from a network entity; providing the first machine learning model; as well as Receiving an indication from the network entity that the first machine learning model has been attacked.

23. A method for use in a communication network, comprising: receiving a request for receiving, from a network entity, a request for a machine learning model associated with an analysis identifier; as well as The machine learning model associated with the analysis identifier is provided to the network entity.

24. A method for use in a communication network, comprising: providing a request from an analytics consumer to a network entity, the request for analytics information from a first machine learning model, the first machine learning model being associated with an analytics identifier; receiving, from the network entity, a first inference output from the first machine learning model; determining feedback information based on the first reasoning output; as well as The feedback information is provided to the network entity.

25. An apparatus for a communication network, comprising: at least one processor, and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least: receiving a request from an analytics consumer for analytics information from the first machine learning model; Obtaining the first machine learning model; Obtaining a second machine learning model, wherein the second machine learning model is trained before the first machine learning model, and the first machine learning model and the second machine learning model have the same analysis identifier; Obtaining a first inference output from the first machine learning model and a second inference output from the second machine learning model; Based on the first reasoning output and the second reasoning output, determining that the first machine learning model has been attacked; as well as Providing an indication to a network entity that the first machine learning model has been attacked.

26. An apparatus for a communication network, comprising: at least one processor, and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least: receiving a request for a first machine learning model from a network entity; providing the first machine learning model; and Receiving an indication from the network entity that the first machine learning model has been attacked.

27. An apparatus for a communication network, comprising: at least one processor, and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least: receiving a request for receiving, from a network entity, a request for a machine learning model associated with an analysis identifier; as well as The machine learning model associated with the analysis identifier is provided to the network entity.

28. An apparatus for a communication network, comprising: at least one processor, and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least: providing a request from an analytics consumer to a network entity, the request for analytics information from a first machine learning model, the first machine learning model being associated with an analytics identifier; receiving, from the network entity, a first inference output from the first machine learning model; determining feedback information based on the first reasoning output; as well as The feedback information is provided to the network entity.

29. A computer-readable medium comprising instructions which, when executed by an apparatus, cause the apparatus to at least perform the following: receiving a request from an analytics consumer for analytics information from the first machine learning model; Obtaining the first machine learning model; Obtain a second machine learning model, the second machine learning model being trained before the first machine learning model, the first machine learning model and the second machine learning model having the same analysis identifier; Obtaining a first inference output from the first machine learning model and a second inference output from the second machine learning model; Based on the first reasoning output and the second reasoning output, determining that the first machine learning model has been attacked; as well as Providing an indication to a network entity that the first machine learning model has been attacked.

30. A computer readable medium comprising instructions which, when executed by an apparatus, cause the apparatus to at least perform the following: receiving a request for a first machine learning model from a network entity; providing the first machine learning model; and Receiving an indication from the network entity that the first machine learning model has been attacked.

31. A computer readable medium comprising instructions which, when executed by an apparatus, cause the apparatus to at least perform the following: receiving a request for receiving, from a network entity, a request for a machine learning model associated with an analysis identifier; and The machine learning model associated with the analysis identifier is provided to the network entity.

32. A computer readable medium comprising instructions which, when executed by an apparatus, cause the apparatus to at least perform the following: providing a request from an analytics consumer to a network entity, the request for analytics information from a first machine learning model, the first machine learning model being associated with an analytics identifier; receiving, from the network entity, a first inference output from the first machine learning model; determining feedback information based on the first reasoning output; as well as The feedback information is provided to the network entity.

Images