Machine unlearning and auditing in a communication network environment with federated learning

Abstract

Federated learning techniques are provided for removing client data and its influence from a machine learning model (eg., machine unlearning), as well as auditing techniques for verifying client data and influence removal from machine learning models.

Description

MACHINE UNLEARNING AND AUDITING IN A COMMUNICATION NETWORK ENVIRONMENT WITH FEDERATED LEARNINGRelated Applications

[0001] This patent application claims the benefit of priority of India Patent Application No.202511027098, filed on March 24, 2025, which is hereby incorporated by reference as if reproduced in its entirety.Field

[0002] The field relates generally to communication networks, and more particularly, but not exclusively, to security management in such communication networks.Background

[0003] This section introduces aspects that may be helpful in facilitating a better understanding of the inventions. Accordingly, the statements of this section are to be read in this light and are not to be understood as admissions about what is in the prior art or what is not in the prior art.

[0004] Advancements in communication network technologies have rapidly progressed over recent years.

[0005] Fourth generation (4G) wireless mobile telecommunications technology, also known as Long Term Evolution (LTE) technology, provided high-capacity mobile multimedia with high data rates particularly for human interaction, as compared with previous generations of communication networks.

[0006] Fifth generation (5G) technology currently provides not only for human interaction use cases, but also for machine type communications in so-called Internet of Things (loT) networks. While 5G networks enable massive loT services (e.g., very large numbers of limited capacity devices) and mission-critical loT services (e.g., requiring high reliability), improvements over 4G communication services are supported in the form of enhanced mobile broadband (eMBB) services providing improved wireless Internet access for mobile devices.

[0007] Sixth generation (6G) technology is now being developed for communication networks that differs from 5G technology by offering, inter alia, significant improvements in speed and latency (e.g., the Ultra-Reliable Low-Latency Communication (URLLC) service that began with 5G is being refined and improved in 6G to address more stringent connectivity requirements), as well as the capability to sense a physical environment through expanded spectrum band usage. Such sensing capability enables creation of a digital twin of the physical environment which leads to new applications such as,but not limited to, highly accurate localization and immersive experiences. Furthermore, in 6G technology, artificial intelligence (Al) applications, which may include machine learning (ML) applications, are intended to be more readily utilized to facilitate various communication network functionalities.

[0008] However, security management is an important consideration in any communication network environment - and now especially ones that provide for applications such as localization, immersion, Al, and the like. Moreover, security management is an ongoing consideration due to continuing attempts to improve the architectures and protocols associated with communication networks in order to increase network efficiency and / or subscriber convenience. Accordingly, security management can present significant technical challenges.Summary

[0009] Illustrative embodiments provide techniques for security management in machine learning (ML) models used in a communication network environment. For example, security management techniques are provided for machine unlearning and auditing in a federated learning (FL) environment.

[0010] In one illustrative embodiment, a method includes receiving a request from a first entity to remove first data that the first entity contributed to the training of a machine learning model in conjunction with at least a second entity that contributed second data to the training of the machine learning model. The method further includes removing the first data of the first entity from the machine learning model. The method further includes initiating an inference test with the at least a second entity on the machine learning model after removal of the first data of the first entity from the machine learning model.

[0011] In one illustrative embodiment, a method includes initiating an FL process with a plurality of FL clients to train a machine learning model, wherein the initiating is performed by an FL server deployed in a radio access network and at least a portion of the FL clients are respectively deployed in sets of user equipment. The method further includes receiving a request from one of the plurality of FL clients to remove data that the requesting FL client contributed to the training of the machine learning model, wherein the request is received as part of a first type of radio resource control message. The method further includes removing the data of the requesting FL client from the machine learning model.

[0012] In one illustrative embodiment, a method includes, in conjunction with participating in a federated learning (FL) process, as an FL client along with one or more other FL clients and an FL server, to train a machine learning model, participate in a negotiation process with the FL server toagree on a method for verifying removal of data that the user equipment contributed to the training of the machine learning model.

[0013] Further illustrative embodiments are provided in the form of a non-transitory computer readable medium having embodied therein executable program code that when executed by a processor causes the processor to perform the above and / or other steps, operations, and the like. Still further illustrative embodiments comprise an apparatus with a processor and a memory configured to perform the above and / or other steps, operations, and the like. Some illustrative embodiments comprise a system configured to perform the above and / or other steps, operations, and the like. Further, some illustrative embodiments comprise an apparatus or a system comprising means for performing the above and / or other steps, operations, and the like.

[0014] Advantageously, illustrative embodiments provide federated learning techniques for removing client data and its influence from an ML model (e.g., machine unlearning). Other illustrative embodiments provide auditing techniques for verifying client data and influence removal from ML models.

[0015] These and other features and advantages of embodiments described herein will become more apparent from the accompanying drawings and the following detailed description.Brief Description of the Drawings

[0016] FIG. 1 illustrates a communication network environment with which one or more illustrative embodiments may be implemented.

[0017] FIG. 2 illustrates user equipment and entities with which one or more illustrative embodiments may be implemented.

[0018] FIGS. 3A through 3C illustrate respective radio access network-based procedures for machine unlearning in a communication network environment according to an illustrative embodiment.

[0019] FIGS. 4A and 4B illustrate respective core network-based procedures for machine unlearning in a communication network environment according to an illustrative embodiment.

[0020] FIGS. 5A and 5B illustrate respective procedures for auditing machine unlearning in a communication network environment according to an illustrative embodiment.Detailed Description

[0021] Embodiments will be illustrated herein in conjunction with example communication systems and associated techniques for security management in communication systems. It should be understood, however, that the scope of the claims is not limited to particular types of communication systems and / or processes disclosed. Embodiments can be implemented in a wide variety of othertypes of communication systems, using alternative processes and operations. For example, although illustrated in the context of wireless cellular systems utilizing the 3rd Generation Partnership Project (3GPP) system elements such as a 3GPP next generation system (5G), the disclosed embodiments can be adapted in a straightforward manner to a variety of other types of communication systems such as 6G communication systems.

[0022] In accordance with illustrative embodiments implemented in 5G / 6G communication system environments, one or more 3GPP technical specifications (TS) and technical reports (TR) may provide further explanation of network elements / functions and / or operations that may interact with parts of the inventive solutions. By way of example only, Technical Specification (TS) 33.501 , entitled “Technical Specification Group Services and System Aspects; Security Architecture and Procedures for the 5G System,” the disclosure of which is incorporated by reference herein in its entirety, describes security management details applicable to 5G and other networks. Other 3GPP TS / TR documents may provide other details that one of ordinary skill in the art will realize, for example, 3GPP TS 23.288, entitled “Technical Specification Group Services and System Aspects; Architecture Enhancements for 5G System (5GS) to Support Network Data Analytics Services,” the disclosure of which is incorporated by reference herein in its entirety. Note that 3GPP TS / TR documents are non-limiting examples of communication network standards (e.g., specifications, procedures, reports, requirements, recommendations, and the like). However, while well-suited for 5G-related and other 3GPP standards, embodiments are not necessarily intended to be limited to any particular standards.

[0023] It is to be understood that the term 5G network, and the like (e.g., 5G system, 5G communication system, 5G environment, 5G communication environment etc.), in some illustrative embodiments, may comprise all or part of an access network and all or part of a core network. However, the term 5G network, and the like, may also occasionally be used interchangeably herein with the term 5GC network, and the like, without any loss of generality, since one of ordinary skill in the art understands any distinctions. Also, it is to be understood that terms and descriptions used for 5G networks can apply to 6G and other networks.

[0024] Prior to describing illustrative embodiments, a general description of certain main components of a communication network environment will be described below in the context of FIGS. 1 and 2.

[0025] FIG. 1 shows a communication system 100 within which illustrative embodiments are implemented. It is to be understood that the elements shown in communication system 100 are intended to represent some main functions provided within the system, e.g., control plane functions, user plane functions, etc. As such, the blocks shown in FIG. 1 reference specific elements in 5Gnetworks that provide some of these main functions. However, other network elements may be used to implement some or all of the main functions represented. Also, it is to be understood that not all functions of a 5G network are depicted in FIG. 1. Rather, at least some functions that facilitate an explanation of illustrative embodiments are represented. Subsequent figures may depict some additional elements / functions (i.e., network entities).

[0026] Accordingly, as shown, communication system 100 comprises user equipment (UE) 102 that communicates via an air interface 103 with an access point 104. It is to be understood that UE 102 may use one or more other types of access points (e.g., access functions, networks, etc.) to communicate with the 5GC network other than a gNB. By way of example only, the access point 104 may be any 5G access network (gNB), an untrusted non-3GPP access network that uses an Non-3GPP Interworking Function (N3IWF), a trusted non-3GPP network that uses a Trusted Non-3GPP Gateway Function (TNGF) or wireline access that uses a Wireline Access Gateway Function (W-AGF) or may correspond to a legacy access point (e.g., eNB). Furthermore, access point 104 may be a wireless local area network (WLAN) access point as may be applicable to illustrative embodiments described herein.

[0027] The UE 102 may be a mobile station, and such a mobile station may comprise, by way of example, a mobile telephone, a computer, an loT device, or any other type of communication device. The term “user equipment” as used herein is therefore intended to be construed broadly, so as to encompass a variety of different types of mobile stations, subscriber stations or, more generally, communication devices, including examples such as a combination of a data card inserted in a laptop or other equipment such as a smart phone. Such communication devices are also intended to encompass devices commonly referred to as access terminals.

[0028] In one illustrative embodiment, UE 102 is comprised of a Universal Integrated Circuit Card (UICC) part and a Mobile Equipment (ME) part. The UICC is the user-dependent part of the UE and contains at least one Universal Subscriber Identity Module (USIM) and appropriate application software. The USIM securely stores a permanent subscription identifier and its related key, which are used to uniquely identify and authenticate subscribers to access networks. The ME is the userindependent part of the UE and contains terminal equipment (TE) functions and various mobile termination (MT) functions. Alternative illustrative embodiments may not use UICC-based authentication, e.g., a Non-Public (Private) Network (NPN).

[0029] Note that, in one example, the permanent subscription identifier is an International Mobile Subscriber Identity (IMSI) unique to the UE. In one embodiment, the IMSI is a fixed 15-digit lengthand consists of a 3-digit Mobile Country Code (MCC), a 3-digit Mobile Network Code (MNC), and a 9-digit Mobile Station Identification Number (MSIN). In a 5G communication system, an IMSI is referred to as a Subscription Permanent Identifier (SUPI). In the case of an IMSI as a SUPI, the MSIN provides the subscriber identity. Thus, only the MSIN portion of the IMSI typically needs to be encrypted. The MNC and MCC portions of the IMSI provide routing information, used by the serving network to route to the correct home network. When the MSIN of a SUPI is encrypted, it is referred to as Subscription Concealed Identifier (SUCI). Another example of a SUPI uses a Network Access Identifier (NAI). NAI is typically used for loT communication.

[0030] The access point 104 is illustratively part of a radio access network or RAN of the communication system 100. Such a radio access network may comprise, for example, a 5G System having a plurality of base stations. Components of a radio access network may, more generally, be considered “radio access entities.”

[0031] Further, the access point 104 in this illustrative embodiment is operatively coupled to an Access and Mobility Management Function (AMF) 106. In a 5G network, the AMF supports, inter alia, mobility management (MM) and security anchor (SEAF) functions.

[0032] AMF 106 in this illustrative embodiment is operatively coupled to (e.g., uses the services of) other network functions 108. As shown, some of these other network functions 108 include, but are not limited to, a Network Data Analytics Function (NWDAF) and an Analytics Data Repository Function (ADRF). These listed network function examples are typically implemented in the home network of the UE subscriber, further explained below.

[0033] The NWDAF is a network function that collects data from various network functions, application functions, as well as operations, administration, and management (OAM) systems, and operational support systems. OAM refers to processes and tools used to manage and maintain a communication network to ensure that network runs smoothly and efficiently. The NWDAF is configured to facilitate the way data is produced and consumed, as well as to generate analytical insights and take actions based on the analytical insights. The ADRF is a network function that stores raw data and associated analytics generated from the network, allowing for further analysis and insights into network performance and user behavior. The ADRF is configured to serve as a data repository for 5G network analytics.

[0034] Other network functions 108 may include network functions that can act as service producers (NFp) and / or service consumers (NFc). Note that any network function can be a service producer for one service and a service consumer for another service. Further, when the service being providedincludes data, the data-providing NFp is referred to as a data producer, while the data-requesting NFc is referred to as a data consumer. A data producer may also be an NF that generates data by modifying or otherwise processing data produced by another NF. Note that NFs may, more generally, be considered “network entities.”

[0035] Note that a UE, such as UE 102, is typically subscribed to what is referred to as a Home Public Land Mobile Network (HPLMN) in which some or all of the functions 106 and 108 reside. Alternatively the UE, such as UE 102, may receive services from a Non-Public Network (NPN) where these functions may reside. The HPLMN is also referred to as the Home Environment (HE). If the UE is roaming (not in the HPLMN), it is typically connected with a Visited Public Land Mobile Network (VPLMN) also referred to as a visited network, while the network that is currently serving the UE is also referred to as a serving network. In the roaming case, some of the functions 106 and 108 can reside in the VPLMN, in which case, functions in the VPLMN communicate with functions in the HPLMN as needed. However, in a non-roaming scenario, access and mobility management functions 106 and the other network functions 108 reside in the same communication network, i.e., HPLMN. Embodiments described herein, unless otherwise specified, are not necessarily limited by which functions reside in which PLMN (i.e., HPLMN or VPLMN).

[0036] The access point 104 is also operatively coupled (via one or more of functions 106 and / or 108) to a Session Management Function (SMF) 110, which is operatively coupled to a User Plane Function (UPF) 112. UPF 112 is operatively coupled to a Packet Data Network, e.g., Internet 114. Note that the thicker solid lines in this figure denote a user plane (UP) of the communication network, as compared to the thinner solid lines that denote a control plane (CP) of the communication network. It is to be appreciated that network (e.g., Internet) 114 in FIG. 1 may additionally or alternatively represent other network infrastructures including, but not limited to, cloud computing infrastructure and / or edge computing infrastructure. Further typical operations and functions of such network elements are not described here since they are not the focus of the illustrative embodiments and may be found in appropriate 3GPP 5G documentation. Note that functions shown in 106, 108, 110 and 112 are examples of network functions (NFs).

[0037] It is to be appreciated that this particular arrangement of system elements is an example only, and other types and arrangements of additional or alternative elements can be used to implement a communication system in other embodiments. For example, in other embodiments, the communication system 100 may comprise other elements / functions not expressly shown herein.

[0038] Accordingly, the FIG. 1 arrangement is just one example configuration of a wireless cellular system, and numerous alternative configurations of system elements may be used. For example, although only single elements / functions are shown in the FIG. 1 embodiment, this is for simplicity and clarity of description only. A given alternative embodiment may of course include larger numbers of such system elements, as well as additional or alternative elements of a type commonly associated with conventional system implementations.

[0039] It is also to be noted that while FIG. 1 illustrates system elements as singular functional blocks, the various subnetworks that make up the network may be partitioned into so-called network slices. Network slices (network partitions) are logical networks that provide specific network capabilities and network characteristics that can support a corresponding service type, optionally using network function virtualization (NFV) on a common physical infrastructure. With NFV, network slices are instantiated as needed for a given service, e.g., eMBB service, massive loT service, and mission-critical loT service. A network slice or function is thus instantiated when an instance of that network slice or function is created. In some embodiments, this involves installing or otherwise running the network slice or function on one or more host devices of the underlying physical infrastructure. UE 102 is configured to access one or more of these services via access point 104.

[0040] FIG. 2 is a block diagram illustrating computing architectures for various participants in methodologies according to illustrative embodiments. More particularly, system 200 is shown comprising user equipment (UE) 202 and a plurality of entities 204-1, . . . . , 204-N. For example, in illustrative embodiments and with reference back to FIG. 1, UE 202 can represent UE 102, while entities 204-1, . . . , 204-N can represent functions 106 and 108 (i.e., network entities such as, but not limited to, NWDAF, NRF, PCF, etc.), and as will be described in illustrative embodiments herein, a Mobile Security Management Function (MSMF), as well as access point 104 (i.e., radio access entity such as, but not limited to, a RAN node or g N B) . It is to be appreciated that the UE 202 and entities 204-1, . . . . , 204-N are configured to interact to provide security management and other techniques described herein.

[0041] The user equipment 202 comprises a processor 212 coupled to a memory 216 and interface circuitry 210. The processor 212 of the user equipment 202 includes a security management processing module 214 that may be implemented at least in part in the form of software executed by the processor. The security management processing module 214 performs security management described in conjunction with subsequent figures and otherwise herein. The memory 216 of the userequipment 202 includes a security management storage module 218 that stores data generated or otherwise used during security management operations.

[0042] Each of the entities (individually or collectively referred to herein as 204) comprises a processor 222 (222-1, . . . , 222-N) coupled to a memory 226 (226-1, . . . , 226-N) and interface circuitry 220 (220-1, . . . , 220-N). Each processor 222 of each entity 204 includes a security management processing module 224 (224-1, . . . , 224-N) that may be implemented at least in part in the form of software executed by the processor 222. The security management processing module 224 performs security management operations described in conjunction with subsequent figures and otherwise herein. Each memory 226 of each entity 204 includes a security management storage module 228 (228-1, . . . , 228-N) that stores data generated or otherwise used during security management operations.

[0043] The processors 212 and 222 may comprise, for example, microprocessors such as central processing units (CPUs), application-specific integrated circuits (ASICs), digital signal processors (DSPs) or other types of processing devices, as well as portions or combinations of such elements.

[0044] The memories 216 and 226 may be used to store one or more software programs that are executed by the respective processors 212 and 222 to implement at least a portion of the functionality described herein. For example, security management operations and other functionality as described in conjunction with subsequent figures and otherwise herein may be implemented in a straightforward manner using software code executed by processors 212 and 222.

[0045] A given one of the memories 216 and 226 may therefore be viewed as an example of what is more generally referred to herein as a computer program product or still more generally as a computer or processor readable (non-transitory or storage) medium that has executable program code embodied therein. Other examples of computer or processor readable media may include disks or other types of magnetic or optical media, in any combination. Illustrative embodiments can include articles of manufacture comprising such computer program products or other computer or processor readable media.

[0046] Further, the memories 216 and 226 may more particularly comprise, for example, electronic random-access memory (RAM) such as static RAM (SRAM), dynamic RAM (DRAM) or other types of volatile or non-volatile electronic memory. The latter may include, for example, non-volatile memories such as flash memory, magnetic RAM (MRAM), phase-change RAM (PC-RAM) or ferroelectric RAM (FRAM). The term “memory” as used herein is intended to be broadly construed, and may additionallyor alternatively encompass, for example, a read-only memory (ROM), a disk-based memory, or other type of storage device, as well as portions or combinations of such devices.

[0047] The interface circuitries 210 and 220 illustratively comprise transceivers or other communication hardware or firmware that allows the associated system elements to communicate with one another in the manner described herein.

[0048] It is apparent from FIG. 2 that user equipment 202 and plurality of entities 204 are configured for communication with each other as security management participants via their respective interface circuitries 210 and 220. This communication involves each participant sending data to and / or receiving data from one or more of the other participants. The term “data” as used herein is intended to be construed broadly, so as to encompass any type of information that may be sent between participants including, but not limited to, identity data, key pairs, key indicators, access tokens, secrets, security management messages, registration request / response messages and data, request / response messages, authentication request / response messages and data, metadata, control data, audio, video, multimedia, consent data, analytics results, other messages, etc.

[0049] It is to be appreciated that the particular arrangement of components shown in FIG. 2 is an example only, and numerous alternative configurations may be used in other embodiments. For example, any given network element / function and / or access point can be configured to incorporate additional or alternative components and to support other communication protocols.

[0050] Other system elements such as access point 104, SMF 110, and UPF 112 may each be configured to include components such as a processor, memory and network interface. Also, entities such as third-party applications and network operators can participate in methodologies described herein via computing devices configured to include components such as a processor, memory and network interface. These elements and devices need not be implemented on separate stand-alone processing platforms, but could instead, for example, represent different functional portions of a single common processing platform.

[0051] More generally, FIG. 2 can be considered to represent processing devices configured to provide respective security management functionalities and operatively coupled to one another in a communication system. By way of example only, all or parts of each of UE 202 and the plurality of entities 204 (e.g., processor and memory) can be considered examples of means for performing one or more operations, one or more steps, one or more functions, one or more processes, etc. as described herein.

[0052] Given the above general description of some features of a communication network environment, problems with existing security approaches in ML model management, particularly in federated learning (FL) environments, and solutions proposed in accordance with illustrative embodiments, will now be described herein below.

[0053] In an FL setup, multiple distributed FL clients (e.g., mobile devices, edge nodes) collaboratively train a global ML model without sharing their raw data. Each FL client contributes model updates based on their local data, which an FL server aggregates to form the global model. This approach maintains data privacy while leveraging collective learning.

[0054] However, due to data privacy regulations, e.g., General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), FL clients must have the right to request the removal (unlearning) of their data from the global model. Machine unlearning ensures that a client’s data and its influence are erased from the model, as if it were never part of the training process. Performing and then verifying this unlearning process is challenging in FL due to the decentralized nature of the system and the aggregation of FL client updates.

[0055] Currently, when a FL client participating in an FL setup requests the removal of their data from the global model, there exists no mechanism to facilitate it, other than completely discarding the model itself. However, it is realized herein that since the ML model has already been trained from multiple different FL client data, in order to preserve the model and also to enable sustainable Al, only the data of the FL client which wants to remove its data should be taken out of the global trained model.

[0056] Accordingly, it would be desirable and advantageous for an unlearning process to be configured to provide functionality to:(i) Eliminate the client’s data contributions (e.g., gradients / parameters) from the global model; and(ii) Ensure that the global model behaves as if the client’s data was never used in training.

[0057] Further, when the data has been removed from the ML model, the FL client should also be given the provision to audit if the data is indeed removed or not. If the verification is not done, there are possibilities that malicious or compromised servers could claim to have removed the data but secretly retained it for model optimization or monetization purposes. This undermines trust in FL systems.

[0058] Illustrative embodiments overcome the above and other technical drawbacks of existing unlearning processes by providing the above and other functionalities of efficient and verifiable machine unlearning.

[0059] More particularly, in some illustrative embodiments, machine unlearning in an FL setup involves removing an FL client’s contributions from the global model, ensuring that the FL client’s data no longer influences the Al model’s output. In a first illustrative embodiment, the FL client stores its local gradients from past training rounds and shares them with the FL server during the unlearning request. The FL server uses these stored gradients to reverse their effect on the global model, effectively removing the FL client’s influence. This approach provides better privacy control for the FL client, as the FL server does not store client-specific gradients. In a second illustrative embodiment, the FL server itself stores the local gradients contributed by each FL client during training. Such an approach is particularly useful for server-initiated unlearning scenarios, such as detecting and removing the effects of a malicious FL client. The FL server uses the stored gradients of the identified FL client to neutralize its impact on the global model.

[0060] Further, verifying the success of machine unlearning is critical, as malicious or compromised FL servers might falsely claim unlearning while retaining residual data influence, leading to privacy breaches or inference attacks. To address this, in accordance with illustrative embodiments, a robust verification mechanism using a membership inference score (MIS) approach is provided. Given an input sample, MIS provides a probability value of whether it belongs the training set or not. MIS is obtained by training a binary classifier on the FL client’s side using samples belonging to FL client’s data and public test data. By comparing MIS values of data samples that need to be forgotten before and after unlearning, FL clients can confirm whether their data has been successfully removed.

[0061] Moreover, with the MIS approach, FL clients use the local models they already have and train a binary classifier using the prediction values of the local model (e.g., a vector of probabilities) on the training and test data of the client. Given an input, the binary classifier provides whether it belongs to the training set (in) or not (out) with a probability value. Each FL client joining the FL system trains their binary classifier locally and that remains private from the FL server as well as other FL clients. The MIS refers to this probability value, which ranges from zero to 100%.

[0062] For example, in some illustrative embodiments, two metrics are calculated to show whether there is a successful machine unlearning procedure or not. The first is a forgetting rate. Given a set of D samples that need to be forgotten, the forgetting rate is calculated as: (AF - BF') / \D\. AF and BF denote the number of samples predicted as out after and before machine unlearning, respectively.| D| refers to the total number of samples that need to be forgotten. In some illustrative embodiments, a machine learning method should achieve at least AF > BF,- otherwise, the unlearning is reversed or failed.

[0063] The second metric is the change in MIS. This metric also assumes that AF > BF has been satisfied. Before machine unlearning, the average MIS should be at least higher than 50%. After machine unlearning, the average MIS after unlearning should be less than 50% and closer to zero in the ideal setting (perfect forgetting). The higher the MIS difference, the more effective the machine unlearning procedure is.

[0064] The range of forgetting rate is: 0 < (AF - BF) / \D\ < 1.0 where AF > BF must be satisfied to verify a meaningful machine unlearning. The range of change in average MIS is: 0 < 1 / I^KSs eo MIS(s)before- MIS(s)after) < 1.0, where again AF > BF as well as ^ls EDMIS(s()before> 0.5 and MIS(s)after< 0.5 must be satisfied.

[0065] Based upon the particular deployment scenario, multiple different illustrative embodiments are described below, considering both UE and core network implementations. However, machine unlearning and auditing as described herein can be applicable in various additional or alternative ML model embodiments.

[0066] FIGS. 3A through 3C illustrate respective radio access network-based procedures for machine unlearning in a communication network environment according to an illustrative embodiment.

[0067] More particularly, in accordance with an illustrative embodiment, FIG. 3A illustrates a procedure 300 for machine unlearning in a radio access network (RAN) based solution where a UE stores per iteration data, e.g., its local gradients and / orotherdatafrom past training rounds. As shown, procedure 300 involves an FL server (deployed in a RAN) 302 and a plurality of FL clients (deployed in UEs) 304-1, 304-2, 304-3 and 304-4 (e.g., collectively referred to herein as FL clients 304, or individually as FL client 304 or, respectively, as UE1, UE2, UE3, and UE4 or as UE FL client 1, UE FL client 2, UE FL client 3, and UE FL client 4). While procedure 300 depicts four FL clients, it is to be understood that additional or alternative embodiments are not so limited and can include more or less FL clients.

[0068] Step 1 a: The FL server 302 in the RAN selects a group of clients (e.g., UE1 to UE4) and initiates the FL training process. These clients form the FL group for this training cycle.

[0069] Steplb: During initialization, FL clients 304 share their capability to support unlearning (e.g., by storing local gradients and / or relevant data for unlearning requests) with a new or existing radio resource control (RRC) message.

[0070] Step 1c: FL server 302 confirms to participate in the unlearning process if requested.

[0071] Steps 1 d1 -1 d3: The FL training happens between the FL clients 304 and the FL server 302. The FL clients 304 share the local model trained with the FL server 302 and the FL server 302 aggregates the updates. This continues for some iterations and once the FL cycle is completed, and the updated global model is shared with all participating FL clients 304 (UE1 to UE4).

[0072] Step 2: A specific FL client (e.g., UE1) requests unlearning, asking the FL server 302 to remove its contribution from the global model. For example, here UE1 shares its locally stored intermediate results such as weights or gradients, to facilitate unlearning. This is done with a new or existing RRC message.

[0073] Step 3a: The FL server 302 removes UE1’s contributions from the aggregated global model, such as adjusting the weights and bias values to neutralize the client’s effect. This step ensures the model no longer reflects the influence of the unlearned client.

[0074] In an FL process, the global model (Wg,bg) is usually updated using a weighted average of the FL client models. For example:nwg=aiwii=lnbg =aibii=lwhere:• Wi,bi: Weights and biases from FL client i• ai: Weighting factor for each FL client (e.g., based on data size or trust level).

[0075] For UE1, the contribution to the global model is:AWUE1= Cf-UEl^UElAbyg! = Ctu£-1b(y£-1aUE1is the weightage given to UE1, with one implementation based on data size:_ D citcis iz6 u IauE1~ / ^=1Datasizei

[0076] To neutralize UE1’s effect, the FL server 302 subtracts UE1’s contribution from the global model:= Wg- AW„£1bg= bg ~ bUE1

[0077] Step 3b: The FL server 302 confirms the completion of the unlearning process to the requesting FL client (e.g., UE1).

[0078] Steps 4a and 4b: After removing UE1’s contributions, the FL server 302 re-normalizes the remaining contributions to ensure the global model remains valid:n-lWg, =aiWii*UEln-lbgi ~i*UEl

[0079] The FL server 302 restarts the FL process using the unlearned model. The remaining FL clients (e.g., UE2, UE3, UE4) participate in this retraining to ensure the model’s performance is not adversely affected.

[0080] Step 5: The FL server 302 initiates a performance test by asking the remaining FL clients (e.g., UE2, UE3, UE4) to evaluate the updated model using their local data. This ensures that the model performs well despite the removal of UE1’s contributions.

[0081] Step 6: the remaining FL clients (e.g., UE2, UE3, UE4) send back the inference results (e.g., performance measures such as accuracy, precision, recall, F1 -score, etc.) from their local data for the FL server 302 to take the decision to proceed with further rounds.

[0082] Step 7: If the FL server 302 has access to global validation data, it can directly evaluate the updated model’s performance instead of relying solely on client feedback. The FL server 302 can pass the validation data to the global model and check the metrics such as accuracy, precision, recall, F1 -score, etc.

[0083] Step 8: Steps 4 to 7 are repeated as necessary until the global model achieves acceptable performance levels without the influence of the unlearned client. Acceptable performance level can already be agreed upon as part of the policy. This may depend on the application and domain. For example, (i) for critical systems (e.g., medical diagnosis): Accuracy > 95%, Precision / Recall > 90%; and (ii) for general use (e.g., image classification): Accuracy > 85%, F1 -score > 80%.

[0084] Turning now to FIG. 3B, in accordance with another illustrative embodiment, a procedure 310 is depicted for machine unlearning in a RAN based solution which is a variation of procedure 300 in FIG. 3A. While, in procedure 300, FL clients 304 store their local gradients from past training rounds, in procedure 310, the FL server stores such per iteration data. More particularly, in procedure 310, itis assumed that one or more FL clients 304 are not capable of storing local gradients for future unlearning and the FL server 302 agrees on storing the intermediate client gradients in every iteration. The FL server 302 uses this stored data for any unlearning request from FL clients 304.

[0085] Step 1 a: The FL server 302 in the RAN selects a group of clients (e.g., UE1 to UE4) and initiates the FL training process. These clients form the FL group for this training cycle.

[0086] Steplb: During initialization, FL clients 304 share their capability to support unlearning (e.g., without storing their own local gradients and / or relevant data associated with unlearning requests) with a new or existing radio resource control (RRC) message.

[0087] Step 1c: FL server 302 confirms to the FL clients 304 that it can store their local gradients and / or other relevant data.

[0088] Step 1d: The FL training process iterates and then completes, and the model is available with the FL server 302 and all the FL clients 304. The FL server 302 stores the per iteration gradients for each FL client 304.

[0089] Step 2: A specific FL client (e.g., UE1) requests unlearning, asking the FL server 302 to remove its contribution from the global model. This is done with a new or existing RRC message.

[0090] Step 3: The FL server 302 removes UE1’s contributions from the aggregated global model, such as adjusting the weights and bias values (which the FL server 302 has stored locally) to neutralize the client’s effect, as illustratively described above in procedure 300 (step 3a). This step ensures the model no longer reflects the influence of the unlearned client.

[0091] Step 4: The FL server 302 restarts the FL process using the unlearned model. The remaining FL clients 304 (e.g., UE2, UE3, UE4) participate in this retraining to ensure the model’s performance is not adversely affected.

[0092] Step 5: In an RRC message, the FL server 302 sends test data to the remaining FL clients 304 (e.g., UE2, UE3, UE4) to get the inference from its local data.

[0093] Step 6: the remaining FL clients (e.g., UE2, UE3, UE4) send back the inference results to the FL server 302.

[0094] Step 7: Steps 4 to 6 are repeated as necessary until the global model achieves acceptable performance levels without the influence of the unlearned client.

[0095] Turning now to FIG. 3C, in accordance with yet another illustrative embodiment, a procedure 320 is depicted for machine unlearning in a RAN based solution. Procedure 320 is a variation of procedure 310 of FIG. 3B. More particularly, procedure 320 can be used as a mitigation step against any poisonous or malicious ones of FL clients 304. For example, when the FL server 302 identifiesone of FL clients 304 as malicious (e.g., in an aggregation procedure as part of training), the FL server 302 can trigger the unlearning by removing the malicious client’s model updates from the global model.

[0096] Step 1 a: The FL server 302 in the RAN selects a group of clients (e.g., UE1 to UE4) and initiates the FL training process. These clients form the FL group for this training cycle.

[0097] Steplb: The FL process is ongoing. The FL server 302 has an anomaly detection algorithm running which checks for any anomaly in the updates from FL clients 304. It is assumed (e.g., similar to procedure 310) that the FL server 302 stores the per iteration data (e.g. local gradients, etc.) of each FL client 304.

[0098] Step 2: During training and aggregation step, the FL server 302 identifies any potential malicious FL client 304 (e.g., UE 1) and triggers machine unlearning forthat particular FL client 304.

[0099] Step 3: The FL server 302 removes UE1’s contributions from the aggregated global model, such as adjusting the weights and bias values (which the FL server 302 has stored locally) to neutralize the client’s effect. This step ensures the model no longer reflects the influence of the unlearned client.

[0100] Step 4: The FL server 302 continues the FL process with the remaining FL clients 304 (e.g., UE2, UE3, UE4) using the unlearned model.

[0101] FIGS. 4A and 4B illustrate respective core network-based procedures for machine unlearning in a communication network environment according to an illustrative embodiment.

[0102] More particularly, in accordance with an illustrative embodiment, FIG. 4A illustrates a procedure 400 for machine unlearning in a core network based solution where a native Al (NAI) FL clients store per iteration data, e.g., its local gradients and / or other data from past training rounds. As shown, procedure 400 involves an FL server (deployed in a core network as an NWDAF) 402, an AMF 404, a RAN (node) 406, and a plurality of NAI FL clients 408-1, 408-2, 408-3 and 408-4 (e.g., collectively referred to herein as FL clients 408, or individually as FL client 408, or as NAI FL client 1, NAI FL client 2, NAI FL client 3, and NAI FL client 4). While procedure 400 depicts four NAI FL clients, it is to be understood that additional or alternative embodiments are not so limited and can include more or less NAI FL clients. Also, in some illustrative embodiments, one or more of and NAI FL client 408 can be UEs (UE1, UE2, UE3, UE4).

[0103] Procedure 400 is similar to procedure 300 of FIG. 3A with the exception that, in the illustrative embodiment of FIG. 4A, the FL server 402 is an NWDAF in a core network which communicates with the NAI FL clients 408 through the AMF 404 and RAN 406.

[0104] Step 1a: The FL server 402 in the NWDAF selects a group of FL clients (e.g., UE1 to UE 4) and initiates the FL training process. These clients form the FL group for this training cycle.

[0105] Steplb: During initialization, FL clients 408 share their capability to support unlearning (e.g., by storing local gradients and / or relevant data for unlearning requests).

[0106] Step 1c: FL server 402 confirms to participate in the unlearning process if requested.

[0107] Steps 1d: The FL training process iterates and then completes, and the model is available with the FL server 402 and all the NAI FL clients 408.

[0108] Step 2a: A specific FL client (e.g., UE1) requests unlearning, asking the FL server 402 to remove its contribution from the global model. For example, here UE1 shares its locally stored intermediate results such as weights or gradients, bias values, etc. to facilitate unlearning.

[0109] Step 2b: The FL server 402 removes UE1’s contributions from the aggregated global model, such as adjusting the weights and bias values to neutralize the client’s effect. This step ensures the model no longer reflects the influence of the unlearned client.

[0110] Step 3: After removing UE1’s contributions, the FL server 402 initiates the FL process using the unlearned model with the remaining NAI FL clients 408 (e.g., UE2, UE3, UE4).

[0111] Step 4: The FL server 402 sends test data to the remaining NAI FL clients 408 (e.g., UE2, UE3, UE4) to get the inference from its local data.

[0112] Step 5: The remaining NAI FL clients 408 (e.g., UE2, UE3, UE4) send back the inference results to the FL server 402.

[0113] Step 6: Steps 3 to 5 are repeated as necessary until the global model achieves acceptable performance levels without the influence of the unlearned client.

[0114] Turning now to FIG. 4B, in accordance with another illustrative embodiment, a procedure 410 is depicted for machine unlearning in a core network based solution which is a variation of procedure 400 in FIG. 4A. While, in procedure 400, NAI FL clients 408 store their local gradients from past training rounds, in procedure 410, the FL server 402 stores such per iteration data. More particularly, in procedure 410, it is assumed that one or more FL clients 408 are not capable of storing local gradients for future unlearning and the FL server 402 agrees on storing the intermediate client gradients in every iteration. The FL server 402 uses this stored data for any unlearning request from FL clients 408.

[0115] Step 1a: The FL server 402 in the RAN selects a group of clients (e.g., UE1 to UE4) and initiates the FL training process. These clients form the FL group for this training cycle.

[0116] Steplb: During initialization, NAI FL clients 408 share their capability to support unlearning (e.g., without storing their own local gradients and / or relevant data associated with unlearning requests).

[0117] Step 1 c: FL server 402 confirms to the NAI FL clients 408 that it can store their local gradients and / or other relevant data.

[0118] Step 1d: The FL training process iterates and then completes, and the model is available with the FL server 402 and all the NAI FL clients 408. The FL server 402 stores the per iteration gradients for each NAI FL client 408.

[0119] Step 2a: A specific FL client (e.g., UE1) requests unlearning, asking the FL server 402 to remove its contribution from the global model. For example, here UE1 shares its locally stored intermediate results such as weights or gradients, bias values, etc. to facilitate unlearning.

[0120] Step 2b: The FL server 402 removes UE1’s contributions from the aggregated global model, such as adjusting the weights and bias values to neutralize the client’s effect. This step ensures the model no longer reflects the influence of the unlearned client.

[0121] Step 3: After removing UE1’s contributions, the FL server 402 initiates the FL process using the unlearned model with the remaining NAI FL clients 408 (e.g., UE2, UE3, UE4).

[0122] Step 4: The FL server 402 sends test data to the remaining NAI FL clients 408 (e.g., UE2, UE3, UE4) to get the inference from its local data.

[0123] Step 5: The remaining NAI FL clients 408 (e.g., UE2, UE3, UE4) send back the inference results to the FL server 402.

[0124] Step 6: Steps 3 to 5 are repeated as necessary until the global model achieves acceptable performance levels without the influence of the unlearned client.

[0125] FIGS. 5A and 5B illustrate respective procedures for auditing machine unlearning in a communication network environment according to an illustrative embodiment.

[0126] More particularly, in accordance with an illustrative embodiment, FIG. 5A illustrates a procedure 500 for machine unlearning auditing where FL servers can be based in a RAN or a core network. As shown, procedure 500 involves a plurality of FL servers (deployed in either the RAN or the core network) 502-1 and 502-2 and a plurality of FL clients 504-1, 504-2, 504-3 and 504-4 (e.g., collectively referred to herein as FL clients 504, or individually as FL client 504, or as UE FL client 1, UE FL client 2, UE FL client 3, and UE FL client 4). While procedure 500 depicts four FL clients, it is to be understood that additional or alternative embodiments are not so limited and can include more or less FL clients. Also, in some illustrative embodiments, one or more of the FL client 504 can be UEs (UE1. UE2, UE3, UE4).

[0127] Step 1. Agreement is achieved between one of the FL servers, e.g., FL server 502-1, and the FL clients 504 to participate in an FL process.

[0128] Steps 2a and 2b. Assume one of the FL clients 504, e.g., FL client 504-1 (UE1), seeks to receive a verification that its previous data contributions have been removed (upon request) from an ML model trained during the FL process. Thus, in steps 2a and 2b, the FL client 504-1 and the FL server 502-1 negotiate an unlearning auditing policy with a specific verification technique.

[0129] For example, as described above, the auditing / verification technique can be a membership inference score (MIS) mechanism which is agreed upon in steps 2a and 2b between the FL server 502-1 and the FL client 504-1. Recall that, given an input sample, MIS provides a probability value of whether the input sample belongs to the training set or not. MIS is obtained by training a binary classifier on the FL client’s side using samples belonging to FL client’s data and public test data. By comparing MIS values of data samples that need to be forgotten before and after unlearning, FL clients can confirm whether their data has been successfully removed. An MIS probability threshold (e.g., as described above) can also be negotiated between the FL server 502-1 and the FL client 504-1.

[0130] Step 3. The FL process is completed.

[0131] Step 4. The FL client 504-1 requests that its previous data contributions to the FL process be unlearned.

[0132] Step 5. The FL client 504-1 computes the MIS scores on the local ML model prior to machine unlearning (pre MU model).

[0133] Step 6. Machine unlearning on the trained ML model is performed to remove the data contribution of the FL client 504-1 as described above in one or more of the illustrative embodiments of FIGS. 3A-3C and FIGS. 4A and 4B.

[0134] Step 7. The FL client 504-1 requests the MIS score on the ML model after machine unlearning (post MU model) from the FL server 502-1.

[0135] Step 8. The FL server 502-1 computes the MIS score for the post MU model.

[0136] Step 9. The FL server 502-1 sends the MIS score of the post MU model to the FL client 504-1.

[0137] Step 10. The FL client 504-1 compares the MIS score of the pre MU model (that it computed in step 5) with the MIS score of the post MU model and, based on the agreed upon MIS probability threshold, determines whether the unlearning was successful or not.

[0138] Accordingly, in the above-described procedure 500 (which can be implemented with the above and any other unlearning procedures), the FL client negotiates on the unlearning audit policy including what verification techniques it is using, the MIS mechanism and the threshold. Before the FL client sends an unlearning request to server, it computes the MIS score on the trained global / localmodel. For example, the FL client can train a binary classifier on their side with their dataset (labeled as “in”) and some public test set (labeled as out). They train this classifier using the confidence score of the actual model (global or local). A binary classifier should give high membership probability values on “in” samples, e.g., ideally 1.0. After unlearning, clients download the unlearned global model. They get new confidence scores using the unlearned global model, pass it to the binary classifier and then the predictions with the binary classification membership probability. MIS is the average of per-sample membership inference probability before and after machine unlearning is implemented. With perfect unlearning, this should be all predicted as “out,” and the probability for “in” class is zero.

[0139] In accordance with another illustrative embodiment, FIG. 5B illustrates a procedure 510 for machine unlearning auditing where FL servers can be based in a RAN or a core network.

[0140] More particularly, in procedure 510 (as compared to procedure 500 of FIG. 5A), the FL client or the network operator can request to an NF or OAM to start the auditing. For example, this may be the case where a UE acting as an FL client does not have the capability / capacity to initiate and process the auditing mechanism, then the NF / OAM can trigger the steps with the respective FL server (present either in the core network or the RAN).

[0141] As shown, procedure 510 involves the plurality of FL servers 502-1 and 502-2 and the plurality of FL clients 504-1, 504-2, 504-3 and 504-4, as in procedure 500, and additionally, a gNB (RAN node) 506 and an OAM (node) or an NF in the core depicted as OAM / NF 508. While procedure 500 depicts four FL clients, it is to be understood that additional or alternative embodiments are not so limited and can include more or less FL clients. Also, in some illustrative embodiments, one or more of the FL client 504 can be UEs (UE1, UE2, UE3, UE4).

[0142] Step 1. Agreement is achieved between one of the FL servers, e.g., FL server 502-1, and the FL clients 504 to participate in an FL process.

[0143] Steps 2a, 2b, and 2c. Assume one of the FL clients 504, e.g., FL client 504-1 (UE1), seeks to receive a verification that its previous data contributions have been removed (upon request) from an ML model trained during the FL process. Thus, in step 2a, the FL client 504-1 sends an unlearning auditing request with an indicator (ID) of the corresponding FL process to the OAM / NF 508 (via gNB 506). The OAM / NF 508, in steps 2b and 2c, then negotiates an unlearning auditing policy with a specific verification technique, e.g., an MIS mechanism as described above, with the FL server 502-1.

[0144] Step 3. The FL process is completed.

[0145] Step 4. The FL client 504-1 decides that its previous data contributions to the FL process should be unlearned.

[0146] Step 5. The FL client 504-1 sends a message to OAM / NF 508 (via gNB 506) with client test data and the FL process and FL server details to trigger the auditing process for unlearning.

[0147] Step 6. The OAM / NF 508 computes the MIS scores on the ML model of FL client 504-1 prior to machine unlearning (pre MU model).

[0148] Step 7. Machine unlearning on the trained ML model is performed to remove the data contribution of the FL client 504-1 as described above in one or more of the illustrative embodiments of FIGS. 3A-3C and FIGS. 4A and 4B.

[0149] Step 8. The OAM / NF 508 requests the MIS score on the ML model after machine unlearning (post MU model) from the FL server 502-1.

[0150] Step 9. The FL server 502-1 computes the MIS score for the post MU model.

[0151] Step 10. The FL server 502-1 sends the MIS score of the post MU model to the OAM / NF 508.

[0152] Step 11. The OAM / NF 508 compares the MIS score of the pre MU model (that it computed in step 6) with the MIS score of the post MU model and, based on the agreed upon MIS probability threshold, determines whether the unlearning was successful or not.

[0153] As used herein, it is to be understood that the term “communication network” in some embodiments can comprise two or more separate communication networks. Further, the particular processing operations and other system functionality described in conjunction with the diagrams described herein are presented by way of illustrative example only and should not be construed as limiting the scope of the disclosure in any way. Alternative embodiments can use other types of processing operations and messaging protocols. For example, the ordering of the steps may be varied in other embodiments, or certain steps may be performed at least in part concurrently with one another rather than serially. Also, one or more of the steps may be repeated periodically, or multiple instances of the methods can be performed in parallel with one another.

[0154] It should again be emphasized that the various embodiments described herein are presented by way of illustrative example only and should not be construed as limiting the scope of the claims. For example, alternative embodiments can utilize different communication system configurations, user equipment configurations, base station configurations, provisioning and usage processes, messaging protocols and message formats than those described above in the context of the illustrative embodiments. These and numerous other alternative embodiments within the scope of the appended claims will be readily apparent to those skilled in the art.

Claims

Claims:

1. A user equipment, comprising:at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the user equipment to perform operations, the operations comprising:in conjunction with participating as a federated learning (FL) client in a FL process to train a machine learning model, participating in a negotiation process with a FL server involved in the FL process to agree on a method for verifying removal of data contributions that the user equipment contributed to the training of the machine learning model.

2. The user equipment of claim 1, wherein the agreed upon verification method comprises a model scoring mechanism.

3. The user equipment of claim 2, wherein the operations further comprise:based on the model scoring mechanism:computing a first score before removal of the data contributions from the machine learning model;computing a second score after the removal of the data contributions from the machine learning model;comparing a difference between the first score and the second score to a threshold value; anddetermining whether the data contributions have been sufficiently removed based on the threshold value comparison.

4. The user equipment of claim 1, wherein the agreed upon verification method comprises a membership probability mechanism.

5. The user equipment of claim 4, wherein the operations further comprise:based on the membership probability mechanism:before requesting removal of the data contributions that the user equipment contributed, computing a first value on a local version of the machine learning model, the first23value indicative of a first probability that the data contributions contributed by the user equipment is part of the local version of the machine learning model;after requesting removal of the data contributions that the user equipment contributed, computing a second value on a version of the machine learning model received from the FL server purporting to have the data contributions that the user equipment contributed removed, the second value indicative of a second probability that the data contributions contributed by the user equipment is part of the received version of the machine learning model; and determining whether the data contributions have been sufficiently removed based on the first probability and the second probability.

6. The user equipment of claim 5, wherein the membership probability mechanism utilizes a binary classifier configured to determine a probability of the data contributions being in a membership class and a probability of the data contributions being out of a membership class.

7. A method performed by a user equipment, the method comprising:in conjunction with participating as a federated learning (FL) client in a FL process to train a machine learning model, participating in a negotiation process with a FL server involved in the FL process to agree on a method for verifying removal of data contributions that the user equipment contributed to the training of the machine learning model.

8. The method of claim 7, wherein the agreed upon verification method comprises a model scoring mechanism.

9. The method of claim 8, further comprising:based on the model scoring mechanism:computing a first score before the removal of the data contributions from the machine learning model;computing a second score after the removal of the data contributions from the machine learning model;comparing a difference between the first score and the second score to a threshold value; anddetermining whether the data contributions have been sufficiently removed based on the threshold value comparison.

10. The method of claim 7, wherein the agreed upon verification method comprises a membership probability mechanism.

11. The method of claim 10, further comprising:based on the membership probability mechanism:before requesting removal of the data the user equipment contributed, computing a first value on a local version of the machine learning model, the first value indicative of a first probability that the data contributed by the user equipment is part of the local version of the machine learning model;after requesting removal of the data the user equipment contributed, computing a second value on a version of the machine learning model received from the FL server purporting to have the data the user equipment contributed removed, the second value indicative of a second probability that the data contributed by the user equipment is part of the received version of the machine learning model; anddetermining whether the data has been sufficiently removed based on the first probability and the second probability.

12. The method of claim 11 , wherein the membership probability mechanism utilizes a binary classifier configured to determine a probability of the data contributions being in a membership class and a probability of the data contributions being out of a membership class.

13. A computer-readable medium storing instructions which, when executed by at least one processor of a user equipment, cause the user equipment to perform the method of any one of claims 7 to 12.

14. A computer program comprising instructions which, when the computer program is executed by a user equipment, cause the user equipment to perform the method of any one of claims 7 to 12.

15. An apparatus comprising:at least one processor; andat least one memory storing instructions of a node of a communication network, wherein the instructions, when executed by the at least one processor, cause the apparatus to perform operations, the operations comprising:in conjunction with a user equipment that is a federated learning (FL) client participating in a FL process to train a machine learning model, participating on behalf of the user equipment in a negotiation process with a FL server involved in the FL process to agree on a method for verifying removal of data contributions that the user equipment contributed to the training of the machine learning model.

16. The apparatus of claim 15, wherein the agreed upon verification method comprises a model scoring mechanism.

17. The apparatus of claim 16, wherein operations further comprise:based on the model scoring mechanism:computing a first score before removal of the data contributions from the machine learning model;computing a second score after the removal of the data contributions from the machine learning model;comparing a difference between the first score and the second score to a threshold value; anddetermining whether the data contributions have been sufficiently removed based on the threshold value comparison.

18. The apparatusof claim 15, wherein the agreed upon verification method comprises a membership probability mechanism.

19. The apparatus of claim 18, wherein the operations further comprise:based on the membership probability mechanism:before a request for removal of the data contributions that the user equipment contributed, computing a first value on a local version of the machine learning model, the first value indicative of a first probability that the data contributions contributed by the user equipment is part of the local version of the machine learning model;after requesting removal of the data contributions that the user equipment contributed, computing a second value on a version of the machine learning model received from the FL server purporting to have the data contributions that the user equipment contributed removed, the second value indicative of a second probability that the data contributions contributed by the user equipment is part of the received version of the machine learning model; and determining whether the data contributions have been sufficiently removed based on the first probability and the second probability.

20. The apparatus of claim 19, wherein the membership probability mechanism utilizes a binary classifier configured to determine a probability of the data contributions being in a membership class and a probability of the data contributions being out of a membership class.

21. The apparatus of claim 15, wherein the operations further comprise:receiving a message from the user equipment to trigger execution of the verification method.

22. The apparatus of claim 21, wherein the message includes one or more of test data from the user equipment, FL process information, and FL server information.

23. The apparatus of any one of claims 8 to 15, wherein the node comprises a network function of the communication network.

24. The apparatus of any one of claims 8 to 15, wherein the node comprises an operations, administration, and management (OAM) node of the communication network.

25. A method performed by a node of a communication network, the method comprising:in conjunction with a user equipment that is a federated learning (FL) client participating in a federated learning (FL) process to train a machine learning model, participating on behalf of the user equipment in a negotiation process with a FL server involved in the FL process to agree on a method for verifying removal of data contributions that the user equipment contributed to the training of the machine learning model.

26. The method of claim 25, wherein the agreed upon verification method comprises a model scoring mechanism.

27. The method of claim 26, further comprising:based on the model scoring mechanism:computing a first score before the removal of the data contributions from the machine learning model;computing a second score after the removal of the data contributions from the machine learning model;comparing a difference between the first score and the second score to a threshold value; anddetermining whether the data contributions have been sufficiently removed based on the threshold value comparison.

28. The method of claim 25, wherein the agreed upon verification method comprises a membership probability mechanism.

29. The method of claim 28, further comprising:based on the membership probability mechanism:before a request for removal of the data contributions that the user equipment contributed, computing a first value on a local version of the machine learning model, the first value indicative of a first probability that the data contributions contributed by the user equipment is part of the local version of the machine learning model;after requesting removal of the data contributions that the user equipment contributed, computing a second value on a version of the machine learning model received from the FL server purporting to have the data contributions that the user equipment contributed removed, the second value indicative of a second probability that the data contributions contributed by the user equipment is part of the received version of the machine learning model; and determining whether the data contributions have been sufficiently removed based on the first probability and the second probability.

30. The method of claim 29, wherein the membership probability mechanism utilizes a binary classifier configured to determine a probability of the data contributions being in a membership class and a probability of the data contributions being out of a membership class.

31. The method of claim 25, wherein the operations further comprise:receiving a message from the user equipment to trigger execution of the verification method.

32. The method of claim 31, wherein the message includes one or more of test data from the user equipment, FL process information, and FL server information.

33. The method of any one of claims 25 to 32, wherein node comprises a network function of the communication network.

34. The method of any one of claims 25 to 32, wherein the node comprises an operations, administration, and management (OAM) node of the communication network.

35. A computer-readable medium storing instructions which, when executed by at least one processor of an apparatus, cause the apparatus to perform the method of any one of claims 25 to 34.

36. A computer program comprising instructions which, when the computer program is executed by an apparatus, cause the apparatus to perform the method of any one of claims 25 to 34.

Images