Method and system for privacy protecting machine learning model

Abstract

Training a ML model for protecting privacy of data by: extracting features from items in a training dataset S106, assigning a privacy parameter to each feature S108, generating a noisy item by adding

Description

Field

[001] The present techniques generally relate to a method and system for training a privacy protecting machine learning, ML, model. In particular, the present techniques provide a method of keeping client data private, while still allowing for accurate machine learning models to be generated by training on client data. Background

[002] In the last decade, neural network, NN, models have been successfully utilized in Artificial Intelligence, Al, systems for various applications. One weakness of these models is that they tend to memorize the training data, leading to the potential privacy concerns. Various techniques address these concerns by intentionally perturbing the data and system characteristics. However, they can reduce the utility, which makes the data harder to use for machine learning, ML, tasks and can lead to a decrease in accuracy when performing ML tasks.

[003] The present applicant has identified the need for an improved technique for generating privacy protecting machine learning models. Summary

[004] In a first approach of the present techniques, there is provided a computer-implemented method, performed on a server, for training a privacy protecting machine learning, ML, model for protecting privacy of data sent from client devices to the server, the method comprising: obtaining a training dataset comprising a plurality of data items; obtaining a task-specific ML model for performing a specific task with respect to a data item and generating an output for the data item; obtaining a discriminative ML model for maximising differences between features of a data item and an output generated by the task-specific ML model for the data item; and training the privacy protecting ML model, using each data item, by: extracting features from the data item using the privacy protecting ML model; assigning, using the privacy protecting ML model, a privacy parameter to each extracted feature of the data item; generating, using the privacy protecting ML model, a corresponding noisy data item by adding noise to each extracted feature of the data item using the assigned privacy parameter of the extracted feature; inputting the data item and the noisy data item into the task-specific ML model, and generating a first output for the data item and a second output for the noisy data item from the task-specific ML model; determining, using the discriminative ML model, the features of the data item and the noisy data item that resulted in differences between the first and second outputs; and training the privacy protecting ML model, using the second output and the features determined by the discriminative ML model, to optimise the assigned privacy parameter for each extracted feature.

[005] Advantageously, the present techniques provide a trained privacy protecting ML model that can be used by client devices to preserve the privacy of their data before transmitting the data to a server for use in either federated learning or federated analytics. Federated learning and federated analytics work well with lots of user data (either raw data or model parameters trained using the raw data), but it is essential to preserve / protect the privacy of users’ data when it is being sent off-device to a server. The present techniques enable the user data to be safely anonymised, which prevents an unauthorised party from reconstructing the anonymised data (e.g. by using a machine learning model trained on the anonymised data). At the same time, noise is added to user data to anonymise the user data in such a way that the underlying structure and features that can be learned from the data are preserved, ensuring that ML models used for federated learning or federated analytics are utilising data that provide accurate results / outputs.

[006] The privacy protecting machine learning, ML, model is also referred to herein as a privacy mechanism or data privatisation model (DPM) f which comprises a neural network y with parameters 0y. The privacy protecting ML model is trained by a server for deployment on user / client devices. The privacy protecting ML model is trained using non-user data, i.e. data that is not obtained from client devices. This is because the model needs to be trained using data that is not anonymised, in order to learn how to anonymise the data. The privacy protecting ML model can be used to anonymise raw data on the client device before it is sent to the server. Similarly, the privacy protecting ML model can be used to anonymise raw data on the client device before it is used to train a local model (i.e. a model on the client device), where parameters of the trained local model are shared with the server (and possibly also other client devices).

[007] In order to train the privacy protecting ML model, two temporary models are used, which may be discarded after use: a task-specific ML model, and a discriminative ML model. The two temporary models are not sent to the client devices after training - i.e. only the privacy protecting ML model is sent to the client devices for use.

[008] The task-specific ML model is also referred to herein as a classification or task model (TM) ce with parameters 0C. In the case of federated learning, the task-specific model is not necessarily linked to the global and local models being trained by the server and client devices. Similarly, in the case of federated analytics, the task-specific model is not necessarily the same as the model used by the server. That is, the task-specific model is simply used to ensure that the privacy protecting ML model anonymises data in a way that still allows that data to be used to perform a task. The task-specific model may be a classifier. For example, the task-specific model may operate on images and classify objects or actions within the images. A goal of the training is to ensure that the task-specific model is able to accurately classify anonymised images as well as non-anonymised images.

[009] The discriminative ML model is also referred to herein as privacy protection model (PPM) de with parameters 0d. The discriminative ML model is used to ensure that the privacyprotecting ML model anonymizes data in such a way that there is a difference between the original, non-anonymised data and the anonymized data. That is, a goal of the discriminative ML model is to maximise differences between the original, non-anonymised data and the anonymized data.

[010] In this way, the task-specific and discriminative ML models work together to ensure that the privacy protecting ML model adds enough noise to the data to anonymise the data, and that the data is still meaningful / useful for use by the server in federated learning or analytics.

[011] The term “data item” is used herein to mean any type of data that may be stored or generated on client devices in any format. For example, the data item may be an image, a video, a music file, location data, geospatial data, search history, etc.

[012] The step of assigning a privacy parameter to each extracted feature of the data item may comprise: assigning a random privacy parameter to each extracted feature. That is, the training process may begin by assigning a random privacy parameter as a starting point, which may then be adjusted or fine-tuned during the training.

[013] Training the privacy protecting ML model to optimise the assigned privacy parameter may comprise: maximising a difference between each data item and the corresponding noisy data item; and minimising a difference between the first output and the second output generated by the task-specific ML model. In other words, the anonymisation of the data item is adjusted so that as much privacy as possible is added without comprising performance of a machine learning model that needs to make predictions using the data. In this sense, the discriminative and task-specific models compete to adapt the anonymisation performed by the privacy protecting model to each of their purposes. Therefore, the loss function used to train the privacy protecting model comprises a term that is aimed to increase the distance between a distribution of original (not private / anonymised) data items and private / noisy data items. The loss function also comprises a second term that is aimed to minimise an error in predictions made by the task-specific model. For example, when the task-specific model is a classification model that makes predictions using labelled data, this term of the loss function aims to minimise a distance between the ground truth label corresponding to the data item and the prediction made by the classification model. This adversarial approach ensures that the anonymisation maximises privacy while also maintaining accuracy.

[014] Training the privacy protecting ML model may comprise: training the task-specific ML model and the discriminative module ML model simultaneously, wherein the task-specific ML model and the discriminative ML model share a common backbone architecture. The common backbone architecture may be for extracting features from the obtained, original data items and the noisy data items. The training of the privacy protecting ML model comprises training the task-specific model and the discriminative model, such that these models are also altered each time the privacy parameter changes during the process to optimise the privacy parameter assigned to each extracted feature.

[015] Both the task-specific and the discriminative model have to extract features from the data to perform their respective functions. That is, the task-specific model has to analyse which features are present in a data item to classify the data item. The discriminative model has to analyse which features are present in a data item to quantify a distance between the data item and its corresponding noisy data item. Since the parameters of both models are used to train the privacy protecting ML model, it is advantageous to use the same backbone for both the task-specific and discriminative models.

[016] Each of the discriminative and task-specific models may further comprise (or take the form of) an adaptor module I adaptor head to perform their respective tasks. An adaptor may simply be one or more fully connected layer that is trained to perform a specific task, along with the backbone. In some cases, only the task-specific models may comprise (or take the form of) an adaptor module or adaptor model, where each adaptor module / model is for a specific task. The discriminative model is used to measure and maximise differences between the original, non-anonymised data and the anonymised data for each adaptor.

[017] Generating a corresponding noisy data item may comprise: generating a feature vector of the data item, each element of the feature vector corresponding to an extracted feature; and adding the assigned privacy parameter for each extracted feature to each corresponding element of the feature vector, to thereby generate a noisy feature vector.

[018] The step of inputting the data item and the noisy data item into the task-specific ML model may comprise inputting: the generated feature vector for the data item into the taskspecific ML model; and the generated noisy feature vector into the task-specific ML model. This feature vector for the data item is used as a ground truth and, as explained above, a goal of the training is to ensure the task-specific ML model produces an output for the noisy feature vector that is close to the ground truth.

[019] The method may further comprise: normalising the generated feature vectors and generated noisy feature vectors after the generating and prior to the inputting. Normalising may comprise determining the mean and standard deviation / variance of the noisy data items (i.e. of the feature vectors) and normalising the noisy data items to have mean ^ = 0 and standard deviation / variance a = 1.

[020] The method may further comprise: masking elements of the generated noisy feature vectors after the generating and prior to the inputting. That is, it may be useful to randomly mask elements of the noisy feature vectors to prevent overfitting. This ensures that the representations of privacy and task concepts cannot become overly correlated during training.

[021] Masking elements of the generated noisy feature vectors may comprise: obtaining a dropout rate coefficient which determines how many elements of the generated noisy feature vectors are to be masked; generating a mask vector comprising at least one masked element, wherein the number of masked elements is based on the obtained dropout rate coefficient, and at least one unmasked element; and applying the mask vector to each generated noisy feature vector. In this way, elements of the noisy feature vector to which the masked elements of the mask vector are applied are effectively masked (e.g. removed, set to zero or some other value), while elements of the noisy feature vector to which the unmasked elements of the mask vector are applied are unchanged.

[022] Applying the mask vector may comprise multiplying the generated noisy feature vector by the mask vector.

[023] Generating a mask vector may comprise: determining a masking value for each masked element of the mask vector by sampling a value from a Bernoulli Distribution. The probability of the Bernoulli Distribution returning a zero value is given by the dropout rate coefficient. When the Bernoulli Distribution returns a zero value, the corresponding element of the noisy feature vector may also be set to zero, meaning that no noise is added to the corresponding extracted feature. When the Bernoulli Distribution returns a value of 1, the corresponding element of the noisy feature vector remains as is. In other words, the output of the Bernoulli Distribution is multiplied with the current noisy feature vector to mask elements of the noisy feature vector.

[024] The step of generating a noisy data item by adding noise may comprise: sampling, for each extracted feature of the data item, a random noise value from a Laplacian Distribution; and adding the sampled random noise value to the extracted feature of the data item.

[025] Sampling a random noise value from a Laplacian Distribution may comprise: using the assigned privacy parameter for the extracted feature to determine a variance of the Laplacian Distribution.

[026] In a second approach of the present techniques, there is provided a server fortraining a privacy protecting machine learning, ML, model for protecting privacy of data sent from client devices to the server, the server comprising: at least one processor coupled to memory, for: obtaining a training dataset comprising a plurality of data items; obtaining a task-specific ML model for performing a specific task with respect to a data item and generating an output for the data item; obtaining a discriminative ML model for maximising differences between features of a data item and an output generated by the task-specific ML model for the data item; and training the privacy protecting ML model, using each data item, by: extracting features from the data item using the privacy protecting ML model; assigning, using the privacy protecting ML model, a privacy parameter to each extracted feature of the data item; generating, using the privacy protecting ML model, a corresponding noisy data item by adding noise to each extracted feature of the data item using the assigned privacy parameter of the extracted feature; inputting the data item and the noisy data item into the task-specific ML model, and generating a first output for the data item and a second output for the noisy data item from the task-specific ML model; determining, using the discriminative ML model, the features of the data item and the noisy data item that resulted in differences between the first and second outputs; and training the privacy protecting ML model, using the second output and the features determined by the discriminative ML model, to optimise the assigned privacy parameter for each extracted feature.

[027] The features described above with respect to the first approach apply equally to the second approach and therefore, for the sake of conciseness, are not repeated.

[028] The at least one processor of the server may be configured to: transmit the trained privacy protecting ML model to a plurality of client devices. That is, after the privacy protecting ML model is trained, it is sent to client devices for use in federated learning and / or federated analytics tasks. As noted above, only the privacy protecting ML model is sent to client devices - the task-specific and discriminative ML models remain on the server and may themselves be discarded.

[029] The server may comprise a communication module or communication means for transmitting the generated privacy protecting ML model to client devices.

[030] In a third approach of the present techniques, there is provided a computer-implemented method, performed on a client device, for protecting privacy of data sent from the client device to a server, the method comprising: retrieving a plurality of user data items stored on the client device; and anonymising the retrieved data items using a trained privacy protecting machine learning, ML, model that has been trained using any of the methods described herein to generate anonymised user data items.

[031] The method may be for federated analytics, FA, wherein data is sent by the client device to a server for federated analytics. The method may further comprise: sending the anonymised data items to a server.

[032] In this case, prior to retrieving a plurality of data items, the method may comprise: receiving, from the server, a request for the plurality of data items. Thus, the client device may only utilise the privacy protecting ML model when a request for anonymised data is received from the server.

[033] The method may be for federated learning, FL, wherein data is sent by the client device to a server for federated learning. The method may further comprise: training a local machine learning, ML, model using the anonymised data items, wherein the local ML model is a version of a global ML model of the server; and sending model parameters of the trained local ML model to the server.

[034] In this case, prior to retrieving a plurality of data items, the method may comprise: receiving, from a server, a request for model parameters of the local ML model that has been trained using the anonymised data items.

[035] In a fourth approach of the present techniques, there is provided a client device for protecting privacy of data sent from the client device to a server, the client device comprising: storage storing a plurality of user data items; a trained privacy protecting machine learning, ML, model that has been trained using any of the methods described herein; and at least one processor coupled to memory, for: retrieving some or all of the plurality of user data items from the storage; and anonymising the retrieved data items using the trained privacy protecting ML model to generate anonymised user data items.

[036] The features described above with respect to the third approach apply equally to the fourth approach and therefore, for the sake of conciseness, are not repeated.

[037] The client device comprises at least one processor and memory storing instructions that, when executed by the at least one processor individually or collectively, cause the client device to perform the steps described above.

[038] The client device comprises a communication module or communication means for communicating with the server.

[039] The client device may be a smart device. The client device may be a smartphone. A smartphone is an example of a smart device. The client device may be a smart appliance. A smart appliance is another example of a smart device. An example of a smart appliance is a smart television (TV), a smart fridge, a smart oven, a smart vacuum cleaner, a smart robotic device, a smart lawn mower, and so on. More generally, the client device may be a constrained-resource device, but which has the minimum hardware capabilities to personalise a trained ML model, and to use the personalised ML model. The client device may be any one of: a smartphone, tablet, laptop, computer or computing device, virtual assistant device, a vehicle, an autonomous vehicle, a robot or robotic device, a robotic assistant, image capture system or device, an augmented reality system or device, a virtual reality system or device, a gaming system, an Internet of Things device, or a smart consumer device (such as a smart fridge, smart vacuum cleaner, smart lawn mower, smart oven, etc). It will be understood that this is a non-exhaustive and non-limiting list of example devices.

[040] In a related approach of the present techniques, there is provided a computer-readable storage medium comprising instructions which, when executed by at least one processor, causes the processor to carry out any of the methods described herein.

[041] In the cases where the present techniques are implemented or executed on a device comprising multiple processors, the present techniques may be implemented by one or more of the multiple processors. That is, the present techniques may be implemented by or executed by the processors individually or collectively.

[042] As will be appreciated by one skilled in the art, the present techniques may be embodied as a system, method or computer program product. Accordingly, present techniques may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[043] Furthermore, the present techniques may take the form of a computer program product embodied in a computer readable medium having computer readable program code embodied thereon. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[044] Computer program code for carrying out operations of the present techniques may be written in any combination of one or more programming languages, including object oriented programming languages and conventional procedural programming languages. Code components may be embodied as procedures, methods or the like, and may comprise subcomponents which may take the form of instructions or sequences of instructions at any of the levels of abstraction, from the direct machine instructions of a native instruction set to high-level compiled or interpreted language constructs.

[045] Embodiments of the present techniques also provide a non-transitory data carrier carrying code which, when implemented on a processor, causes the processor to carry out any of the methods described herein.

[046] The techniques further provide processor control code to implement the abovedescribed methods, for example on a general purpose computer system or on a digital signal processor (DSP). The techniques also provide a carrier carrying processor control code to, when running, implement any of the above methods, in particular on a non-transitory data carrier. The code may be provided on a carrier such as a disk, a microprocessor, CD- or DVD-ROM, programmed memory such as non-volatile memory (e.g. Flash) or read-only memory (firmware), or on a data carrier such as an optical or electrical signal carrier. Code (and / or data) to implement embodiments of the techniques described herein may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as Python, C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language such as Verilog (RTM) or VHDL (Very high speed integrated circuit Hardware Description Language). As the skilled person will appreciate, such code and / or data may be distributed between a plurality of coupled components in communication with one another. The techniques may comprise a controller which includes a microprocessor, working memory and program memory coupled to one or more of the components of the system.

[047] It will also be clear to one of skill in the art that all or part of a logical method according to embodiments of the present techniques may suitably be embodied in a logic apparatus comprising logic elements to perform the steps of the above-described methods, and that such logic elements may comprise components such as logic gates in, for example a programmable logic array or application-specific integrated circuit. Such a logic arrangement may further be embodied in enabling elements for temporarily or permanently establishing logic structures in such an array or circuit using, for example, a virtual hardware descriptor language, which may be stored and transmitted using fixed or transmittable carrier media.

[048] In an embodiment, the present techniques may be realised in the form of a data carrier having functional data thereon, said functional data comprising functional computer data structures to, when loaded into a computer system or network and operated upon thereby, enable said computer system to perform all the steps of the above-described method.

[049] The method described above may be wholly or partly performed on an apparatus, i.e. an electronic device, using a machine learning or artificial intelligence model. The model may be processed by an artificial intelligence-dedicated processor designed in a hardware structure specified for artificial intelligence model processing. The artificial intelligence model may be obtained by training. Here, "obtained by training" means that a predefined operation rule or artificial intelligence model configured to perform a desired feature (or purpose) is obtained by training a basic artificial intelligence model with multiple pieces of training data by a training algorithm. The artificial intelligence model may include a plurality of neural network layers. Each of the plurality of neural network layers includes a plurality of weight values and performs neural network computation by computation between a result of computation by a previous layer and the plurality of weight values.

[050] As mentioned above, the present techniques may be implemented using an Al model. A function associated with Al may be performed through the non-volatile memory, the volatile memory, and the processor. The processor may include one or a plurality of processors. At this time, one or a plurality of processors may be a general purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and / or an Al-dedicated processor such as a neural processing unit (NPU). The one or a plurality of processors control the processing of the input data in accordance with a predefined operating rule or artificial intelligence (Al) model stored in the non-volatile memory and the volatile memory. The predefined operating rule or artificial intelligence model is provided through training or learning. Here, being provided through learning means that, by applying a learning algorithm to a plurality of learning data, a predefined operating rule or Al model of a desired characteristic is made. The learning may be performed in a device itself in which Al according to an embodiment is performed, and / o may be implemented through a separate server / system.

[051] The Al model may consist of a plurality of neural network layers. Each layer has a plurality of weight values, and performs a layer operation through calculation of a previous layer and an operation of a plurality of weights. Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GAN), and deep Q-networks.

[052] The learning algorithm is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Brief description of the drawings

[053] Implementations of the present techniques will now be described, by way of example only, with reference to the accompanying drawings, in which:

[054] Figure 1A is a schematic diagram showing a problem with the existing techniques;

[055] Figure 1B is a schematic diagram showing an advantage of the present techniques

[056] Figure 2A shows a privacy scenario where machine learning (ML) models are trained and deployed using private data;

[057] Figure 2B shows a framework for implementing privacy models according to the present techniques;

[058] Figure 2C shows further optional modules of the learnable privacy mechanism;

[059] Figures 3A and 3B are images illustrating targeted privacy protection;

[060] Figure 4 is a flowchart illustrating the present techniques;

[061] Figure 5 shows an algorithm for implementing an overall optimisation step for the framework described herein;

[062] Figure 6A shows results for data leakage risk and ML task model accuracy on private data;

[063] Figure 6B shows data leakage risk (Acc(A)) and the accuracy of TMs c_m (Acc) obtained under data leakage attacks;

[064] Figures 7A and 7B show experimental results for reconstruction attacks;

[065] Figure 8A shows how the privacy-utility trade-off changes during training;

[066] Figure 8B shows privacy and utility (i.e., task accuracy) trade-off for the data leakage risk attack;

[067] Figure 9A shows that using a targeted privacy budget (i.e., Target. Budget) significantly enhances the accuracy of the ML tasks;

[068] Figure 9B shows experimental results for evaluating the robustness of DPMs;

[069] Figure 10 shows examples from image classification datasets used in the present experiments;

[070] Figure 11A shows the samples distorted with noise for privacy protection;

[071] Figure 11B shows the average localized noise estimated by the present frameworks; and

[072] Figure 12 shows a system for implementing the present techniques. Detailed description of the drawings

[073] Broadly speaking, the present techniques generally relate to a method for training a privacy protecting machine learning, ML, model. Advantageously, the present techniques enable a machine learning, ML, model to learn how much noise to apply to a specific data item to ensure that the data item is sufficiently anonymised while still being an accurate representation of the original data. The ML model is trained to optimise for both data accuracy and privacy by using parameters of two other models that learn how to perform a task, and learn how different noisy and original data are.

[074] The present techniques provide a framework that estimates the optimal privacy from data to improve the utility for NN models in ML tasks. All models employed in the present framework are differentiable, allowing us to optimize the privacy-utility trade-off in an end-to-end model training scheme. The experimental results demonstrate that the present framework significantly improves the utility for image classification tasks while ensuring similar or stronger privacy protection against data reconstruction and data leakage attacks compared to the state-of-the-art baseline.

[075] Machine learning, ML, models that rely on large-scale datasets, such as neural network, NN, models, poses significant security risks. These models tend to memorize their training data which can lead to the exposure of sensitive information, including personally identifiable data, and compromise individual privacy. Machine learning, ML, models memorizing training data might open new attack surfaces, such as reconstruction of private training data by malicious users. Improving the privacy in training task-specific models (TMs) can minimize the information leakage, but also reduce their utility (i.e., task accuracy). Therefore, it is essential to carefully determine the scale of privacy by considering the trade-off between privacy and utility of TMs.

[076] Differential privacy has emerged as noise-based techniques to ensure privacy when handling sensitive information. The statistical information extracted from data are obscured by adding random noise, and the level of privacy is calibrated by privacy budget. However, obtaining an optimal privacy-utility trade-off can be challenging, as the privacy budget requires manual tuning. An alternative approach involves clipping the gradients of loss calculated on the training data. While these methods protect privacy and defend machine learning, ML, models, the clipping step can alter the gradients, potentially leading to a decrease in accuracy for further applications. Consequently, they do not guarantee an optimal solution for task specific ML models. To this end, the present techniques provide a framework to improve the privacy-utility trade-off using trainable parameters shared among privacy and task models.

[077] Figure 1A is a schematic diagram showing a problem with the existing techniques. Existing techniques do not optimize privacy and accuracy trade-off. Hence, ML models trained with private data generated using existing techniques obtain low accuracy results.

[078] Figure 1B is a schematic diagram showing an advantage of the present techniques. The present techniques (Learnable Privacy Model) optimizes privacy and accuracy trade-off. Hence, ML models trained with private data generated using the present privacy protecting ML model obtain both high accuracy and high privacy.

[079] Figure 2A shows a privacy scenario where machine learning (ML) models are trained and deployed using private data. In this scenario, an adversary participant attempts to access either a private dataset xe e of a trusted participant or an ML model ce trained on Si by the trusted participant to perform a ML task. It is assumed that an adversary participant has unauthorized access to either the private sample xe or the ML task model ce obtained from a trusted participant. The ideal private system must balance privacy and utility effectively. If privacy is inadequate, then the sensitive information about data or ML models can be easily compromised. Conversely, strong privacy can compromise the utility, leading to degraded performance. Inherently, there is a trade-off between privacy and utility, and it should be carefully balanced to obtain an optimal system.

[080] Consider a simple threat scenario. In this scenario, a trusted user trains a TM ce using data xe that is privatized by a model f with a privacy budget e and deploys the model ce on an application (or product) to perform an ML task (e.g. image classification). Later, a malicious user attempts to access the private samples xe and the TM ce by just prompting the application.

[081] In the literature, vanilla noise-based methods randomly obscure the statistics of data. Recent approaches apply the noise-based techniques in the optimization of model parameters, such as by clipping gradients and adding noise to the gradients. This strategy offers privacy protection for ML models by reducing the risk of memorization of data, at the cost of accuracy. Another approach utilizes latent spaces learned from data. These methods are used to provide protection against reconstruction attacks.

[082] Ideally, privacy models utilized in an ML framework must be sufficient enough to protect sensitive information, such as feature representations learned by the task-specific models (TMs) (e.g. image classifiers) and their training data. On the other hand, strong privacy can potentially harm the utility of TMs. This trade-off between privacy and utility must be tuned through privacy budget e by privacy models.

[083] Figure 2B shows a framework for implementing privacy models that learn privacy budget e using neural networks (NNs). The present techniques provide a framework for implementing privacy models that learn privacy budget e using neural networks (NNs). To improve the accuracy and privacy of Task-Specific Models (TMs), the present techniques integrate two models, named, Data Privatization Model (DPM) and Privacy Protection Model (PPM). In particular, learnable parameters are utilised that are shared among privacy models and taskspecific models through an NN-based backbone model as illustrated in Fig. 2B. Ultimately, this structure helps to learn shared latent features rather than exclusive ones from the original (non-private) and private data while minimizing a task loss. To this end, this establishes a relationship between privacy and utility through the shared parameters, and the chance of overfitting to a particular model (privacy or task-specific models) is reduced.

[084] Figure 2C shows further optional modules of the learnable privacy mechanism. These are described in more detail below.

[085] The present techniques formulate improvement of the privacy-utility trade-off by a minimax optimization problem and a novel objective function. The proposed objective function enhances the stability of training the present privacy models. Moreover, it forces the present privacy models to learn distinct privacy characteristics (e.g., the present models can estimate separate privacy budgets for separate data partitions using the same privacy mechanism) that are hard to distinguish. Experimental results demonstrate that the privacy characteristics learned by the present framework differ from that of the vanilla noise-based privacy models.

[086] A new threat model is presented, under the scenario of employment of a training and inference ML pipeline with private data. In this scenario, an adversary participant can access private datasets or ML models that utilize the private datasets to perform ML tasks, in an unauthorized manner to extract sensitive information from data and ML models.

[087] The present techniques propose a learnable privacy mechanism (LPM) that automatically estimates the optimal privacy budget from data, enabling effective privacy protection while preserving model utility. For this purpose, the differentiable Laplace mechanism described by Dwork et al. and Dinur et al is adopted, and an NN-based model is introduced to implement the mechanism in the LPM. The estimated privacy budget is then used to add noise to the data, sampled from the Laplace distribution. Notably, all models employed in the present framework are designed to be differentiable, allowing for end-to-end optimization of the parameters that control the trade-off between privacy and utility during model training.

[088] Figures 3A and 3B are images illustrating targeted privacy protection. Figure 3A shows a vanilla Laplace mechanism as described by Dwork et al. Figure 3B shows how the present framework can estimate separate privacy budgets for different data parts by learning features from data. The present framework can selectively balance the level of privacy for certain parts of the data. The present learnable mechanism has the capability to estimate distinct privacy budgets for different data partitions, allowing for selective and targeted privacy protection. For instance, different budgets can be estimated on separate image regions containing useful patterns (e.g. whiskers of cats) to discriminate objects (e.g. to categorize cats and birds) while preserving privacy as illustrated in Figure 3B.

[089] Figure 4 is a flowchart illustrating the present techniques. In particular, the present techniques comprise a computer-implemented method, performed on a server, for training a privacy protecting machine learning, ML, model for protecting privacy of data sent from client devices to the server, the method comprising: obtaining a training dataset (S‘) comprising a plurality of data items x (step S100); obtaining a task-specific ML model for performing a specific task with respect to a data item and generating an output for the data item (step S102); obtaining a discriminative ML model (discriminative head d) for determining and maximising differences between features of a data item and an output generated by the task-specific ML model for the data item (step S104); and training the privacy protecting ML model, using each data item, by: extracting features (xi) from the data item using the privacy protecting ML model (step S106); assigning, using the privacy protecting ML model, a privacy parameter (ei) to each extracted feature of the data item (step S108); generating, using the privacy protecting ML model, a corresponding noisy data item (xe) by adding noise to each extracted feature of the data item using the assigned privacy parameter of the extracted feature (step S110); inputting the data item and the noisy data item into the task-specific ML model, and generating a first output for the data item and a second output for the noisy data item from the taskspecific ML model (step S112); determining, using the discriminative ML model, the features of the data item and the noisy data item that resulted in differences between the first and second outputs (step S114); and training the privacy protecting ML model, using the second output and the features determined by the discriminative ML model, to optimise the assigned privacy parameter for each extracted feature (step S116).

[090] Obtaining a training dataset S100 may comprise a training dataset stored on the server. The training dataset may comprise labelled or unlabelled training data. For example, the training dataset may comprise images, audio files, text files and / or any other suitable training data items. It will be appreciated that the training dataset may be adapted to the type of task that a trained machine learning is ultimately aiming to solve. For example, when a trained ML model is trained to classify images, the training dataset may comprise labelled images.

[091] Obtaining a task-specific ML model (step S102) may comprise obtaining a classification or task model (TM) ce with parameters 0C. The task-specific model is simply used to ensure that the privacy protecting ML model anonymises data in a way that still allows that data to be used to perform a task.

[092] Obtaining a discriminative ML model (step S104) may comprise obtaining a privacy protection model (PPM) de with parameters 0d.

[093] Extracting features from the data items (step S106) may comprise extracting lines, shapes, edges, peaks, throughs, or any other features in the training data item that may be identified in images, audio data, and / or text data.

[094] The step (S108) of assigning a privacy parameter to each extracted feature of the data item may comprise: assigning a random privacy parameter to each extracted feature. That is, the training process may begin by assigning a random privacy parameter as a starting point, which may then be adjusted or fine-tuned during the training. Some features may be more important than other features. Less important features will have less impact when a ML model analyses the data item during inference time. Therefore, more noise can be added to less important features, whereas more important features should be preserved as much as possible without compromising privacy.

[095] The step (S110) of generating a corresponding noisy data item may comprise: generating a feature vector of the data item, each element of the feature vector corresponding to an extracted feature; and adding the assigned privacy parameter for each extracted feature to each corresponding element of the feature vector, to thereby generate a noisy feature vector.

[096] Noise is added based on the privacy parameters that are assigned to each feature. The privacy parameter e is related to the variance of a Laplacian distribution from which noise that is added to each feature is sampled. In particular, the privacy parameter e is inversely proportional to the variance of the Laplacian distribution. That is, the smaller e, the more noise is added to a feature. Feature may mean a single pixel, a group of pixels in any shape, a single audio feature, a group of audio features and / or an audio sequence of a certain length, a single word in a text, a sequence of letter and / or words in a text and / or one or more embeddings when text is analysed in embedding space.

[097] The data item and the noisy data item are input (step S112) into the task-specific ML model. As noted above, the task-specific model is trained during the training process. When labels for the training data items exist, the task specific model can be trained using supervised learning techniques. The model parameters that result from training the task-specific model are saved so they can later be used to train the privacy protecting ML model.

[098] The discriminative model is trained to determine a difference between noisy and the original (not noisy) data items. For example, the discriminative model may find a representation of each original and noisy data item in embedding space. From these representations, the discriminative model may learn how different original and noisy data items are. The discriminative model may output a difference between each original and noisy data item, or the discriminative model may determine a difference between a whole distribution of original and corresponding noisy data items.

[099] Step S116 comprises training the privacy protecting model to maximise privacy, i.e. a difference between original and noisy data items, while maintaining accuracy, i.e. a good result in the task as represented by the task-specific model.

[100] As noted below, experiments show that the present framework outperforms baselines in terms of protection against various privacy threats while achieving high utility on ML tasks. In other words, it significantly improves the utility of TMs while ensuring similar or stronger privacy protection compared to the state-of-the-art baselines.

[101] The present threat model operates under the scenario of employing a training and inference ML pipeline with private data. Here, a trusted participant trains an ML model on a given private training dataset to accomplish a specific task such as image classification. The trained ML model is then deployed for inference on private test data to predict corresponding labels. Meanwhile, an adversary participant attempts to obtain sensitive information, including the reconstruction of original data and its underlying characteristics.

[102] Trusted Participant - Knowledge and Capabilities: A dataset is privatized by a privacy mechanism f-.xeSt^x£e S£, and an ML model c£ is trained using samples x£ e S£ with their ground-truth labels y e y by a trusted participant to perform a task such as image classification. Note that private data or model is denoted by subscript e.Later, the trained ML model c£ is applied on a private test dataset T£ to categorize unseen samples given in inference time.

[103] Trusted Participant - Objective: There are two main objectives that the trusted participant must fulfil: 1) It must protect privacy against various threats by utilizing a privacy mechanism f. 2) The ML models must maintain high accuracy with a strong privacy protection. To this end, trusted participant must optimize the balance between privacy and utility in this pipeline.

[104] Trusted Users: Each trusted user t prepares a set of privatized samples xe e S£ by employing a privacy model on given samples x e with a privacy budget e. For instance, vanilla noise-based privacy models determine the scale of privacy by sampling an additive noise r|e from a distribution. The Laplace mechanism samples r|e from the Laplacian distribution Lap(jj. = 0,b =^) to obtain x£ = x + r|e. Later, each user t trains a task-specific model (TM) c£ using the privatized samples x£ e S£ with their ground-truth labels y e yf (e.g. class labels). The TM c£ = 0 is composed of a backbone model < / >e (e.g. a feature extractor) and a task submodel ip£ (e.g. a classifier).

[105] Malicious Users: A malicious user m has a separate set of samples x e Sm. The user m can use these samples to train separate TMs cm for data leakage attacks. The goal is to access the trusted user’s data and TMs c£ in an unauthorized manner, and perform privacy attacks on them to extract sensitive information. In particular, data reconstruction attacks aim to maximize the distribution p(x|h) in order to recover the sensitive information about a sample x e by accessing the latent representations h e of a TM ce.

[106] Adversary Participant - Knowledge and Capabilities: It is assumed that the adversary participant has unauthorized access to either the private data xe or the ML model ce of the trusted participant. Additionally, the adversary participant is assumed to possess the following capabilities: 1) It has a separate dataset Sa. 2) It can train separate ML models using its dataset Sa. 3) It has knowledge about the architecture of M L models for the tasks performed by the trusted participant. 4) When it accesses to the task model ce of a trusted participant, it can compute latent representations and / or the class labels of an input sample.

[107] Adversary Participant - Objective: An adversary participant has unauthorized access to the dataset xe e T* or ML model ce utilized by a trusted participant. It can perform the following attacks: • Data Reconstruction Attacks: The objective is to recover sensitive information about a sample x e by accessing the latent representations he e or outputs y of an ML model ce. Hence, the adversary participant uses different optimization techniques to gradually reconstruct input samples that can produce similar outputs of a ML model. • Data Leakage Attacks: The goal of the adversary participant is to analyse the content of the private dataset. It trains a separate ML model ca using its own data Sa. The ML model ca is then used to analyze the trusted participant’s dataxe e T / with the model’s outputs. The success of these threats depends on the privacy mechanism f. If privacy protection is inadequate, privacy leakage is likely to occur. On the other hand, if the privacy mechanism perturbs the data excessively, then the accuracy of the ML model ce can be negatively affected by the level of privacy.

[108] The present framework estimates the privacy budget while improving privacy and utility. For this purpose, a model f is employed that implements the present learnable privacy mechanism (LPM). The present framework learns how much of the data characteristics should be perturbed for privacy while improving its utility in ML tasks. It employs a neural network (NN) model f to learn this perturbation from a training dataset

[109] The objective of trusted users is to maximize the privacy protection while maintaining the accuracy of TMs. To determine the scale of privacy that can minimize the information leakage and the training loss, two models are integrated to the TMs: (1) Data Privatization Model (DPM): To determine the scale of privacy, the present techniques propose to estimate the privacy budget from data. For this purpose, the present techniques implement a trainable model f named DPM identified by an NN y with parameters 0y. The DPM / (x,r|e; 0y) first estimates the budget e by the learnable privacy budget e. It then generates t]e using the estimated e with a differentiable Laplacian mechanism, as described by Jankowiak and Obermeyer, and Jankowiak and Karaletsos. Finally, the generated noise t]e is added to the data sample x, i.e., f (x,r]e; 0y) = xe where xe = x + r]e. (2) Privacy Protection Model (PPM): Suppose that h and he denote the latent representations of data x and xe computed by a TM c£. The similarity of distributions p(he|xe) and p(h|x) impact the protection of TMs against data reconstruction attacks. That is, a larger difference between the distributions makes it more challenging to reconstruct the original data x from the latent space of TMs. To control the distance between the distributions p(h|x) and p(he|xe) of TMs, a privacy protection model (PPM) d£ = <p£° <p£ is utilised, where < / >e denotes a shared backbone and <p£ is a privacy submodel. The shared backbone < / >e is used to tune the latent representation of TMs with PPM, while the privacy submodel <p£ is used to discriminate the latent representations of original data x and privatized data xe.

[110] In the present framework, i) a PPM d£ is used for learning how to discriminate original data (non-private) x and private data xe, and ii) a TM c£ is used for learning how to make predictions on the private data xe related to ML tasks. When these models are trained separately, it learns exclusive representations that are either useful for discriminating xe from x (e.g., those learned by d^, or for discriminating samples among categories (e.g., those learned by ce) in ML tasks. Inevitably, this architecture leads to misalignment and overfitting, which, in turn, degrades the performance of privacy and utility of TMs ce.

[111] For instance, representations learned by d£ can be useful to estimate how much privacy is needed to perturb the original data x by shifting its distribution away from that of the private data xe. However, this perturbation can remove the task-specific information from the representations leading to a degradation of accuracy of c£. Conversely, the model c£ aims to learn how to categorize the privatized data samples. When these models are trained separately, the ML model ce may overfit to the private dataset by just learning the representations of the private data.

[112] To avoid such problems, a shared backbone < / >e is used (e.g., a ResNet model, such as the one described by He et al.) while designing both de and ce models. The model de employs a privacy submodel <pe to calculate the probability scores o, which are used to discriminate private data (p(o = 0|xe)) and original data (p(o = l|x)). The model ce employs a task submodel \p£ (e.g., a classifier) to calculate task-specific predictions (e.g. classes of samples) y using the shared features obtained from <p£.

[113] A vanilla Laplacian mechanism utilizes a fixed privacy budget e that must be manually tuned. In the present framework, e is estimated using an NN model. More precisely, the NN model y projects a sample x from a data space X onto a private latent space 8 by y. x e X e e 8 where the Ith variable et of the e denotes a trainable parameter representing the privacy budget e used to add noise to the Ith variable xt of the data x. The estimated parameters e are later utilized in the differentiable Laplace mechanism by / (X,Tie; 0 / ) = X + T]e, where 0y is the set of network parameters of the model f that will be optimized during training and the Ith variable of rie is a Lap(p = 0,b = l / e^ random variable, VL Explicitly, rie denotes the noise sampled from the Laplace mechanism with the estimated privacy budget e. Note that the dimension of the estimated privacy budget e can vary, allowing independent local adjustments for the privacy protection of data. At the end, private data xe is computed by the model f.

[114] While designing the models, two critical constraints are considered: 1) The estimated privacy budgets should be generic and not overfit to the training data. This ensures that the privacy budgets are not biased towards the specific data. 2) Since the model is designed to learn the amount of privacy protection to be applied from the training data, there is a risk that it may memorize the noise characteristics of the most frequently observed concepts in ML tasks. Under these constraints, two additional modules q and p are integrated with non-linear transformations for the model y to design the model f implementing the LPM. Figure 2C illustrates the overall architecture of the model f, including a stochastic dropout module.

[115] Normalization Module. A normalization module q is introduced after each linear transformation. It enables us to learn stable feature distributions, ensuring that the feature space is generalized during training. Formally, a feature vector h is normalized with mean nh and oh, estimated on the feature space Jf. In the experiments, group normalization is employed, as described by Wu et al. by dividing the activations into 16 groups. It groups data patterns based on their underlying latent data correlations.

[116] Stochastic Dropout Module. A stochastic dropout module p is utilized at the end of the privacy model to prevent overfitting, where the representations e are randomly dropped during training as described by Srivastava et al. It ensures that the representations of privacy and task concepts cannot become overly correlated during training. Formally, some variables of the estimated privacy budget e are dropped using a mask m by e = m * e, where the mask is element-wise sampled from a Bernoulli distribution by m ~ Bernoulli(n) with the dropout rate coefficient n, and * denotes the element-wise multiplication.

[117] Thus, masking elements of the privacy parameter may comprise: obtaining a dropout rate coefficient which determines how many elements of the privacy parameter are masked; and sampling from a Bernoulli Distribution to determine a masking value for each element of the privacy parameter, wherein the probability of the Bernoulli Distribution returning a zero value is given by the dropout rate coefficient. When the Bernoulli Distribution returns a zero value, the element corresponding element of the privacy parameter may also be set to zero, meaning that no noise is added to the corresponding extracting feature. When the Bernoulli Distribution returns a value of 1, the privacy parameter remains as is. In other words, the output of the Bernoulli Distribution is multiplied with the current privacy parameter to mask elements of the privacy parameter.

[118] The new private latent space 8 learned in the framework must comprise two essential properties to address the limitations of other privacy methods: 1) Optimizing the Privacy-Utility Trade-off: Determining an optimal privacy budget for data is crucial, since it directly affects both the level of privacy and the ML task utility. 2) Optimization for Targeted Privacy Protection: Adding privacy protection to the entire dataset is impractical when certain data parts are more relevant than others for privacy protection and performing ML tasks. A targeted privacy protection can help to avoid adding unnecessary noise.

[119] Training Models. To preserve the privacy, the aim is to maximize the distance between the distributions of the original (non-private) datap(x) and private data p(xe). For this purpose, a PPM d£ is trained to estimate how much privacy is needed to distinguish the original and private data. On the other hand, a DPM f learns the privacy budget which is sufficient enough to separate the private data from the original data using the PPM d£.

[120] Training Privacy Protection Models. The NN-based model is trained using a minimax objective inspired by generative adversarial training. The adversarial training is reformulated for privacy problem, since vanilla adversarial training is not applicable to solve it. To defend against reconstruction attacks and protect sensitive information, the distance is maximized between the distributions of the original (non-private) p(x) and private data p(x£\ This differs from the conventional approach of minimizing the difference between the original data x and the private data xe, as the present goal is to ensure that the private data is sufficiently distant from the original data to prevent reconstruction attacks. First, PPM d£ is trained using the Wasserstein distance by max Ex[de(x)] — EXe[de(xe)]+A.Ex[(||Vxde(x)||2 — l)2], (1) s--------r-----■—i----------- gradientpenaltyterm where E[-] is the expectation operator, x = tx+ (1 -r)xe, and t is a uniformly distributed random variable over the interval of [0,1], To enforce the Lipschitz constraint as described in Gulrajani et al., regularization is applied in the Eq. (1) using the gradient penalty term, where A denotes the penalty term (which is set to 10 in experiments). Note that a PPM d£ implicitly learns distinct latent features to discriminate the original and private data. However, this information is ignored while training TMs in a vanilla setting and not used in the inference. By utilizing a shared backbone model < / >e, this critical information is incorporated into the present framework.

[121] Training Data Privatization Models. While training a DPM f, it is not possible to simply maximize the separation between the private data and the original data, since this can lead to a solution with no upper privacy bound. Hence, a constraint is introduced to the objective function. The optimization problem is reformulated and the objective function of the DPM f by min AsEXe [de(xe)] - EXe [de(s(xe))], (2) where the objective of the model s is to remove the random noise from the private data that is generated by the DPM f. Therefore, s is identified by a denoising model. In Eq. (2), the contribution of s to the optimization step is controlled by a constant As (empirically setting its value to 0.1). The objective function enforces that the distributions of the denoised private data xe and original data x must be closer. To this end, this model provides two advantages: 1) It reduces the random noise on the private data during training and implicitly avoids the collapse of privacy protection to infinite. 2) Instead of adding random noise, it forces the model y to learn distinct privacy characteristics that cannot be easily removed by the model s. In particular, it is desirable to learn distinct privacy characteristics that are different from random noise generated by the vanilla noise-based privacy methods that is harder to distinguish.

[122] Training Task-specific Models. The aim is to reduce the distance between the distributions p(y|x) and p(y|xe) so that the utility of TMs improves. An additional objective function is introduced to ensure that the distributions of p(y|x) and p(y|xe) do not significantly alter (i.e., ML task objective) during the minimax optimization step in Eq. 2. To this end, the ML model accuracy is preserved. An ML task model ce is utilised to categorize the samples accurately, and the models ce and f are trained simultaneously using the cross-entropy objective loss calculated using the ground truth labels y of data x. A loss is defined to maximize the protection against privacy threats, where the accuracy of TMs is ignored. To address this problem, an objective function is used to preserve the accuracy of TMs by optimization of model parameters. The objective function calculates the cross-entropy using the ground truth labels y of data x by max A2EXf[yO ^(ce(xe))],    (3) where A2 denotes a coefficient that scales the contribution of the objective function and O denotes the Hadamard product. Note that this task-specific loss function may vary depending on ML tasks. For instance, cross-entropy can be used for classification, and focal loss can be used for object detection.

[123] Figure 5 shows an algorithm for implementing an overall optimisation step for the framework described above.

[124] Implementation Details. The present framework is evaluated on image classification tasks. The final TM of the trusted participant ce is obtained by jointly optimizing the TM and the privacy models, utilizing the Adam optimizer as described by Diederik et al. with a learning rate a of 0.001, a batch size b of 256, and a total iteration of 10K. A2 is set to 0.4 based on the results obtained on the validation data. The dropout rate coefficient n and the scale coefficient Xs are respectively set to 0.5 and 0.1.

[125] For y and s models, NNs are utilized with an encoder-decoder structure (U-network concept as described by Ronneberger et al., 64 filters with 3x3 convolution kernels). Moreover, the submodels ip£ and <p£ are based on a two-layer fully connected network. Lastly, experiments with several NN architectures for DPM y model are conducted, including AlexNet as described by Krizhevsky et al., ResNet as described by He et al. and VGGnet as described by Simonyan et al., and obtained the best privacy-utility performance with ResNet. Depth of ¢, y and s is determined based on the resolution of input images.

[126] Experiments. The present framework is tested on image classification tasks. The models are trained using the Adam optimizer with a learning rate of 0.001, a batch size of 256, and a total iteration of 10K. The dropout rate coefficient n is set to 0.5. For the privacy model, a convolutional neural network (CNN) is employed based on the U-network architecture, which extracts high-level features from spatial pixel relationships. A ResNet model is utilized to implement feature extractors and classifiers for the classification tasks. In terms of computational complexity, the training time takes less than 30 mins and inference time is approximately 10 ms per image.

[127] Four benchmark datasets are used: MNIST, Fashion-MNIST (F-MNIST), CIFARIOand SVHN. The present framework is compared with three baselines: the vanilla Laplace mechanism (Dwork et al.), Dp-Pixel, DP-Adam and Adversarial Learning (Adv. Learning). For vanilla Laplace mechanism and Dp-Pixel, a similar privacy budget is utilized, estimated by the present framework for a fair comparison, varying between 0.5 and 1.0.

[128] Data Reconstruction Attack: An approach proposed by Zecheng He, Tianwei Zhang, and Ruby B Lee is used. In a white-box attack scenario, this approach reconstructs the input data from latent representations provided by a TM c£. Then, the reconstructed data and the original data are used to calculate Mean Square Error (MSE) and Structural Similarity Index (SSIM).

[129] Data Leakage Attack: To calculate leakage on private data xe e T£, the true classification rate (Acc(A)) is reported of ML task models ca learned by the adversary participant / malicious user and consider it as the data leakage risk. A high accuracy indicates that the utilized privacy mechanism may not sufficiently protect sensitive information, and data leakage may happen.

[130] Task Model Accuracy with Private Data: The true classification rate (Acc) of the TMs learned by trusted users with private data is reported.

[131] The present framework is primarily evaluated for two scenarios: an adversary participant has unauthorized access to either 1) the private data or 2) the ML task model (i.e., classification model) obtained by a trusted participant.

[132] Accessing Private Data. Private data xe obtained from a trusted user can be accessed by a malicious user m. Hence, a TM cm is used to replicate privacy threats. The adversary participant uses its own non-private data Sm to train a separate ML model cm to replicate privacy threats. Figure 6A shows results for data leakage risk and ML task model accuracy on private data, simultaneously, adversary ML model accuracy (Acc(a)) (for data leakage attack) (lower is better) and trusted ml model accuracy (Acc) (higher is better) using private data are reported. The desired outcome of this scenario is that the adversary participant accurately classifies the private data (i.e., minimal data leakage with low accuracy). In contrast, the ML model ce trained by the trusted participant should achieve high classification accuracy on the private data xe.

[133] First, the results for this scenario without adding privacy as a baseline are presented. As expected, the Laplace mechanism provides a privacy protection, but it comes at the cost of reduced accuracy for the trusted ML model. The reason is that privacy and utility must be carefully balanced through manual tuning. Second, the Adversarial Learning model is tested. The protection against privacy leakage attacks remains largely unchanged. Notably, the accuracy of the trusted ML model is improved on all datasets with the present framework, while preserving a similar level of privacy compared to other baselines. Figure 6B shows data leakage risk (Acc(A)) and the accuracy of TMs cm (Acc) obtained under data leakage attacks. The results show data leakage risk and ML task model accuracy on private data, simultaneously. Malicious TM cm accuracy (Acc(A)) (for data leakage attack) (lower is better) and trusted TM ce accuracy (Acc) (higher is better) using private data are reported. The results show that baseline models ensure privacy protection for privacy leakage attacks. However, they decrease the accuracy of TMs by up to 50%.

[134] The impact of using the same backbone is tested for both PPMs de and TMs ce. This setup is referred to as Shared B.Bone in the tables. The results show that the present framework with a shared backbone (i.e., w / Shared B.Bone) consistently achieves the best privacy-utility trade-off on all four datasets with up to 10% accuracy improvement for TMs.

[135] Accessing Task-specific Models. In this scenario, the adversary participant has unauthorized access to only a TM ce. Figures 7A and 7B show experimental results for reconstruction attacks. MSE (higher is better), SSIM (lower is better) and trusted TM ce accuracy (Acc) (higher is better) are reported. The reconstruction performance is evaluated by analyzing the feature spaces of TMs ce. More precisely, the activations are calculated of the conv2 layer of the backbone (pe, and employ the approach proposed by Zecheng He, Tianwei Zhang, and Ruby B Lee.

[136] The results show that the employment of the vanilla Laplace mechanism enhances the privacy, but the performance improvement is limited compared to the other methods. Dp-Adam obtains high privacy particularly for the CIFAR10 and SVHN datasets, at the cost of accuracy (accuracy drops up to 40%). PATE obtains high privacy and accuracy at the same time compared to the other methods. For the present framework, sharing backbone amongst models helps to learn distinct features by separating the distributions of the latent representation of original and private data. It is shown that the protection is significantly enhanced for privacy threats, while preserving high accuracy of TMs with the present framework.

[137] The Change of Privacy-Utility Trade-off Over Time. Figure 8A shows how the privacyutility trade-off changes during training. Iteration and high-order polynomial regression results are plotted for each dataset to visualize how the trade-off evolves. The results demonstrate that privacy-utility trade-off is effectively optimized by the present framework. Figure 8B shows privacy and utility (i.e., task accuracy) trade-off for the data leakage risk attack. Each curve consists of five points (□) representing different privacy budgets used by the baseline privacy model (as in Dwork). The privacy budgets for the baseline model are 0.4, 0.5, 0.65, 1.0, and 2.0 (left-to-right). The task accuracy is dropped for lower privacy leakage risks. The results provided by the present framework are illustrated as separate points (*). The present framework improves the privacy-utility trade-off compared to the baseline.

[138] The Impact of Targeted Privacy Protection. One property of the present framework is to estimate targeted privacy budgets for privacy protection, allowing for more precise and effective protection compared to adding randomized noise to the entire data. This section analyses how this property affects the overall performance. A similar setup is used, as described above. The estimated privacy budgets are similar to those previously reported in Figure 6A, so the focus is on reporting the accuracy of the trusted ML model with private data. The results presented in Figure 9A show that using a targeted privacy budget (i.e., Target. Budget) significantly enhances the accuracy of the ML tasks. The present framework can locally optimize the balance between privacy and utility.

[139] Ablations: Robustness of Privacy Models. The robustness of the DPM is analysed, compared to vanilla privacy methods. Ideally, privacy characteristics of the present framework and vanilla methods might be similar, since both of them use Laplacian mechanisms. This can lead to a privacy risk by replicating the TMs or their private training data estimated by the present framework. Figure 9B shows experimental results for evaluating the robustness of DPMs. Malicious TM cm accuracy (Acc(A)) is reported (lower is better) to analyze data leakage attacks, lower Acc(A) values indicate stronger privacy protection. Results for both cases are presented when either the private data x£ (data) or the TM c£ (model) is accessed by a malicious user. The results demonstrate that the present framework learns distinct privacy characteristics for both the data and TMs that are harder to be replicated.

[140] Moreover, the noise patterns of the framework are analysed on benchmark datasets. First, some samples from benchmark datasets are provided that are privatized by the present framework. Figure 10 shows examples from image classification datasets used in the present experiments. Figure 11A shows the samples distorted with noise for privacy protection. Later, Figure 11B shows the average localized noise estimated by the present frameworks. In particular, noise is mainly added to the parts where content information is presented. During training, no constraint is imposed for this property and the present framework automatically decides that this type of noise is the best for privacy-data utility trade-off. This critical feature provides protection to privacy threats and makes harder to reconstruct / infer data and task models.

[141] In summary, a framework is presented for implementing privacy models that learn privacy budget to improve the privacy-utility trade-off of TMs. For this purpose, a shared backbone is utilised for privacy model and TMs that learns representative privacy and task representations simultaneously. In particular, this feature provides privacy protection against reconstruction attacks for TMs. Also, a novel objective function is proposed using a minimax optimization that improves the stability of privacy estimation and facilitates learning distinct privacy characteristics. The experimental results indicate the superiority of the present framework over baselines in terms of privacy protection and utility of TMs.

[142] Figure 12 shows a system 1000 for implementing the present techniques. The system comprises a server 100 and a plurality of client devices 200. For the sake of simplicity, a single client device 200 is shown here.

[143] The server 100 is fortraining a privacy protecting machine learning, ML, model 106 for protecting privacy of data sent from client devices to the server. The server comprise: at least one processor 102 coupled to memory 104. The server is configured for: obtaining a training dataset comprising a plurality of data items; obtaining a task-specific ML model 108 for performing a specific task with respect to a data item and generating an output for the data item; obtaining a discriminative ML model 110 for determining a relationship between features of a data item and an output generated by the task-specific ML model for the data item; and training the privacy protecting ML model 106, using each data item, by: extracting features from the data item using the privacy protecting ML model 106; assigning, using the privacy protecting ML model 106, a privacy parameter to each extracted feature of the data item; generating, using the privacy protecting ML model 106, a corresponding noisy data item by adding noise to each extracted feature of the data item using the assigned privacy parameter of the extracted feature; inputting the data item and the noisy data item into the task-specific ML model 108, and generating a first output for the data item and a second output for the noisy data item from the task-specific ML model 108; determining, using the discriminative ML model 110, the features of the data item and the noisy data item that resulted in differences between the first and second outputs; and training the privacy protecting ML model 106, using the second output and the features determined by the discriminative ML model 110, to optimise the assigned privacy parameter for each extracted feature.

[144] The client device 200 is for protecting privacy of data sent from the client device 200 to the server 100. The client device 200 comprises: storage storing a plurality of user data items; a trained privacy protecting machine learning, ML, model 206 that has been trained using any of the methods described herein; and at least one processor 202 coupled to memory 204, for: retrieving some or all of the plurality of user data items from the storage; and anonymising the retrieved data items using the trained privacy protecting ML model to generate anonymised user data items.

[145] The client device 200 may comprise a local ML model for use in federated learning, wherein the local ML model is a version of a global ML model of the server.

[146] References: • Dwork et al - Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA • Dinur et al - Irit Dinur and Kobbi Nissim, “Revealing information while preserving privacy,” in Proceedings of the Twenty-Second ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, New York, NY, USA, 2003, PODS ’03, p. 202-210, Association for Computing Machinery. • Martin Jankowiak and Fritz Obermeyer, “Pathwise derivatives beyond the reparameterization trick,” in International Conference on Machine Learning. PMLR, 2018, pp. 2235-2244. • Martin Jankowiak and Theofanis Karaletsos, “Pathwise derivatives for multivariate distributions,” in The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 2019, pp. 333-342. • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778. • Yuxin Wu and Kaiming He, “Group normalization,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 3-19. • Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929-1958, 2014. • DP-Adam: Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville, “Improved training of Wasserstein GANs,” Advances in Neural Information • Diederik P. Kingma and Jimmy Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, 2015. • Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention-MICCAI. Springer, 2015, pp. 234-241. • Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing systems, vol. 25, 2012. • Karen Simonyan and Andrew Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014. • MNIST : Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner, “Gradientbased learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998. • Fashion-MNIST: Han Xiao, Kashif Rasul, and Roland Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv: 1708.07747, • CIFAR10: Alex Krizhevsky, Geoffrey Hinton, et al., “Learning multiple layers of features from tiny images,” 2009. • SVHN: Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al., “Reading digits in natural images with unsupervised feature learning,” in NIPS Workshop on Deep Learning and Unsupervised Feature Learning. Granada, Spain, 2011, vol. 2011, p. 7. • Dp-Pixel: Liyue Fan, “Image pixelization with differential privacy,” in Data and Applications Security and Privacy XXXII: 32nd Annual IFIP WG 11.3 Conference, DBSec 2018, Bergamo, Italy, July 16-18, 2018, Proceedings 32. Springer, 2018, pp. 148-162. • Adversarial Learning: Sicong Liu, Junzhao Du, Anshumali Shrivastava, and Lin Zhong, “Privacy adversarial network: representation learning for mobile data privacy,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 3, no. 4, pp. 1-18, 2019. • Zecheng He, Tianwei Zhang, and Ruby B Lee, “Model inversion attacks against collaborative inference,” in Proceedings of the 35th Annual Computer Security Applications Conference, 2019, pp. 148-162.

[147] Those skilled in the art will appreciate that while the foregoing has described what is considered to be the best mode and where appropriate other modes of performing present techniques, the present techniques should not be limited to the specific configurations and methods disclosed in this description of the preferred embodiment. Those skilled in the art will recognise that present techniques have a broad range of applications, and that the embodiments may take a wide range of modifications without departing from any inventive concept as defined in the appended claims.

Claims

1. A computer-implemented method, performed on a server, for training a privacy protecting machine learning, ML, model for protecting privacy of data sent from client devices to the server, the method comprising:obtaining a training dataset comprising a plurality of data items;obtaining a task-specific ML model for performing a specific task with respect to a data item and generating an output for the data item;obtaining a discriminative ML model for maximising differences between features of a data item and an output generated by the task-specific ML model for the data item; andtraining the privacy protecting ML model, using each data item, by:extracting features from the data item using the privacy protecting ML model;assigning, using the privacy protecting ML model, a privacy parameter to each extracted feature of the data item;generating, using the privacy protecting ML model, a corresponding noisy data item by adding noise to each extracted feature of the data item using the assigned privacy parameter of the extracted feature;inputting the data item and the noisy data item into the task-specific ML model, and generating a first output for the data item and a second output for the noisy data item from the task-specific ML model;determining, using the discriminative ML model, the features of the data item and the noisy data item that resulted in differences between the first and second outputs; andtraining the privacy protecting ML model, using the second output and the features determined by the discriminative ML model, to optimise the assigned privacy parameter for each extracted feature.

2. The method as claimed in claim 1 wherein assigning a privacy parameter to each extracted feature of the data item comprises:assigning a random privacy parameter to each extracted feature.

3. The method as claimed in claim 1 or 2 wherein training the privacy protecting ML model to optimise the assigned privacy parameter comprises:maximising a difference between each data item and the corresponding noisy data item; andminimising a difference between the first output and the second output generated by the task-specific ML model.

4. The method as claimed in any preceding claim wherein training the privacy protecting ML model comprises:training the task-specific ML model and the discriminative module ML model simultaneously,wherein the task-specific ML model and the discriminative ML model share a common backbone architecture.

5. The method as claimed in claim 4 wherein training the task-specific ML model and the discriminative module ML model simultaneously comprises:training the task-specific ML model and the discriminative ML model each time the privacy parameter changes during the process to optimise the privacy parameter assigned to each extracted feature.

6. The method as claimed in any preceding claim, wherein generating a corresponding noisy data item comprises:generating a feature vector of the data item, each element of the feature vector corresponding to an extracted feature; andadding the assigned privacy parameter for each extracted feature to each corresponding element of the feature vector, to thereby generate a noisy feature vector.

7. The method as claimed in claim 6 wherein inputting the data item and the noisy data item into the task-specific ML model comprises inputting:the generated feature vector for the data item into the task-specific ML model; andthe generated noisy feature vector into the task-specific ML model.

8. The method as claimed in claim 6 or 7 wherein the method further comprises: normalising the generated feature vectors and generated noisy feature vectors after the generating and prior to the inputting.

9. The method as claimed in any of claims 6 to 8 further comprising:masking elements of the generated noisy feature vectors after the generating and prior to the inputting.

10. The method as claimed in claim 9 wherein masking elements of the generated noisy feature vectors comprises:obtaining a dropout rate coefficient which determines how many elements of the generated noisy feature vectors are to be masked;generating a mask vector comprising at least one masked element, wherein the number of masked elements is based on the obtained dropout rate coefficient, and at least one unmasked element; andapplying the mask vector to each generated noisy feature vector.

11. The method as claimed in claim 10 wherein applying the mask vector comprises multiplying the generated noisy feature vector by the mask vector.

12. The method as claimed in claim 10 or 11 wherein generating a mask vector comprises: determining a masking value for each masked element of the mask vector by sampling a value from a Bernoulli Distribution.

13. The method as claimed in any preceding claim wherein generating a noisy data item by adding noise comprises:sampling, for each extracted feature of the data item, a random noise value from a Laplacian Distribution; andadding the sampled random noise value to the extracted feature of the data item.

14. The method as claimed in claim 13 wherein sampling a random noise value from a Laplacian Distribution comprises:using the assigned privacy parameter for the extracted feature to determine a variance of the Laplacian Distribution.

15. A server for training a privacy protecting machine learning, ML, model for protecting privacy of data sent from client devices to the server, the server comprising:at least one processor coupled to memory, for:obtaining a training dataset comprising a plurality of data items;obtaining a task-specific ML model for performing a specific task with respect to a data item and generating an output for the data item;obtaining a discriminative ML model for maximising differences between features of a data item and an output generated by the task-specific ML model for the data item; andtraining the privacy protecting ML model, using each data item, by:extracting features from the data item using the privacy protecting ML model;assigning, using the privacy protecting ML model, a privacy parameter to each extracted feature of the data item;generating, using the privacy protecting ML model, a corresponding noisy data item by adding noise to each extracted feature of the data item using the assigned privacy parameter of the extracted feature;inputting the data item and the noisy data item into the task-specific ML model, and generating a first output for the data item and a second output for the noisy data item from the task-specific ML model;determining, using the discriminative ML model, the features of the data item and the noisy data item that resulted in differences between the first and second outputs; andtraining the privacy protecting ML model, using the second output and the features determined by the discriminative ML model, to optimise the assigned privacy parameter for each extracted feature.

16. The server as claimed in claim 15, wherein the at least one processor is configured to: transmit the trained privacy protecting ML model to a plurality of client devices.

17. A computer-implemented method, performed on a client device, for protecting privacy of data sent from the client device to a server, the method comprising:retrieving a plurality of user data items stored on the client device; andanonymising the retrieved data items using a trained privacy protecting machine learning, ML, model that has been trained using the methods of any of claims 1 to 14 to generate anonymised user data items.

18. The method as claimed in claim 17 wherein data is sent to the server for federated analytics, FA, and wherein the method further comprises:sending the anonymised data items to the server.

19. The method as claimed in claim 18 wherein the method further comprises, prior to retrieving a plurality of data items:receiving, from the server, a request for the plurality of data items.

20. The method as claimed in claim 17 wherein data is sent to the server for federated learning, FL, and wherein the method further comprises:training a local machine learning, ML, model using the anonymised data items, wherein the local ML model is a version of a global ML model of the server; andsending model parameters of the trained local ML model to the server.

21. The method as claimed in claim 20 wherein the method further comprises, prior to retrieving a plurality of data items:receiving, from a server, a request for model parameters of the local ML model.

22. A client device for protecting privacy of data sent from the client device to a server, the client device comprising:storage storing a plurality of user data items;a trained privacy protecting machine learning, ML, model that has been trained using the methods of any of claims 1 to 14; andat least one processor coupled to memory, for:retrieving some or all of the plurality of user data items from the storage; andanonymising the retrieved data items using the trained privacy protecting ML model to generate anonymised user data items.

23. A computer-readable storage medium comprising instructions which, when executed by a processor, causes the processor to carry out the method of any of claims 1 to 14, or 17 to 21.

Images