Method and mechanism for confidential and privacy protection multi-benefit related party machine learning

Abstract

There is described a computing device (900) for confidential multi-benefit correlator machine learning, the computing device (900) being configured to: obtain a trained machine learning model (302), the trained machine learning model (302) being obtained by executing machine learning code trained on input training data (301); evaluating the trained machine learning model (302) to establish a privacy leak level of the input training data (301); and outputting the trained machine learning model (302) if the privacy leak level of the input training data (301) is below a privacy leak level threshold. In this manner, the trained machine learning model is output only when the privacy leak level is below the privacy leak level threshold, which can protect confidentiality and privacy.

Description

Technical Field

[0001] The present invention relates to an apparatus and method for confidential multi-stakeholder machine learning. Background Art

[0002] Confidential multi-stakeholder machine learning (CML) using a trusted execution environment (TEE) may enable multiple parties / stakeholders to conduct collaborative machine learning (ML) computations without exposing their intellectual property (IP), such as ML source code and input datasets. However, this approach may still suffer from issues protecting the privacy of input datasets, as trained models may be shared among the parties, and this approach may expose the privacy of the training input datasets. Stakeholders are able to perform various attacks on the trained models, such as model inversion attacks, membership inference attacks, and model extraction attacks, to exploit the privacy of the input training datasets.

[0003] ML has become a popular approach for building functional artificial intelligence (AI) systems such as image / speech recognition systems, natural language processing systems, and medical expert systems. To build such AI systems, collaboration between multiple parties or stakeholders with different knowledge domains may be required, as machine learning is fundamentally multi-stakeholder computing (see Figure 1 ).

[0004] Figure 1 An exemplary conventional machine learning apparatus 100 from the prior art is schematically illustrated. Training data 101 is supplied for training 102 to produce a trained model 103. Stakeholders may include cloud providers 104, data owners 105, training code owners 106, and model owners 107. Stakeholders share source code, data, and computing power to build ML applications. They require a framework to establish mutual trust and securely share code and data.

[0005] Stakeholders can jointly perform machine learning to build AI systems by sharing their intellectual property, including private training data 101, ML source code 102, and ML models 103. In this setting, stakeholders may be keen to protect their IP, ensuring confidentiality during the joint machine learning computation. Training data owners 105 may want to ensure that training data 101 is protected at rest, in transit, and during the training computation. This is because the data may include sensitive information; for example, healthcare data used to train diagnostic models may include privacy-sensitive patient records. Training code owners 106 may want to protect their training code 102 (e.g., Python code), which typically includes optimized training model architecture and tuning parameters. This source code can provide business value and inference model quality; therefore, its confidentiality is as important as the training data 101. The output model 103 of the machine learning computation pipeline is shared among stakeholders but may need to be protected from non-stakeholders, such as cloud providers 104 (training computations can require significant computing power and therefore benefit from running on a scalable cloud infrastructure). To collaborate, stakeholders must trust that the others adhere to the rules that protect each other's IP. However, building trust among them can be difficult. First, some stakeholders might collude to gain an advantage over others. Second, even trustworthy stakeholders might lack the expertise to protect their IP from skilled attackers accessing their computing resources. Several recent work has been proposed to support stakeholders in jointly performing machine learning without exposing their IP. Many systems rely on cryptographic techniques, such as secure multi-party computation and fully homomorphic encryption, to protect the confidentiality of stakeholders' IP. However, these cryptographic techniques can introduce significant overhead, which may prevent them from being used for computationally intensive training calculations.

[0006] Figure 2 The schematic diagram shows an exemplary federated machine learning apparatus 200 of the prior art compared to an exemplary conventional machine learning apparatus 100 of the prior art. Training data 201a, 201b, 201c is provided and trained by each of clients 204a, 204b, 204c. The trained local models are then combined 202 to produce a trained model 203.

[0007] Recent work has employed trusted execution environments (TEEs) to build ML systems, demonstrating orders of magnitude speedups relative to cryptographic-only schemes. TEE technology provides hardware-protected memory regions called enclaves. The CPU protects the integrity and confidentiality of applications executing within the enclave from the operating system, hypervisor, and system administrators. Systems relying on TEEs can ensure the integrity and confidentiality of IP (e.g., training data, training code, models). While promising at first glance, TEEs can be subject to model membership inference attacks or model inversion attacks, which exploit confidential information in models shared between stakeholders (see Figure 2 ). In these attacks, the attacker can analyze the shared model to identify the presence of specific records in the input training dataset. In practice, some work has demonstrated that images extracted from face recognition systems can look similar to images in the underlying training data. The attacker can perform these attacks using both black-box and white-box settings. In the black-box setting, the attacker can only observe the predictions of the model to measure the privacy risk to legitimate users of the model who seek predictions for their queries. In the white-box setting, the attacker can observe the parameters of the model. This reflects the scenario when the output model is shared between stakeholders as described above or in aggregated computations in federated learning (see Figure 2 ).

[0008] Turning to existing technology systems:

[0009] Secure ML using TEEs. Recently, some systems have been developed that use TEEs to support secure ML. However, these systems may not be able to handle the aforementioned model attacks. Furthermore, some systems may not support secure multi-stakeholder machine learning computations. Some systems only support ML inference computations, but not ML training computations.

[0010] Secure multi-party computation. While cryptographic techniques such as secure multi-party computation (MPC) and fully homomorphic encryption hold promise for supporting secure multi-stakeholder ML computation, their practical application may be limited. They may introduce high-performance overhead (a limiting factor for computationally intensive ML) and may require significant modifications to existing ML code. Furthermore, systems may not support all ML algorithms, such as deep neural networks. Some systems may also require additional assumptions, such as MPC protocols requiring a subset of honest stakeholders. Consequently, many systems may lack support for training computations.

[0011] Privacy-preserving ML. Some systems use differential privacy mechanisms to protect shared models. However, these systems may focus solely on privacy protection and fail to support multi-stakeholder machine learning. They also do not ensure the confidentiality and integrity of ML code and input data. Secure ML computation requires that the confidentiality and integrity of code and input data be protected not only at rest but also during computation.

[0012] It would be desirable to develop an apparatus and method that overcomes the above-mentioned problems. Summary of the Invention

[0013] According to a first aspect, a computing device for confidential multi-stakeholder machine learning is provided, the computing device comprising one or more processors and a memory, the memory storing data defining program code executable by the one or more processors in a non-transitory form, wherein the program code is executable by the one or more processors such that the computing device is configured to: obtain a trained machine learning model, the trained machine learning model being obtained by executing the machine learning code trained on input training data; evaluate the trained machine learning model to establish a privacy leakage level of the input training data; and output the trained machine learning model when the privacy leakage level of the input training data is below a privacy leakage level threshold. In this manner, the trained machine learning model is output only when the privacy leakage level is below the privacy leakage level threshold, thereby protecting confidentiality and privacy, thereby reducing the risk of privacy leakage of the input data.

[0014] In some implementations, a computing device may be configured to: enhance the privacy of input training data in a trained machine learning model when the privacy leakage level of the input training data is below a privacy leakage level threshold; and output the trained machine learning model with enhanced privacy. In this manner, the trained machine learning model may be made more confidential and private before being output, which may reduce the risk of input data privacy leakage.

[0015] In some implementations, a computing device can be used to evaluate a trained machine learning model by simulating attacks against the trained machine learning model. In this way, the trained machine learning model can be tested for privacy using a method that could be used by a hacker attempting to obtain private input training data.

[0016] In some implementations, a computing device can be configured to evaluate a trained machine learning model by simulating one or more of model inversion, membership inference, or model extraction attacks against the trained machine learning model. In this manner, the trained machine learning model can be tested for privacy using a method that could be exploited by a hacker attempting to obtain private input training data.

[0017] In some implementations, a computing device can be configured to evaluate a trained machine learning model based on a probability that an attack correctly identifies candidate training data as input training data for the trained machine learning model. In this manner, the trained machine learning model can be prevented from outputting capabilities that compromise the privacy of the input training data.

[0018] In some implementations, the computing device can be configured to determine a privacy leakage level threshold based on a function between the privacy leakage level of the input training data and the performance level of the machine learning model. In this way, privacy and performance can be optimized based on the requirements of stakeholders.

[0019] In some implementations, a computing device can be configured to enhance the privacy of input training data in a trained machine learning model by reducing the quality of the input training data. In this manner, the privacy of the input training data can be enhanced by reducing the ability to understand what the input training data is.

[0020] In some implementations, a computing device may be configured to enhance the privacy of input training data in a trained machine learning model by introducing noise into the trained machine learning model. In this manner, the privacy of the input training data may be enhanced by reducing the ability to understand what the input training data is.

[0021] In some implementations, a computing device can be configured to introduce differentially private noise into a trained machine learning model. In this way, the privacy of input training data can be altered by the differential noise.

[0022] In some implementations, the computing device can be configured to determine differential privacy noise based on a function of the privacy leakage level of the input training data and the performance level of the trained machine learning model. In this manner, privacy and performance can be optimized using differential noise based on stakeholder requirements.

[0023] In some implementations, a computing device can be configured to introduce zero-knowledge privacy noise into a trained machine learning model. In this way, privacy and performance can be optimized based on stakeholder requirements using zero-knowledge privacy noise.

[0024] In some implementations, a computing device can be configured to enhance the privacy of input training data in a trained machine learning model by randomizing the input training data. In this way, for binary data, privacy and performance can be optimized based on the randomized parameters as required by stakeholders.

[0025] In some implementations, a computing device can be configured to: obtain input training data and machine learning code; encrypt the input training data and the machine learning code; and train the machine learning code using the input training data to generate a trained machine learning model, wherein the trained machine learning model is encrypted. In this manner, the device can train the machine learning code while also encrypting any private data in a standard machine learning implementation.

[0026] In some implementations, a computing device may be configured to: obtain multiple pairs of local input training data and corresponding local machine learning code; generate corresponding trained local machine learning models using the local input training data of the local machine learning code; encrypt the corresponding trained local machine learning models; and aggregate the corresponding trained local machine learning models to generate a trained machine learning model, where the trained machine learning model is a globally trained machine learning model. In this manner, the device may be able to train the machine learning code while also encrypting any private data in a federated machine learning implementation.

[0027] In some implementations, a computing device can be configured to perform encryption via a trusted execution environment (TEE), so that data on the device can be encrypted.

[0028] According to a second aspect, a method for confidential multi-stakeholder machine learning is provided, comprising the steps of: obtaining a trained machine learning model, the trained machine learning model being obtained by executing machine learning code trained on input training data; evaluating the trained machine learning model to establish a privacy leakage level of the input training data; and outputting the trained machine learning model when the privacy leakage level of the input training data is below a privacy leakage level threshold. In this manner, the trained machine learning model is output only when the privacy leakage level is below the privacy leakage level threshold, thereby protecting confidentiality and privacy, thereby reducing the risk of privacy leakage of the input data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be described below by way of examples with reference to the accompanying drawings. In the accompanying drawings:

[0030] Figure 1 An exemplary conventional machine learning apparatus of the prior art is schematically shown.

[0031] Figure 2 An exemplary federated machine learning apparatus of the prior art is schematically shown for comparison with an exemplary traditional machine learning apparatus of the prior art.

[0032] Figure 3An exemplary arrangement of the first embodiment is schematically shown.

[0033] Figure 4 An exemplary algorithm for adding privacy noise is shown.

[0034] Figure 5 An exemplary list of strategies for training code is shown.

[0035] Figure 6 An exemplary list of ML strategies for a model is shown.

[0036] Figure 7 An exemplary arrangement of the second embodiment is schematically shown.

[0037] Figure 8 An example of a computer-implemented method for confidential multi-stakeholder machine learning is shown.

[0038] Figure 9 An example of an apparatus for performing the methods described herein is shown. DETAILED DESCRIPTION

[0039] The apparatus and methods described herein relate to confidential multi-stakeholder machine learning.

[0040] Embodiments of the present invention can address one or more of the aforementioned issues by evaluating a trained machine learning model to establish a privacy leakage level of input training data; and outputting the trained machine learning model when the privacy leakage level of the input training data is below a privacy leakage level threshold. In this way, the trained machine learning model is output only when the privacy leakage level is below the privacy leakage level threshold, thereby protecting confidentiality and privacy.

[0041] The system and method described in this paper aims to address confidentiality and privacy issues in multi-stakeholder machine learning. Specifically, the system can not only ensure the confidentiality and integrity of intellectual property (input datasets, ML code, and models), but also support them to measure the privacy leakage of output trained models and strike a balance between privacy and utility. Compared with previous work, the core idea behind this system is that it may introduce an additional component that measures the privacy leakage of the trained model before sharing it with the participants. If the privacy indicator value exceeds a threshold agreed upon by the stakeholders, it may automatically add noise to the model until it reaches an allowed value. The additional component itself can also run within the enclave using a TEE to ensure the integrity of the measurement computations.

[0042] This system potentially enables multiple stakeholders (who don't necessarily trust each other) to come together and perform machine learning to reap the benefits of AI and big data. The system ensures the confidentiality and integrity of ML computations (including legitimate input training data and ML code), meaning no one can interfere with the computation, including attackers with privileged access. Furthermore, the system protects shared models from membership inference attacks and model inversion attacks. The system can be applied to both traditional machine learning and federated learning settings.

[0043] Specifically, another system is described. It provides a framework that enables stakeholders to share their code and data only with certain ML applications running within an enclave, and to publish shared model outputs only if they pass privacy leak checks. The system can rely on encryption provided by the TEE to protect IP and on a trusted key management service to generate and distribute encryption keys. The TEE can provide confidentiality and integrity guarantees for the ML applications and key management service. Furthermore, the system can use a privacy check component to ensure the privacy protection of the output training model. If the output model fails the privacy check, the system can activate a privacy protection mechanism, for example, by adding differential privacy noise to the machine learning pipeline, until the shared output model passes the privacy check.

[0044] The core of the system described in this article is to provide a framework to enable multiple stakeholders (who do not necessarily trust each other) to still come together and perform machine learning to gain the benefits of AI and big data. The system can provide the following capabilities:

[0045] Ensure the confidentiality and integrity of ML computations (including the soundness of input training data).

[0046] No one can interfere with software execution, including attackers with privileged access.

[0047] Provides an attestation mechanism to enable stakeholders to ensure that ML computations are running in a TEE.

[0048] Remote attestation mechanisms help stakeholders establish trust in ML frameworks deployed in cloud providers’ infrastructure.

[0049] Transparent and automatic attestation helps stakeholders deploy confidential and scalable ML computations.

[0050] Ensure the privacy of input training data and protect the output model from membership inference attacks or model inversion attacks.

[0051] A calibration controller is introduced to enforce output models to follow stakeholder-defined policies before publishing them. It can use privacy and quality checkers to verify privacy leaks and the quality of output models.

[0052] A privacy-preserving component is provided to protect the privacy of the output models, providing a mechanism to add DP noise directly to the output models to keep them differentially private.

[0053] The calibration controller, privacy and quality checkers, and privacy protection components run within the TEE enclave, which may be attested to by stakeholders.

[0054] The provided mechanisms and methods can be extended to be applicable to federated learning.

[0055] Figure 3 An exemplary apparatus 300 of the first embodiment is schematically shown. The apparatus 300 may include: Figure 3 Pipeline shown.

[0056] The device can support multi-stakeholder ML computations. Stakeholders can establish trust in a security policy manager using a remote attestation mechanism provided by a trusted execution environment (TEE). In the security policy, they define which stakeholder applications can access the encryption keys that support decrypting confidential code or data, as well as the level of privacy assurance they want to achieve for shared output models. The TEE can protect code, data, and encryption keys.

[0057] Device 300 may be implemented on a TEE. The TEE may provide a first layer of protection, privacy, and / or confidentiality for device 300. All of device 300 may be implemented on a TEE. Alternatively, specific portions of device 300 may be implemented on a TEE. For example, portions of device 300 requiring a higher level of protection may be implemented on a TEE.

[0058] TEEs, such as ARM TrustZone, Intel SGX, AMD SEV SNP, and Intel TDX, provide a typical environment isolated from the main operating system and other untrusted software running on the same platform. This ensures that sensitive applications running in the TEE are protected from external interference or tampering. A TEE is a tamper-resistant processing environment that guarantees the authenticity, integrity, and confidentiality of its executed code, data, and runtime state (such as CPU registers, memory, and sensitive I / O). The contents of the TEE remain resistant to software attacks (even from privileged code) and any physical attacks on the system's main memory. Furthermore, the TEE can provide attestation to prove its trustworthiness to third parties. TEE technologies, such as Intel Software Guard Extensions (SGX), have garnered significant attention in both industry and academia. TEEs protect cloud-native applications (code and data) not only at rest and in transit, but also during computing (in use), against powerful attackers with privileged access to the underlying system software (such as the operating system or hypervisor) and hardware. To ensure application confidentiality and integrity, TEEs execute their code and data within encrypted memory areas called enclaves. Attackers with privileged access cannot read or interfere with the memory region; only the processor can decrypt and execute applications within the enclave. Furthermore, TEEs like Intel SGX provide users with a mechanism to verify that the TEE is genuine and that an adversary has not altered the applications running within the TEE enclave. This verification process is called remote attestation and can enable users to establish trust in applications running within the enclave on a remote host.

[0059] Specifically, the device 300 can use TensorFlow and the SCONE platform because SCONE provides an ecosystem for running unmodified applications within the TEE (i.e., Intel SGX). In addition, the key management system provided by SCONE can be extended to implement a security policy manager component. Intel SGX can be used as a TEE engine.

[0060] Figure 3The high-level architecture of apparatus 300 is shown. The apparatus 300 includes the following components: (i) stakeholders 304, 306, 308, i.e., parties that wish to jointly perform ML while protecting their IP in terms of security and privacy; (ii) a security policy manager 310, i.e., a key management and configuration service that enables stakeholders 304, 306, 308 to share IP for ML computations without exposing them; (iii) ML training 312, i.e., ML training computations; (iv) a calibration controller 314, i.e., a component that verifies / executes the output model to comply with the policies defined by the stakeholders 304, 306, 308 by using two components: (1) a privacy and quality checker 315, i.e., a component that performs privacy and quality measurements and checks whether they meet the requirements defined in the policies provided by the stakeholders, (2) a privacy protector 316, i.e., a component that can be activated to enhance the privacy guarantees of the output model (e.g., add differential privacy noise) until it meets the policies; and (v) a TEE, i.e., a hardware security element that enables confidentiality and integrity of ML computations on untrusted computing resources (e.g., in the public cloud). To support multiple stakeholders in executing ML while maintaining the confidentiality of their IP, which may remain under the control of the stakeholders, the output model shared between stakeholders (it should be noted that the ML model owner can be a group of stakeholders) needs to ensure that no sensitive information of the underlying training data is exposed. To achieve this goal, the system can use a security policy manager 310 that acts as a root of trust. Stakeholders use the remote attestation mechanism provided by the TEE (for example, Intel SGX remote attestation) to establish trust in this component.

[0061] Apparatus 300 may be configured to obtain input training data 301. In other words, apparatus 300 may be configured to request and / or receive input training data 301. Apparatus 300 may receive training data 301 from a training data owner 304. Training data owner 304 may be a computing system distinct from apparatus 300. Alternatively, training data owner 304 may be part of apparatus 300. Training data owner 304 may provide training data 301 to apparatus 300 for use in training machine learning code.

[0062] Training data 301 may include supervised machine learning data pairs. Training data 301 may include input and ground truth pairs. Training data 301 may relate to any suitable machine learning data. For example, training data 301 may include one or more of image data, text data, and numerical data.

[0063] Apparatus 300 may be configured to obtain a training data policy 305. In other words, apparatus 300 may be configured to request and / or receive a training data policy 305. Apparatus 300 may receive training data policy 305 from a training data owner 304. Training data owner 304 may provide training data policy 305 to apparatus 300 for use in training machine learning code. Alternatively, apparatus 300 may define or generate training data policy 305.

[0064] Training data policy 305 may include restrictions on the use of training data 301. Training data policy 305 may include information about intellectual property rights in training data 301. Training data policy 305 may include information about encryption keys to decrypt input training data 301.

[0065] Figure 5 An example training code policy 307 listing 500 is shown. This policy has a unique name (line 1), which typically combines the name of the stakeholder and their IP name. The ML computation definition can include the commands required to execute the computation within the TEE enclave (line 3) and a cryptographic hash of the source code content that implements the ML computation (line 9). The security policy manager 310 can use the hash to authenticate the ML computation before providing encryption keys to the ML computation. A data body (line 4) includes code, input, or output models. A data body is a collection of files encrypted with encryption keys managed by the security policy manager 310. Policies can import and export keys from other policies. For example, the training code owner policy 307 can import keys from the training data owner policy 305 to obtain keys for decrypting training data bodies (line 6). It can also export keys to model owners or privacy checker policies to decrypt candidate model data bodies (line 8). Only authorized computations (training, privacy checking, and adding noise) can access the keys required to decrypt data bodies and access IP. The public keys of the stakeholders are embedded in the policy (line 14). The security policy manager only accepts policies that include a valid signature issued with the corresponding private key owned by the interested party.

[0066] Apparatus 300 may be used to obtain machine learning code. In other words, apparatus 300 may be used to request and / or receive machine learning code. Apparatus 300 may receive machine learning code from a training code owner 306. Training code owner 306 may be a computing system distinct from apparatus 300. Alternatively, training code owner 306 may be part of apparatus 300. Training code owner 306 may provide the machine learning code to apparatus 300 for training the machine learning code.

[0067] The machine learning code may include machine learning steps for using supervised machine learning data pairs. The machine learning code may use input and ground truth pairs. The machine learning code may relate to any suitable machine learning context. For example, the machine learning code may operate on one or more of image data, text data, and numerical data. The machine learning code may include a neural network (NN). Specifically, the machine learning code may include a deep neural network (DNN). The machine learning code may include multiple NN layers, such that the machine learning code is a DNN. The machine learning code may include Python code.

[0068] The apparatus 300 can be used to obtain a training code policy 307. In other words, the apparatus 300 can be used to request and / or receive a training code policy 307. The apparatus 300 can receive the training code policy 307 from the training code owner 306. The training code owner 306 can provide the training code policy 307 to the apparatus 300 for training the machine learning code. Alternatively, the apparatus 300 can define or generate the training code policy 307.

[0069] Training code policy 307 may include restrictions on the use of machine learning code. Training code policy 307 may include information about intellectual property in machine learning code. Training code policy 307 may include information about encryption keys to decrypt machine learning code in training module 312.

[0070] Apparatus 300 may be configured to encrypt training data 301. Encryption of training data 301 may provide a first layer of protection, privacy, and / or confidentiality for training data 301. Apparatus 300 may be configured to encrypt training data 301 via a TEE. A module that receives training data 301 may be implemented on the TEE.

[0071] The apparatus 300 may be used to encrypt machine learning code. Encryption of machine learning code may provide a first layer of protection, privacy, and / or confidentiality for the machine learning code. The apparatus 300 may be used to encrypt the machine learning code via a trusted execution environment (TEE). The module that receives the machine learning code may be implemented on the TEE.

[0072] Apparatus 300 may be used to train machine learning code using input training data 301. Training of the machine learning code may be performed by a training module 312. Input training data 312 and the machine learning code may be loaded into training module 312. Prior to training, the integrity and freshness of input training data 301 may be verified. Training the machine learning code using input training data 301 may generate a trained machine learning model 302. The machine learning code may be trained through supervised learning based on input training data 301. Apparatus 300 may be used to encrypt the trained machine learning model 302. Encryption of the trained machine learning model 302 may provide a first layer of protection, privacy, and / or confidentiality for the trained machine learning model 302. Apparatus 300 may be used to encrypt the trained machine learning model 302 using a trusted execution environment (TEE). Training module 312, which receives the machine learning code, may be implemented on a TEE. Training module 312 may perform training within a TEE enclave.

[0073] Alternatively, the apparatus 300 may directly obtain the trained machine learning model 302. In other words, the apparatus 300 may be configured to request and / or receive the trained machine learning model 302. The apparatus 300 may obtain the trained machine learning model 302 from another computing device. The apparatus 300 may receive the machine learning model 302 from a machine learning model owner 308. The machine learning model owner 308 may be a computing system different from the apparatus 300. Alternatively, the machine learning model owner 308 may be part of the apparatus 300. The machine learning model owner 308 may provide the trained machine learning model 302 to the apparatus 300 for use.

[0074] The trained machine learning model 302 can be used to use supervised machine learning data pairs. The trained machine learning model 302 can use input and ground truth pairs. The trained machine learning model 302 can involve any suitable machine learning context. For example, the trained machine learning model 302 can work on one or more of image data, text data, and numerical data.

[0075] The apparatus 300 may be configured to obtain a machine learning model policy 309. The apparatus 300 may receive the machine learning model policy 309 from the machine learning model owner 308. The machine learning model owner 308 may provide the machine learning model policy 309 to the apparatus 300 for use with the trained machine learning model 302. Alternatively, the apparatus 300 may define or generate the machine learning model policy 309.

[0076] The machine learning model policy 309 may include restrictions on the use of the trained machine learning model 302. The machine learning model policy 309 may include information about the intellectual property in the trained machine learning model 302.

[0077] Figure 6 An exemplary listing 600 for a model ML policy is shown. Listing 600 shows an example of a privacy policy 309 for an output ML model. This policy 309 supports defining privacy assurance levels and balancing the privacy and utility / accuracy of the output model. Similar to the training code policy 307, it indicates the commands required to run the privacy check computation within the TEE enclave (line 3) and the cryptographic hash of the privacy check computation (line 7). It imports the training code owner policy 307 to obtain the key used to decrypt the output candidate model stored in the encrypted candidate model data body. The security policy manager 310 only provides the key for the privacy check computation after attestation has been performed, i.e., ensuring that the cryptographic hash of the computation remains unchanged. In the privacy section (lines 9-11), it defines the privacy guarantees of the output model, such as the differential privacy parameter (ε) and the privacy measure (AUC value).

[0078] To support stakeholders 304, 306, and 308 in performing ML training while maintaining the confidentiality of their IP (training data, code, and models), the Security Policy Manager module 310 serves as a root of trust. Stakeholders 304, 306, and 308 establish trust in this component using the remote attestation mechanism 311 provided by the TEE. A TEE, such as Intel Software Guard Extensions (SGX), ensures the confidentiality and integrity of processed code and data. After stakeholders 304, 306, and 308 ensure that the Security Policy Manager module 310 executes within the TEE, they submit security policies 305, 307, and 309 defining access controls for their encryption keys and IP. Each stakeholder's 304, 306, and 308 IP is encrypted with a different key, and the Security Policy Manager module 310 uses the security policy to determine who has access to which keys. Technically, the Security Policy Manager module 310 generates keys within the TEE and only sends them to authenticated computations executed within the TEE. Therefore, these keys are invisible to everyone.

[0079] Stakeholders 304, 306, 308 can strike a balance between privacy protection of input data and machine learning utility by defining the privacy metrics they wish to achieve in the ML model policy (e.g., differential privacy parameters, privacy leakage thresholds). The security policy manager module 310 can execute the calibration controller module 314, the privacy and quality checker module 315, and the privacy protection module 316 to follow the machine learning model policy, as described herein. The output model 302 or 303 can only be released if the output model 302 or 303 meets the privacy leakage threshold defined by the stakeholders.

[0080] Security policy manager module 310 can receive training data policy 305, training code policy 307, and / or machine learning model policy 309. Security policy manager module 310 can combine training data policy 305, training code policy 307, and / or machine learning model policy 309 into a single security policy. In this way, each of the protection, privacy, and / or confidentiality requirements of training data owner 304, training code owner 306, and / or machine learning model owner 308 can be considered and combined. During the use of trained machine learning model 302, the device can use a single security policy.

[0081] The security policy manager module 310 can output a single security policy to the attestation and policy enforcement module 311. The attestation and policy enforcement module 311 can use the single security policy to control the use of the trained machine learning model 302.

[0082] The attestation and policy enforcement module 311 can use a single security policy to control the training of the machine learning code. The attestation and policy enforcement module 311 can output the single security policy to the training module 312. The training module 312 can use the single security policy to control the training so that protection, privacy, and / or confidentiality requirements are maintained.

[0083] The attestation and policy enforcement module 311 can use a single security policy to control the use of the trained machine learning model 302. The attestation and policy enforcement module 311 can output the single security policy to the calibration controller 314. The calibration controller 314 can use the single security policy to control the use of the trained machine learning model 302 so that protection, privacy, and / or confidentiality requirements are maintained. The attestation and policy enforcement module 311 can verify the integrity of the machine learning computations.

[0084] The calibration controller module 314 can validate / enforce the output model 302 or 303 to comply with the policies defined by the stakeholders 304 , 306 , 308 by using two components: (1) a privacy and quality checking module 315 and (2) a privacy protection module 316 .

[0085] The apparatus 300 can be used to evaluate the trained machine learning model 302. The apparatus 300 can be used to evaluate the trained machine learning model 302 to establish a privacy leakage level of the input training data 301. In other words, the trained machine learning model 302 can be evaluated to determine the privacy level in the input training data 301. The calibration controller 314 can include a privacy and quality checker module 315. The privacy and quality checker module 315 can be used to evaluate the trained machine learning model 302 to establish a privacy leakage level of the input training data 301. Other modules may also be suitable for evaluation. The privacy and quality checker module 315 can receive the trained machine learning model 302 from the training module 312. The privacy and quality checker module 315 can receive a single security policy from the calibration controller 314. The privacy and quality checker module 315 can evaluate the trained machine learning model 302 using instructions from the single security policy.

[0086] The primary goal of the privacy and quality checker module 315 is to assess the privacy risk of the output model of the input training data 301. The privacy and quality checker module 315 can automatically check / assess the privacy risk and quality of the output model. One example of verifying the quality is using a test dataset. The privacy and quality checker module 315 can be used to determine the privacy leakage level of the input training data 301. In other words, the privacy and quality checker module 315 can determine the degree of privacy of the information in the input training data 301. The privacy and quality checker module 315 can then determine a value associated with the input training data 301, namely, the privacy leakage level.

[0087] The privacy and quality checker module 315 may receive a privacy leakage level threshold. The privacy leakage level threshold may be part of a single security policy received from the attestation and policy enforcement module 311. The privacy leakage level threshold may depend on the privacy requirements set by the training data owner 304 and information in the training data policy 305. The privacy leakage level threshold may be predefined. The privacy leakage level threshold may depend on the type of input training data 301. For example, private data (e.g., medical data) may have a higher privacy leakage level threshold than non-private data. The privacy and quality checker module 315 may compare the privacy leakage level of the input training data 301 with the privacy leakage level threshold. If the privacy leakage level of the input training data 301 is higher than the privacy leakage level threshold, the privacy and quality checker module 315 may determine that the input training data 301 is acceptable. If the privacy leakage level of the input training data 301 is lower than the privacy leakage level threshold, the privacy and quality checker module 315 may determine that the input training data 301 is unacceptable.

[0088] Privacy leakage levels and privacy leakage level thresholds can be implemented using differential privacy. Differential privacy is a mathematical / statistical concept that provides a method for quantifying the privacy of individuals in a dataset. At a high level, it can be expressed as follows, for example, as shown in Formula 1. Let M be a function that takes a dataset D as input and generates some output (for example, the results of data analysis). If for any two datasets D and D' that differ by only one element, the inequality in Formula 1 holds, then the function M is considered differentially private:

[0089] Pr[M(D)∈ S]≤ e ε *Pr[M (D ′ )∈ S] Formula 1

[0090] Here, ε is a parameter that determines the privacy level. Smaller values ​​of ε provide stronger privacy guarantees. Roughly speaking, this equation states that the probability that function M generates a given output S when applied to dataset D should not be significantly greater than the probability that the same output S is generated when applied to dataset D'. This ensures that individual data has minimal influence on the function's output, thus protecting individual privacy.

[0091] In the context of machine learning, differential privacy can be achieved by adding noise to the training dataset or directly to the model, minimizing the impact of the presence or absence of any individual data on the overall outcome of the machine learning training computation. This ensures that the output machine learning model does not reveal any sensitive information about the individual members of the training dataset. Differential privacy is used to protect the privacy of individuals when the data being trained is sensitive, such as medical records or personal information.

[0092] One approach to privacy checking involves performing a membership inference attack on the output model. This can be done using established algorithms to quantify the privacy risk of training data 301, thereby measuring the privacy risk of machine learning models through membership inference attacks. The Privacy Checker works by performing a membership inference attack on the output candidate model. It simulates attackers with varying levels of access and model knowledge. The considered attackers only have access to the model's predictions, loss values, and model parameters. For each simulated attack, the tool reports a risk score for all records in the input training data 301. These scores represent the attacker's belief that the record is part of the training dataset. The distribution of these scores for records in the training data is significantly different from that for records not in the training data. The attacker's success can be quantified using a ROC curve, which represents the balance between the attacker's false positive rate and true positive rate. True positives represent correctly identifying members present in the training data 301, while false positives represent identifying non-members as members in the training data 301. The area under the curve (AUC) quantifies the overall privacy risk posed by the candidate model to the data. Higher AUC values ​​indicate greater risk. Therefore, this value can support stakeholders in defining the desired privacy level for the output model.

[0093] The privacy and quality checker module 315 can be used to evaluate the trained machine learning model 302 by simulating attacks against the trained machine learning model 302. In other words, the privacy and quality checker module 315 can simulate virtual attacks against the trained machine learning model 302 to see what information can be determined from the input training data 302. Based on how much information can be determined from the input training data 302 from the attack, the level of privacy leakage of the input training data 301 can be determined.

[0094] The privacy and quality checker module 315 can be used to evaluate the trained machine learning model 302 by simulating one or more of model inversion, membership inference, or model extraction attacks against the trained machine learning model. The type of attack used in the evaluation can depend on the type of input training data 301, the type of machine learning code, and / or the type of trained machine learning model 302.

[0095] The privacy and quality checker module 315 can be used to evaluate the trained machine learning model 302 based on the probability that an attack correctly identifies the candidate training data as the input training data 301 for the trained machine learning model 302. In other words, if the attack cannot determine that the data in the machine learning model 302 is the input training data 301, this can indicate an acceptable level of privacy. If the attack cannot determine that the data in the machine learning model 302 is the input training data 301, this can indicate an unacceptable level of privacy. If the input training data 301 can be correctly identified, this can result in a security breach of the input training data 301. The level of privacy leakage can depend on the ease or ability to correctly identify the candidate training data 301.

[0096] The privacy and quality checker module 315 can be used to determine a privacy leakage level threshold based on a function between the privacy leakage level of the input training data 301 and the performance level of the machine learning model 302. In other words, the privacy leakage level threshold can be set based on a balance between the privacy and performance requirements of the trained machine learning model 302. The privacy and performance levels can be linked by a function. If the privacy level of the input training data 301 is increased, this may reduce the performance of the machine learning model 302. If the privacy level of the input training data 301 is reduced, this may increase the performance of the machine learning model 302. This is because increasing the privacy of the input training data 301 may make the input training data 301 more difficult to review, audit, analyze, and / or evaluate. As a result, the input training data 301 may be more difficult to use in the machine learning model 302, which may reduce the performance level. The privacy leakage level can be determined based on the optimal balance between the privacy leakage level of the input training data 301 and the performance level of the machine learning model 302.

[0097] The privacy and quality checker module 315 may be configured to receive keys (encryption / decryption keys) directly from the security policy manager module 310. The keys (encryption / decryption keys) may enable the privacy and quality checker module 315 to access information in the input training data 301, machine learning code, and machine learning models.

[0098] The apparatus 300 can be configured to output a trained machine learning model 302. The apparatus 300 can be configured to output the trained machine learning model 302 based on the privacy leakage level of the input training data 301 being lower than a privacy leakage level threshold. In other words, if the privacy and quality checker module 315 determines that the privacy leakage level of the input training data 301 is higher than the privacy leakage level threshold, the apparatus 300 can output the trained machine learning model 302. In this manner, if the privacy and quality checker module 315 determines that the privacy of the input training data 301 is acceptable, the apparatus 300 is enabled to output the trained machine learning model 302. If the privacy and quality checker module 315 determines that the privacy of the input training data 301 is unacceptable, the apparatus 300 is not enabled to output the trained machine learning model 302.

[0099] If the AUC value is greater than the value defined in the model policy, the privacy protection module 316 can activate a privacy protection mechanism (e.g., differential privacy). The privacy and quality checker module 315 can also measure the quality / utility of the model to ensure that the privacy protection mechanism does not significantly affect the quality of the output model 303.

[0100] Apparatus 300 may be configured to enhance the privacy of input training data 301 in a trained machine learning model 302. Apparatus 300 may be configured to enhance the privacy of input training data 301 in a trained machine learning model 302 based on a privacy leakage level of the input training data 301 being greater than a privacy leakage level threshold. In other words, if the privacy leakage level of the input training data 301 is greater than the privacy leakage level threshold, apparatus 300 may enhance the privacy of the input training data 301. Calibration controller 314 may include a privacy protection module 316. Privacy protection module 316 may be configured to enhance the privacy of input training data 301 in a trained machine learning model 302. Other modules may also be adapted to perform the enhancement. Privacy protection module 316 may receive the trained machine learning model 302 from training module 312 or from privacy and quality checker module 315. Privacy protection module 316 may receive a single security policy from calibration controller 314.

[0101] The privacy protection module 316 can use instructions from a single security policy to add privacy to the input training data 301. The privacy protection module 316 can be used to enhance the privacy of the input training data 301 in the trained machine learning model 302 to generate a privacy-enhanced trained machine learning model 303.

[0102] The privacy protection module 316 can be used to enhance the privacy of the input training data 301 in the machine learning model 302 by reducing the quality of the input training data 301. The quality of the input training data 301 can be reduced in any suitable manner.

[0103] One of the mechanisms provided in the privacy protection module 316 is to add noise to the output model 302 to protect the privacy of the input data 301. The privacy protection module 316 itself runs within the TEE enclave, and the security policy manager module 310 can attest to the privacy protection module 316 to ensure that no one has modified the privacy protection module 316 before executing it to run in accordance with the defined ML model policy. The noise can be of various types, helping to achieve differential privacy or even zero-knowledge privacy.

[0104] The privacy protection module 316 can be used to enhance the privacy of the input training data 301 in the machine learning model 302 by introducing noise into the trained machine learning model 302. In this way, the machine learning model 302 may be more difficult to analyze the data. Therefore, the privacy of the input training data 301 in the machine learning model 302 can be increased. Additionally or alternatively, the privacy protection module 316 can be used to enhance the privacy of the input training data 301 in the machine learning model 302 by introducing noise into the input training data 301. Any type of suitable noise can be used alone, in combination, or with different types of noise.

[0105] An example of noise introduced into machine learning model 302 and / or input training data can be differential privacy noise. Privacy protection module 316 can be configured to determine differential privacy noise based on a function between the level of privacy leakage in input training data 301 and the performance level of machine learning model 302. In other words, differential privacy noise can be set based on a balance between the privacy and performance requirements of trained machine learning model 302. The privacy level (affected by differential privacy noise) and performance can be linked by a function. If the privacy level in input training data 301 (affected by differential privacy noise) increases, this can reduce the performance of machine learning model 302. If the privacy level in input training data 301 (affected by differential privacy noise) decreases, this can increase the performance of machine learning model 302. This is because increasing the privacy of input training data 301 (affected by differential privacy noise) can make input training data 301 more difficult to review, examine, analyze, and / or evaluate. Consequently, input training data 301 may be more difficult to use in machine learning model 302, which can reduce performance. The differential privacy noise can be determined based on the optimal balance between the privacy leakage level of the input training data 301 and the performance level of the machine learning model 302.

[0106] Figure 4An exemplary algorithm 400 for adding privacy noise is shown. Algorithm 400 illustrates how to add noise to make the output ML model differentially private. The inputs to the algorithm include the ML model parameters W, the differential privacy parameters provided by the stakeholders, and the L2 sensitivity of the model λ. To make the ML model (ε, δ) differentially private, a centered Gaussian noise with a variance of at least σ^2 can be added, as defined in Equation 2:

[0107] W*∶=W+N(0,σ) Formula 2

[0108] in:

[0109]

[0110] In practice, the λ value is selected and tuned based on the input training dataset 301. Selecting a large λ may increase the likelihood that the ML model is differentially private, however, it may also reduce the performance of the ML model in terms of utility or accuracy. For adding a noise component, two possible hyperparameter λ selection methods are disclosed, as follows. The first method is to start with a very small λ value, i.e., λ=10-6, and sequentially increase λ by a factor of 2 until the model satisfies the privacy and quality checker module 315. The second method is to select a fixed λ value, such as λ=10-3, which can be obtained as a practical value through experimentation.

[0111] Another example of noise introduced into the machine learning model 302 and / or the input training data 301 can be zero-knowledge privacy noise introduced into the trained machine learning model 302. The zero-knowledge privacy noise can be set independently of the performance level of the trained machine learning model 302.

[0112] Another example of noise introduced into the machine learning model 302 may be randomizing the input training data 301 in the trained machine learning model 302. The randomization of the input training data 301 may be set independently of the performance level of the trained machine learning model 302. Randomizing the input data 301 may be suitable for binary data.

[0113] The privacy quality checker module 315, the calibration controller module 314, and the privacy protection module 316 can all run within the TEE enclave. The security policy manager component 310 can attest the privacy quality checker module 315, the calibration controller module 314, and the privacy protection module 316 to ensure that no one has modified them before executing them to run in accordance with the defined ML model policy.

[0114] The apparatus 300 can be configured to output a trained machine learning model 303 with enhanced privacy. The apparatus 300 can be configured to derive the trained machine learning model 303 with enhanced privacy. The privacy protection module 316 can return the trained machine learning model 303 with enhanced privacy to the privacy and quality checker 316 for review after performing privacy enhancement. The apparatus 300 can be configured to output the trained machine learning model 303 with enhanced privacy based on the privacy leakage level of the input training data 301 being greater than a privacy leakage level threshold. In other words, if the privacy and quality checker module 315 determines that the privacy leakage level of the input training data 301 is less than the privacy leakage level threshold, the apparatus 300 can output the trained machine learning model 303 with enhanced privacy. In this manner, if the privacy and quality checker module 315 determines that the privacy of the input training data 301 is acceptable, the apparatus 300 is enabled to output the trained machine learning model 303 with enhanced privacy. If the privacy and quality checker module 315 determines that the privacy of the input training data 301 is unacceptable, the device 300 will not be supported to output the privacy-enhanced trained machine learning model 303. In this case, the privacy-enhanced trained machine learning model 303 can be input into the privacy protection module 316 again, and the privacy of the input training data 301 can be enhanced again. The process between the privacy and quality checker module 315 and the privacy protection module 316 can be iterated until the desired privacy level of the input training data 301 is reached. Once the privacy leakage level threshold is met, the output model 303 can be output.

[0115] The apparatus 300 can be used to execute the output trained machine learning model 302. The apparatus 300 can be used to execute the output privacy-enhanced trained machine learning model 303. Alternatively, the apparatus 300 can output the trained machine learning model 302, and a different computing system can execute the trained machine learning model 302. Similarly, the apparatus 300 can output the privacy-enhanced trained machine learning model 303, and a different computing system can execute the privacy-enhanced trained machine learning model 303.

[0116] Figure 7 The exemplary apparatus of the second embodiment is schematically shown. The apparatus 700 may include: Figure 7 Pipeline shown.

[0117] The apparatus 700 can be used for confidential federated learning. Federated learning (FL) is a machine learning technique that enables participating clients to collaboratively train a joint global machine learning model without sharing their local training data 701. FL can reduce the privacy risk of local training data 701. The idea behind FL is that each participating client 706 trains locally with its data 701, instead of sharing the training data 701 to a centralized training system deployed in an untrusted environment (such as a public cloud). For each training iteration, participating clients 706 can send their local parameters to the central system to train a global model 702, which will benefit from all contributions from the clients. Basically, the central system can aggregate the local model parameters provided by the clients 704 and send the aggregated parameters back to the clients 706. The training process continues until the global model 702 reaches a certain expected accuracy. In this setting, since the aggregated model parameters are sent to all clients 706, it is vulnerable to membership inference attacks, as described in this article. Figure 1 To address this problem, this paper discusses the traditional machine learning described in Figure 3 The described mechanisms and methods can be applied in federated learning settings. Typically, privacy protection mechanisms (privacy checker 715 and noise addition mechanism 716) can be applied on the client side before sending its local model parameters, or applied in the model aggregation before sending the global parameters back to all clients 706. To ensure fairness between clients (e.g., adding the same differential privacy noise), applying privacy protection mechanisms in the model aggregation is a better approach.

[0118] Device 700 may be implemented in a trusted execution environment (TEE). The TEE may provide a first layer of protection, privacy, and / or confidentiality for device 700. All of device 700 may be implemented in a TEE. Alternatively, specific portions of device 700 may be implemented in a TEE. For example, portions of device 700 that require a higher level of protection may be implemented in a TEE.

[0119] Figure 7 This article shows Figure 3 The described system can be applied in a federated learning setting. Clients 706 can jointly collaborate to define privacy parameters in an ML model policy 709. After attesting to the security policy manager 710, they upload the policy to the security policy manager 710. The security policy manager 710 then acts as a root of trust and performs model convergence to adhere to the policy, i.e., at each training iteration, the global model parameters are checked to see if they meet the requirements defined in the policy before being shared with clients 706.

[0120] The apparatus 700 may be used to obtain local input training data 701a, 701b, 701c. In other words, the apparatus 700 may be used to request and / or receive local input training data 701a, 701b, 701c. The apparatus 300 may receive local input training data 701a, 701b, 701c from a plurality of client owners 704a, 704b, 704c. Figure 7 In the embodiment of the present invention, three client owners 704a, 704b, 704c are shown, but any number of client owners 704a, 704b, 704c may be used. The client owners 704a, 704b, 704c may be computing systems distinct from the apparatus 700. Alternatively, the client owners 704a, 704b, 704c may be part of the apparatus 700. The client owners 704a, 704b, 704c may provide local input training data 701a, 701b, 701c to the apparatus 700 for training the machine learning code.

[0121] The local input training data 701a, 701b, 701c may include supervised machine learning data pairs. The local input training data 701a, 701b, 701c may include input and ground truth pairs. The local input training data 701a, 701b, 701c may relate to any suitable machine learning data. For example, the local input training data 701a, 701b, 701c may include one or more of image data, text data, and numerical data.

[0122] Apparatus 700 can be used to obtain local machine learning code. In other words, apparatus 700 can be used to request and / or receive local machine learning code. Apparatus 700 can receive machine learning code from client owners 704a, 704b, 704c. Client owners 704a, 704b, 704c can provide the local machine learning code to apparatus 300 for training the local machine learning code. There can be a corresponding pair of local machine learning code and local input training data 701a, 701b, 701c.

[0123] The local machine learning code may include machine learning steps for using supervised machine learning data pairs. The local machine learning code may use input and ground truth pairs. The local machine learning code may involve any suitable machine learning context. For example, the local machine learning code may work on one or more of image data, text data, and numerical data. The local machine learning code may include a neural network (NN). Specifically, the local machine learning code may include a deep neural network (DNN). The local machine learning code may include multiple NN layers, so that the local machine learning code is a DNN. The local machine learning code may include Python code.

[0124] The apparatus 700 can be configured to train each local machine learning code using corresponding local input training data 701a, 701b, 701c. The training of the machine learning code can be performed by a training module at a client owner 704a, 704b, 704c. Training the machine learning code using the local input training data 701a, 701b, 701c can generate a corresponding trained machine learning model. The machine learning code can be trained using supervised learning based on the local input training data 701a, 701b, 701c.

[0125] The apparatus 700 may be configured to encrypt the corresponding trained machine learning model. Encryption of the corresponding trained machine learning model may provide a first layer of protection, privacy, and / or confidentiality for the corresponding trained machine learning model. The apparatus 700 may be configured to encrypt the corresponding trained machine learning model via a trusted execution environment (TEE). A training module 712 that receives machine learning code may be implemented on the TEE.

[0126] The apparatus 700 may also be used to encrypt local input training data 701a, 701b, 701c and / or local machine learning code.

[0127] The apparatus 700 may be configured to aggregate the trained local machine learning models. The apparatus 700 may be configured to aggregate the trained local machine learning models to generate a trained machine learning model 702. The trained machine learning model may be referred to as a global machine learning model 702. The global machine learning model 702 is a combination of the local machine learning models. The training module 712 may be configured to aggregate the trained local machine learning models to generate a global trained machine learning model 702.

[0128] Alternatively, the apparatus 700 may directly obtain the globally trained machine learning model 702. In other words, the apparatus 700 may be configured to request and / or receive the globally trained machine learning model 702. The apparatus 700 may obtain the globally trained machine learning model 702 from another computing device. The apparatus 700 may receive the globally trained machine learning model 702 from a machine learning model owner. The machine learning model owner may be a computing system different from the apparatus 700. Alternatively, the machine learning model owner may be part of the apparatus 700. The machine learning model owner may provide the globally trained machine learning model 702 to the apparatus 700 for use.

[0129] The global training machine learning model 702 can be used to use supervised machine learning data pairs. The global training machine learning model 702 can use input and ground truth pairs. The global training machine learning model 702 can involve any suitable machine learning context. For example, the global training machine learning model 702 can work on one or more of image data, text data, and numerical data.

[0130] The apparatus 700 may be configured to obtain a machine learning model policy 709. The apparatus 700 may receive the machine learning model policy 709 from each of the client owners 704a, 704b, and 704c. The client owners 704a, 704b, and 704c may provide the machine learning model policy 709 to the apparatus 700 for use with the trained machine learning model 702.

[0131] The machine learning model policy 709 may include restrictions on the use of the corresponding trained local machine learning model. The machine learning model policy 709 may include information about intellectual property rights in the corresponding trained local machine learning model.

[0132] The machine learning model policy 709 may be received by the security policy manager module 710. The training data policy 305 and the training code policy 307 may also be received by the security policy manager 710, as described herein with respect to Figure 3 As described above, the security policy manager module 710 can combine the training data policy 305, the training code policy 307, and / or the machine learning policy 709 into a single security policy. In this way, each protection, privacy, and / or confidentiality requirement of each client owner 704a, 704b, 704c can be considered and combined. During the use of the global training machine learning model 702, the device 700 can use a single security policy.

[0133] The security policy manager module 710 can output a single security policy to the attestation and policy enforcement module 711. The attestation and policy enforcement module 711 can use the single security policy to control the use of the global trained machine learning model 702.

[0134] The attestation and policy enforcement module 711 can use a single security policy to control the training of the machine learning code. The attestation and policy enforcement module 711 can output the single security policy to the training module 712. The training module 712 can use the single security policy to control the training so that protection, privacy, and / or confidentiality requirements are maintained.

[0135] The attestation and policy enforcement module 711 can use a single security policy to control the use of the global trained machine learning model 702. The attestation and policy enforcement module 711 can output the single security policy to the privacy and quality checker module 715 and the privacy protection module 716. The privacy and quality checker module 715 and the privacy protection module 716 can use a single security policy to control the use of the global trained machine learning model 702 such that protection, privacy, and / or confidentiality requirements are maintained.

[0136] This article about Figure 3 The evaluation of the trained machine learning model 302 described herein is applied to Figure 7 Evaluation of the global trained machine learning model 702 described herein. Figure 3 The output of the trained machine learning model 302 described herein is used for Figure 7 The output of the global training machine learning model 702 described herein. Figure 3 The privacy enhancements described for training machine learning models 302 are applicable to the privacy enhancements described herein. Figure 7 Privacy enhancement of the global training machine learning model 702 described herein. Figure 3 The output of the privacy-enhancing trained machine learning model 303 described herein is used for Figure 7 Evaluation of the privacy-enhanced globally trained machine learning model 703 described.

[0137] Figure 8 An example of a method 800 for confidential multi-stakeholder machine learning is summarized. At step 801, method 800 includes obtaining a trained machine learning model, obtained by executing machine learning code trained on input training data. At step 802, method 800 includes evaluating the machine learning model to establish a privacy leakage level of the input training data. At step 803, method 800 includes outputting the trained machine learning model if the privacy leakage level of the input training data is below a privacy leakage level threshold.

[0138] Figure 9 An example of an apparatus 900 for implementing the method 800 is schematically shown. The computing apparatus 900 may include the apparatus 300. The apparatus 900 may be implemented on an electronic device such as a laptop, a tablet computer, a smartphone, or a television.

[0139] The apparatus 900 includes a processor 901 for processing a data set in the manner described herein. For example, the processor 901 may be implemented as a computer program running on a programmable device such as a central processing unit (CPU). The apparatus 900 includes a memory 902 for communicating with the processor 901. The memory 902 may be a non-volatile memory. The processor 901 may also include a cache ( Figure 9 (not shown) which can be used to temporarily store data from memory 902. The apparatus may include one or more processors and one or more memories. The memories may store data executable by the processors. The processors may be configured to operate according to a computer program stored in a non-transitory form on a machine-readable storage medium. The computer program may store instructions for causing the processors to perform the methods described herein.

[0140] Applicants hereby disclose individually each individual feature described herein, as well as any combination of two or more such features, to the extent such features or combinations can be implemented according to the common general knowledge of those skilled in the art based on the specification as a whole, whether or not such features or combinations of features solve any problem disclosed herein, and without limiting the scope of the claims. Applicants indicate that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description, it will be apparent to those skilled in the art that various modifications may be made within the scope of the present invention.

Claims

1. A computing device (900) for confidential multi-stakeholder machine learning, characterized in that The computing device (900) comprises one or more processors (901) and a memory (902), the memory (902) storing data defining program code executable by the one or more processors (901) in a non-transitory form, wherein the program code is executable by the one or more processors (901) such that the computing device (900) is configured to: Obtaining a trained machine learning model (302), the trained machine learning model (302) obtained by executing machine learning code trained on input training data (301); evaluating the trained machine learning model (302) to establish a privacy leakage level of the input training data (301); When the privacy leakage level of the input training data (301) is lower than a privacy leakage level threshold, the trained machine learning model (302) is output.

2. The computing device (900) according to claim 1, characterized in that The computing device (900) is used to: enhancing the privacy of the input training data (301) in the trained machine learning model (302) when the privacy leakage level of the input training data (301) is higher than the privacy leakage level threshold; Outputting a privacy-enhanced trained machine learning model (303).

3. The computing device (900) according to claim 1 or 2, characterized in that The apparatus (900) is configured to evaluate the trained machine learning model (302) by simulating attacks against the trained machine learning model (302).

4. The computing device (900) according to any one of the preceding claims, characterized in that The apparatus (900) is configured to evaluate the trained machine learning model (302) by simulating one or more of a model inversion, membership inference, or model extraction attack against the trained machine learning model (302).

5. The computing device (900) according to claim 3 or 4, characterized in that The apparatus (900) is configured to evaluate the trained machine learning model (302) based on a probability that the attack correctly identifies candidate training data as input training data (301) to the trained machine learning model (302).

6. The computing device (900) according to any one of the preceding claims, characterized in that The device (900) is used to determine the privacy leakage level threshold based on a function between the privacy leakage level of the input training data (301) and the performance level of the machine learning model (302).

7. The computing device (900) according to any one of claims 2 to 6, characterized in that: The apparatus (900) is configured to enhance the privacy of the input training data (301) in the trained machine learning model (302) by reducing the quality of the input training data (301).

8. The computing device (900) according to any one of claims 2 to 7, characterized in that The apparatus (900) is used to enhance the privacy of the input training data (301) in the trained machine learning model (302) by introducing noise into the trained machine learning model (302).

9. The computing device (900) according to claim 8, characterized in that The apparatus (900) is configured to introduce differentially private noise into the trained machine learning model (302).

10. The computing device (900) according to claim 9, characterized in that The device (900) is used to determine the differential privacy noise based on the function between the privacy leakage level of the input training data (301) and the performance level of the trained machine learning model (302).

11. The computing device (900) according to any one of claims 8 to 10, characterized in that: The apparatus (900) is configured to introduce zero-knowledge privacy noise into the trained machine learning model (302).

12. The computing device (900) according to any one of claims 2 to 6, characterized in that: The apparatus (900) is used to enhance the privacy of the input training data (301) in the trained machine learning model (302) by randomizing the input training data (301).

13. The computing device (900) according to any one of the preceding claims, characterized in that The device (900) is used to: Obtaining the input training data (301) and the machine learning code; Encrypting the input training data (301) and the machine learning code; The machine learning code is trained with the input training data (301) to generate the trained machine learning model (302), wherein the trained machine learning model (302) is encrypted.

14. The computing device (900) according to any one of claims 1 to 11, characterized in that: The device (900) is used to: Obtain multiple pairs of local input training data (701a, 701b, 701c) and corresponding local machine learning codes; training each local machine learning code using its respective local input training data (701a, 701b, 701c) to generate a corresponding trained local machine learning model; encrypting the corresponding trained local machine learning model; The corresponding trained local machine learning models are aggregated to generate a trained machine learning model (702), which is a global trained machine learning model (702).

15. The computing device (900) according to claim 13 or 14, characterized in that The device (900) is used to perform encryption through a trusted execution environment (TEE).

16. A method (800) for confidential multi-stakeholder machine learning, characterized in that The method (800) comprises the following steps: Obtaining a trained machine learning model, the trained machine learning model being obtained by executing machine learning code trained on input training data (801); evaluating the trained machine learning model to establish a privacy leakage level of the input training data (802); When the privacy leakage level of the input training data is lower than a privacy leakage level threshold, the trained machine learning model is output (803).

Images