Optimized use of privacy budget

Abstract

The present disclosure relates to maintaining privacy of private data, embodiments of the present disclosure provide a method that includes determining, in a trusted execution environment, values of hyper-parameters of a machine learning model based on private data stored in the trusted execution environment, where the hyper-parameters include: system-specific hyper-parameters and model-specific hyper-parameters. Training a machine learning model in the trusted execution environment, the determined values of the system-specific and model-specific hyper-parameters being applied to the machine learning model to obtain a fully trained machine learning model after one or more periods of training; outputting a fully trained machine learning model from the trusted execution environment; and prohibiting output of the determined value of the system-specific hyper-parameter from the trusted execution environment; wherein the system-specific hyper-parameter is inaccessible in the output fully trained machine learning model.

Description

Technical Field

[0001] This disclosure relates to the protection of privacy regarding private data. abbreviation CCPA (California Consumer Privacy Act) CPU (Central Processing Unit) DP Differential Privacy DP-SGD Differential Privacy Stochastic Gradient Descent FL Federal Learning GDPR (General Data Protection Regulation) ML Machine Learning OS operating system SGX Security Extension TEE Trusted Execution Environment Background Technology

[0002] Machine learning. Typically, machine learning (ML) requires a framework in which large datasets can be used. To increase the amount of data, different entities can collaborate to collectively produce large datasets. This collaboration requires different entities to have guarantees that their confidential information (i.e., the data) will not be publicly disclosed (i.e., data privacy is maintained).

[0003] Data privacy. Data privacy refers to keeping records used as part of training confidential. Confidential data can include, but is not limited to, medical records, credit card information, personal email addresses, and measurements from sensors and industrial processes. These records may need to be kept private / confidential for a variety of reasons: - Economic benefits: The process of data collection and processing can be expensive and represents a costly investment, and leaking the information could mean financial loss. - Contractual requirements: Companies that process data from client companies or private consumers must fulfill their contractual obligations to keep such data private. - Regulatory requirements: Regulations such as GDPR and CCPA require the protection of consumer data privacy. Summary of the Invention

[0004] The purpose of this invention is to improve upon existing technologies.

[0005] According to a first aspect, an apparatus is provided, comprising: means for determining values ​​of hyperparameters of a machine learning model in a trusted execution environment based on private data stored in the trusted execution environment, wherein the hyperparameters include system-specific hyperparameters and model-specific hyperparameters; means for training the machine learning model in the trusted execution environment, wherein the determined values ​​of the system-specific and model-specific hyperparameters are applied to the machine learning model to obtain a fully trained machine learning model after one or more training epochs; means for outputting the fully trained machine learning model from the trusted execution environment; and means for preventing the output of the determined values ​​of the system-specific hyperparameters from the trusted execution environment, wherein the system-specific hyperparameters are inaccessible in the output fully trained machine learning model.

[0006] The apparatus may further include: a component for calculating the total value of the privacy budget by adding the privacy budget spent for determining the value of the hyperparameters to the privacy budget spent for one or more epochs of training; wherein the value of the privacy budget spent for determining the value of the system-specific hyperparameters is 0.

[0007] The component used for output can be configured to additionally output the total value of the privacy budget from the trusted execution environment.

[0008] The apparatus may further include: components for estimating an estimate of the privacy budget to be spent for another period of training; components for calculating an estimated total value of the privacy budget by adding the total value of the privacy budget spent so far to the estimated value of the privacy budget to be spent for another period of training; and components for checking whether the estimated total value of the privacy budget exceeds the maximum privacy budget, wherein the components for output are configured to output a trained machine learning model obtained by the period of training performed without the other period of training as a fully trained machine learning model if the estimated total value of the privacy budget exceeds the maximum privacy budget.

[0009] The component used to determine the value of the hyperparameter can be configured to apply a privacy protection mechanism to private data and determine the value of the hyperparameter based on the private data to which the privacy protection mechanism is applied.

[0010] The components used to train machine learning models can be configured to apply privacy protection mechanisms to the data used for training.

[0011] Privacy protection mechanisms can include differential privacy.

[0012] At least one of the hyperparameters can be configured with a privacy protection mechanism.

[0013] At least one of the hyperparameters in the configuration of the privacy protection mechanism can be a noise level or a clipping boundary, and the noise level is to be added to at least one of the following: private data or data used for training.

[0014] The noise level can be system-specific. The clipping boundary can be model-specific.

[0015] At least one of the hyperparameters can configure the training of a machine learning model.

[0016] At least one of the hyperparameters configured for training a machine learning model can be: the batch size of the data used in training, or the learning rate for training.

[0017] Batch size can be system-specific. Learning rate can be model-specific.

[0018] The component used for training can be configured to use at least a subset of private data for training.

[0019] The device may also include components for deleting or encrypting system-specific hyperparameters when a fully trained machine learning model is acquired.

[0020] The device may also include a component for determining whether one of the hyperparameters is system-specific.

[0021] A trusted execution environment can be configured to ensure that the code, hyperparameters, and private data representing the device are protected within the trusted execution environment with respect to confidentiality and integrity.

[0022] According to a second aspect, a method is provided, comprising: determining values ​​of hyperparameters of a machine learning model in a trusted execution environment based on private data stored in the trusted execution environment, wherein the hyperparameters include system-specific hyperparameters and model-specific hyperparameters; training the machine learning model in the trusted execution environment, wherein the determined values ​​of the system-specific and model-specific hyperparameters are applied to the machine learning model to obtain a fully trained machine learning model after one or more training epochs; outputting the fully trained machine learning model from the trusted execution environment; and prohibiting the output of the determined values ​​of the system-specific hyperparameters from the trusted execution environment, wherein the system-specific hyperparameters are inaccessible in the output fully trained machine learning model.

[0023] The method may further include: calculating a total privacy budget by adding the privacy budget spent for determining the values ​​of hyperparameters to the privacy budget spent for one or more epochs of training; wherein the privacy budget spent for determining the values ​​of system-specific hyperparameters is 0.

[0024] The method may also include: outputting the total value of the privacy budget from the trusted execution environment.

[0025] The method may further include: estimating an estimate of the privacy budget to be spent for another period of training; calculating an estimated total privacy budget by adding the total privacy budget spent so far to the estimated privacy budget to be spent for another period of training; checking whether the estimated total privacy budget exceeds the maximum budget, and if the estimated total privacy budget exceeds the maximum privacy budget, outputting the trained machine learning model obtained by the training period performed without the other training period as a fully trained machine learning model.

[0026] Determining the value of a hyperparameter may include applying a privacy protection mechanism to private data and determining the value of the hyperparameter based on the private data to which the privacy protection mechanism is applied.

[0027] Training machine learning models can include applying privacy protection mechanisms to the data used for training.

[0028] Privacy protection mechanisms can include differential privacy.

[0029] At least one of the hyperparameters can be configured with a privacy protection mechanism.

[0030] At least one of the hyperparameters in the configuration of the privacy protection mechanism can be a noise level or a clipping boundary, and the noise level is to be added to at least one of the following: private data or data used for training.

[0031] The noise level can be system-specific. The clipping boundary can be model-specific.

[0032] At least one of the hyperparameters can configure the training of a machine learning model.

[0033] At least one of the hyperparameters configured for training a machine learning model can be: the batch size of the data used in training, or the learning rate for training.

[0034] Batch size can be system-specific. Learning rate can be model-specific.

[0035] Training may include using at least a subset of private data for training.

[0036] The method may also include: deleting or encrypting system-specific hyperparameters when a fully trained machine learning model is acquired.

[0037] The method may also include: determining whether one of the hyperparameters is system-specific.

[0038] A trusted execution environment can be configured to ensure that code representing methods, hyperparameters, and private data are protected within the trusted execution environment in terms of confidentiality and integrity.

[0039] According to a third aspect, a computer program product including an instruction set is provided, which, when executed on a device, is configured to cause the device to perform the method according to the second aspect. The computer program product may be embodied in a computer-readable medium or may be directly loaded into a computer.

[0040] According to some example embodiments, at least one of the following advantages can be achieved: Data privacy can be protected; Given a total privacy budget, the model-specific hyperparameters can be determined, and the privacy budget used for training can be increased. For the same number of training periods, the privacy budget can be smaller than usual. Attached Figure Description

[0041] Further details, features, objectives, and advantages will be apparent from the following detailed description of preferred exemplary embodiments, taken in conjunction with the accompanying drawings, wherein:

[0042] Figure 1 The illustration shows a privacy budget (left), a regular segmentation of the privacy budget (middle), and a segmentation of the privacy budget according to some example embodiments.

[0043] Figure 2 An overview of a system according to some example embodiments is shown;

[0044] Figure 3 The illustrations depict methods according to some example embodiments;

[0045] Figure 4 The illustration shows the use of a temporary environment for determining the value of hyperparameters according to some example embodiments.

[0046] Figure 5 An apparatus according to an example embodiment is shown;

[0047] Figure 6 A method according to an example embodiment is shown; and

[0048] Figure 7 An apparatus according to an example embodiment is shown. Detailed Implementation

[0049] In the following description, certain exemplary embodiments are illustrated with reference to the accompanying drawings, wherein features of the exemplary embodiments may be freely combined with each other unless otherwise described. However, it should be clearly understood that the description of certain exemplary embodiments is given by way of example only and is in no way intended to be construed as limiting this disclosure to the details disclosed.

[0050] Furthermore, it should be understood that the apparatus is configured to perform the corresponding method, although in some cases only the apparatus or only the method is described.

[0051] Differential privacy (DP) is a technique used to publicly share information about a dataset while preserving information about the individuals within it. The shared information represents the general patterns of the dataset, not about individual points. In the context of machine learning, DP exposes (either during training or through the final model) general trends about the training data without revealing whether any particular data point was used to train the model.

[0052] Differential privacy (DP) is a scheme used to provide privacy while sharing information about a group of individuals by describing patterns within the group while retaining information about specific individuals. This is accomplished by making small, random changes to individual data that do not (or hardly) alter the statistics of interest. Therefore, the data cannot be used to infer much about any individual. Another way to describe differential privacy is as a constraint on algorithms used to publish aggregated information about a statistical database that restricts the disclosure of private information about the records in the database. Roughly speaking, an algorithm is differentially private if an observer seeing its output cannot discern whether information about a specific individual was used in the computation.

[0053] The concept of ε-differential privacy provides a mathematical definition of the privacy loss associated with the publication of any data extracted from a statistical database. Here, the term statistical database means a set of data collected with confidentiality guaranteed, for the purpose of not compromising the privacy of the individuals providing the data.

[0054] The intuition behind the definition of ε-differential privacy is that an individual's privacy cannot be compromised by statistical publication if their data is not in a database. Therefore, the goal of differential privacy is to give each individual roughly the same privacy that would result from the removal of their data. That is, statistical functions running on the database should not be unduly dependent on the data of any single individual.

[0055] The level of ε-differential privacy is defined by the value of ε. The larger the value of ε, the higher the risk of privacy breach. In some cases, a second parameter δ exists so that (ε, δ) defines the level of privacy. However, δ is typically very small (e.g., 10). -5This can generally be ignored for the purposes of this application. However, this application can also be applied to cases involving (ε, δ) privacy.

[0056] The training of a neural network that provides (ε, δ) privacy or ε-privacy is performed such that random values ​​are added to each piece of training data in the training dataset before the training data is input into the neural network. The training data is labeled, i.e., the ground truth is indicated.

[0057] DP Limitation Privacy leak operation How can it be done? In DP, privacy is represented by two values, usually called... - Privacy. Those values ​​were used to calculate Privacy leaks Privacy breach refers to the likelihood that an attacker can "guess" private data points. Privacy breaches increase when any calculations are performed on private data; that is, the likelihood of guessing the data increases.

[0058] Typically, privacy requirements are defined as Privacy Budget In other words, the upper limit for privacy breaches. Once the breach reaches the predefined privacy budget, privacy breach calculations should no longer be performed to prevent the privacy of some data from being violated with at least some probability.

[0059] Hyperparameter search for ML models. This involves, independently of the framework, determining how to tune the model during training. Training hyperparameters number Some decisions regarding the value of these hyperparameters must be made. These hyperparameters include, but are not limited to, […]. Learning rate , Batch size ,as well as Training damage lose The correct tuning of hyperparameter values ​​can be relevant to ensuring that the model converges to a state that produces satisfactory results, and how quickly that state will be reached.

[0060] In addition to tuning the values ​​of training hyperparameters, the use of dynamic programming (DP) in the context of machine learning requires additional privacy hyperparameters. These privacy hyperparameters must be tuned to achieve good convergence of model training and efficient use of the allocated privacy budget, similar to tuning the training hyperparameters. Privacy hyperparameters include, but are not limited to, the amount of noise to be added, the subsampling rate of data elements, and gradient clipping boundaries.

[0061] Trusted Execution Environment (TEE). A TEE is a secure zone on the main processor. It guarantees confidentiality and integrity; code and data loaded into the TEE are protected. The TEE provides applications with isolated execution and its integrity, as well as the integrity and confidentiality of their assets. Generally, the TEE provides an execution space that offers trusted applications a higher level of security than a rich operating system (OS) and more functionality than a 'secure element'.

[0062] TEE ensures the integrity of code running on the chip by providing hardware-based guarantees that the code matches publicly available, agreed-upon programs. Because the program must be trustworthy, the source code can be assumed to be available to all parties authorized to verify the TEE without any loss of generality. Although the program itself is public, the input it receives remains private to both the OS and external attackers.

[0063] TEE is integrated into several processor platforms from chip manufacturers such as Intel and ARM. The most popular commercially available TEE solution is Intel's Software Guard Extensions (SGX). SGC is a set of security-related instruction code built on modern Intel CPUs. They allow user-level and operating system code to define private regions of memory (called enclaves) whose contents are protected and cannot be read or saved by any process outside the enclave itself, but can be programmatically verified by a remote party.

[0064] In this application, terms used in SGX (such as "enclave") are used, but embodiments are not limited to SGX. It is guaranteed that the code, hyperparameters, and other execution environment representing the device are protected with respect to confidentiality within the TEE, and that integrity can be used, not in accordance with SGX, or in any other way than SGX.

[0065] Some example implementations improve the use of limited privacy budgets, specifically in the context of privacy-preserving mechanisms such as DP or k-anonymity. More specifically, in some example implementations, the portion of the privacy budget used to determine the value of hyperparameters based on privacy data is reduced. For this purpose, a TEE-based learning environment is used.

[0066] If the entity performing the tuning of hyperparameter values ​​and / or model training is different from the entity providing the private data (otherwise, privacy breaches within the data owner's own domain are not a problem), a TEE can be used to guarantee data confidentiality and prevent direct access by unauthorized parties: the data is decrypted within the TEE, used to train the model, and then the trained model is released while the data remains confidential. Examples of TEEs are confidential virtual machines (such as Intel's TDX (Trusted Domain Extension) or AMD's SEV-SNP (Secure Encrypted Virtualization - Secure Nested Paging)) and application enclaves (such as Intel's SGX (Software Protected Extension) or ARM TrustZone). However, the trained model must include some fingerprint of the private data so that privacy can still be violated. That is, some privacy budget is spent on determining hyperparameter values ​​and / or training the model based on the private data. Specifically, if the hyperparameters are determined based on the private data, a portion of the privacy budget can be spent on determining the hyperparameter values, and if training is performed based on the private data, a portion of the privacy budget can be spent on each training epoch.

[0067] This problem stems from the following characteristics of dynamic programming (DP): Any determination based on private data that results in a breach of the privacy of that data lose While the sensitivity of a decision (or the query that leads to that decision) can be small in optimistic cases, it will never be zero if private data influences the outcome. The same applies to other privacy protection mechanisms.

[0068] Figure 1 The diagram illustrates how a given privacy budget (shown on the left) can be conventionally partitioned (shown in the middle) and partitioned according to some example embodiments (shown on the right). Conventionally, the privacy budget is used to tune (determine) the values ​​of hyperparameters (training hyperparameters and privacy hyperparameters) and for model training, where the determined hyperparameter values ​​are applied to the model. If both determining the hyperparameter values ​​and model training are based on private data, each of these actions incurs some privacy budget.

[0069] Hyperparameters can be categorized into four distinct classes, as shown in Table 1. From a functional perspective (horizontal axis in Table 1), there are training hyperparameters relevant to training without the application of privacy protection mechanisms, and privacy hyperparameters relevant to privacy protection mechanisms such as Discrete Perception (DP). From a visibility perspective (vertical axis in Table 1), there are system-specific hyperparameters that are not visible in the trained ML model (and therefore inaccessible from the trained ML model), and model-specific hyperparameters that are visible in the trained ML model (and therefore accessible from the trained ML model). System-specific hyperparameters are used only by the system and are known only to the system.

[0070] Therefore, it exists: System-specific training hyperparameters; Model-specific training hyperparameters; System-specific privacy hyperparameters; and Model-specific privacy hyperparameters.

[0071] Table 1 shows typical examples for each of these types of hyperparameters. Table 1: Types and Examples of Hyperparameters

[0072] Batch size, noise level, learning rate, and clipping boundaries are typically classified as hyperparameters of the corresponding type for the following reasons: Batch size: The number of data elements processed in each training round. It is system-specific if and only if the model receives the average for that batch (as is usually the case). Noise level: The noise value added by the DP mechanism is sampled from the variance of this probability distribution. Because the model typically does not know how much of the received updates is noise and how much is true, this hyperparameter is usually system-specific. Learning rate: A hyperparameter used to determine the magnitude of updates applied to the model. It directly affects how model updates are applied and can be controlled by the learning algorithm, so it is generally expected to be visible to the model and / or the learning algorithm, and therefore model-specific. Clipping boundary: The maximum magnitude of the gradient used for model updates. Although not directly visible to the model, this value can often be guessed by the model by simply observing the maximum value received in the model update, and is therefore usually model-specific.

[0073] Under normal circumstances ( Figure 1 In the middle), based on private data, the values ​​of both system-specific hyperparameters and model-specific hyperparameters are adjusted (determined), incurring some privacy budget. Figure 1 The text refers to "system-specific tuning budget" and "model-specific tuning budget". In some example embodiments, the values ​​of system-specific hyperparameters are determined (adjusted) without incurring any privacy budget. That is, the system-specific tuning budget is 0, as shown in the example. Figure 1 The right side is shown. Therefore, the model-specific tuning budget and / or model training budget can be increased.

[0074] Because the privacy budget portion used to determine hyperparameter values ​​is reduced compared to the normal case (since the system-specific tuning budget is 0), given a fixed and finite privacy budget, some example embodiments increase the number of training epochs the system can use to train the model specifically based on private data to improve its performance and accuracy. Alternatively, for the same number of training epochs as in the normal case, in some example embodiments, the privacy budget used can be reduced so that privacy can be improved even if the trained ML model has the same performance and accuracy as in the normal case.

[0075] In some example embodiments, the system-specific tuning budget is 0 because both the tuning of hyperparameter values ​​and the training of the ML model for applying the tuned hyperparameter values ​​are performed within the TEE, and the system-specific hyperparameter values ​​are not output from the TEE. For example, after the trained model (either a fully trained model or a finally trained model) is output from the TEE, the system-specific hyperparameters remain in the TEE (preferably in encrypted form); or the system-specific hyperparameters are deleted after the trained model is output from the TEE. Since the system-specific hyperparameters are not visible in the output trained model and are inaccessible within the TEE, no one can derive them, and therefore they do not contribute to the privacy budget.

[0076] like Figure 2 As shown, in some example embodiments, the following actions are performed: - Data providers upload their private data to the TEE enclave (system). Preferably, the TEE enclave runs verified code. The model trainer uses this proprietary data to tune (determine) the values ​​of system-specific and model-specific hyperparameters. Determining hyperparameter values ​​can be done by any program. TEE Enclave ensures that this program does not reveal any information. The system uses the determined hyperparameter values ​​to perform model training. This training is conducted within a TEE enclave and the hyperparameters are not exposed. - When training is complete (i.e., the model is fully trained), the values ​​of system-specific hyperparameters are discarded (or encrypted) within the TEE enclave and may not be recovered. - Fully trained models are output outside the model trainer and TEE enclave. System-specific hyperparameters are not output outside the TEE enclave.

[0077] Some relevant aspects of some example embodiments are summarized below: 1. Determine the values ​​of (system-specific and model-specific) hyperparameters and run the ML model training within a TEE enclave. The code running within this enclave tracks the privacy budget spent and excludes privacy leaks from tuning the values ​​of system-specific (training and privacy) hyperparameters from this privacy account. These hyperparameters can be specific to this training session. 2. A model trainer provider that determines the values ​​of ML training and privacy-preserving hyperparameters based on a private dataset. This program runs within a TEE enclave. It cannot disclose any information to the outside world and only returns the determined hyperparameter values ​​to trusted components of the system within the enclave. 3. The tuning of hyperparameter values ​​(i.e., the decision and selection of which ML training and privacy hyperparameters to use) can take one of the following forms. a. Calculate directly from dataset features, values, or formats. b. It is determined experimentally by running partial training (such as training for a small number of iterations). This partial training can occur either once or multiple times (e.g., grid search). c. A combination of the two options above. In these cases, after determining the values ​​of the system-specific hyperparameters, no intermediate results can be used. If a particular hyperparameter should not be used (not selected), this non-selection is equivalent to setting the value of that hyperparameter to a specific value (such as 0 for summation or 1 for multiplication). 4. The system running the training session keeps the hyperparameters secret during training and discards them after use. In some example embodiments, this action can be verified by any dataset provider, as they can verify and confirm that the code responsible for tracking the privacy budget and discarding the hyperparameters is correct and protects dataset privacy. 5. The system distinguishes between system-specific hyperparameters (used during training but only visible to the system during the training session) and model-specific hyperparameters (visible to the model and published at the end of training). Note that model-specific hyperparameters... No Excluded from privacy accounting (i.e., Figure 1 The model-specific tuning budgets shown at the bottom of the stack are accessible from the output of the trained model and thus compromise privacy.

[0078] The privacy budget spent for specific model parameters can be calculated as follows, for example:

[0079] The pruning boundary (typically a model-specific privacy hyperparameter) can be determined from directly observed data and computed gradients to select a value that ensures no excessive gradients will be pruned. To ensure privacy, the sensitivity of this value must first be computed (a standard step in applying dynamic programming to a specific mechanism), and then a proportional amount of noise must be added to this value to protect the privacy of the data. The sensitivity and amount of noise together determine how much of the privacy budget must be spent to determine the hyperparameter "pruning boundary".

[0080] Another approach to determining the optimal values ​​of hyperparameters is to run training on a small number of epochs. For example, train the model from scratch for five training epochs using different values ​​of the learning rate hyperparameter (typically a model-specific training hyperparameter). Then, among those tested learning rates, the one that produces the model with the highest accuracy is selected, and the privacy budget spent is determined by a standard DP-SGD privacy kernel combined with any additional privacy mechanisms applied to the learning rate value.

[0081] Figure 3 A method according to some example embodiments is illustrated. The method may include... Figure 3 Further actions not shown in the text. 1. Data sets, including private data, are uploaded to a secure TEE enclave. 2. Dataset preparation (including data encoding and discarding of idle elements) occurs within the TEE. Figure 3 The remaining processes shown assume that each data element will be used for both tuning the values ​​of hyperparameters and training the model, although this assumption is not mandatory. 3. (Not mandatory) Figure 3 (Not shown in the image) A temporary environment is created to perform tuning of hyperparameter values. The temporary environment can be created by copying or checking the base environment. This temporary environment can be used with arbitrary operations to tune training and privacy hyperparameter values ​​(including customization and the use of secret functions), but no information may be leaked outside the temporary environment. 4. The dataset is sent to a hyperparameter tuning environment that determines the values ​​of system-specific hyperparameters based on the dataset, which includes private data. Because the values ​​of system-specific hyperparameters are not output before or after training is complete, this tuning incurs no privacy budget. Furthermore, the hyperparameter tuning environment determines the values ​​of model-specific hyperparameters based on the dataset, which includes private data. Because model-specific hyperparameters are accessible from the output trained ML model, this tuning incurs some privacy budget (≠ 0). The sequence of determining the values ​​of system-specific and model-specific hyperparameters is arbitrary. The values ​​of system-specific and model-specific hyperparameters can be determined completely or partially in parallel. exist Figure 3 In this context, the tuning of hyperparameters is performed by a "custom decision function": this means that the value of the hyperparameter depends on a specific algorithm (and is often specific to a particular model and / or learning algorithm) that is independent of the training framework. In other words, it is a black box that takes data as input and outputs the chosen values, along with the privacy budget spent (for model-specific parameters), but is otherwise completely "customizable". 5. (Not mandatory) Figure 3 (Not shown in the image) The hyperparameter tuning environment is destroyed (if made by copying) or rolled back (if made by checkpointing) to erase any privacy loss that occurs as a result of seeing private data. 6. Model training is performed, where the determined hyperparameter values ​​are applied and held constant (unmodified). The system tracks the privacy budget spent as a result of both the tuning of the hyperparameter values ​​and training. For example, for each additional training epoch, the training manager can estimate the privacy budget to be spent for that corresponding additional training epoch. If the sum of the privacy budget already spent (attributed to determining the model-specific hyperparameter values ​​and previous training epochs) and the estimated privacy budget for the corresponding additional training epoch exceeds the total privacy budget, then the model can be considered fully trained (for the given total privacy budget) and output from the TEE. Therefore, the training can be terminated. 7. At the end of training (e.g., when the total privacy budget is spent, as described in 6; or when some other condition for terminating training is met before the total privacy budget is spent), the trained model is output (published) outside the TEE (e.g., to the model trainer). The output trained model can be used for inference. 8. The TEE is destroyed, or if it remains alive, the system-specific hyperparameters are ensured to no longer be used for any other purpose. For example, the system-specific hyperparameters are discarded. The actions of the TEE can be verified by the dataset owners, as they can examine the code responsible for discarding the system-specific hyperparameters and track the privacy budget spent during training. The sequence of actions 7 and 8 is arbitrary. They can be executed fully or partially in parallel.

[0082] The tuning of hyperparameter values ​​can be implemented individually. Configuration Function This configuration allows the system to automatically determine hyperparameter values ​​without diminishing the model provider's ability to (fine-tune) hyperparameter values ​​using specific optimization algorithms. The configuration takes a prepared dataset as input and outputs system-specific and model-specific hyperparameters to be used during training. Any computation to determine hyperparameter values ​​can occur as long as the computation does not leak any information about private data or system-specific hyperparameters outside the TEE.

[0083] Examples of system-specific hyperparameters include: batch size, DP noise level (σ), DP clipping boundaries, subsampling usage, individual component timeout, and aggregation method.

[0084] Examples of model-specific hyperparameters include: the number of training rounds, the learning rate, the loss function, and the gradient shape.

[0085] Note that the type of a hyperparameter depends on the implementation. For example, in some implementations, the batch size can be a system-specific training hyperparameter, while in others it can be a model-specific hyperparameter.

[0086] In some example implementations, the model trainer may predefine whether a hyperparameter is system-specific or model-specific. That is, many hyperparameters are well understood by ML experts, whose expertise helps them assess whether the value of a particular hyperparameter will be visible to the model. In some example implementations, the system automatically determines whether a hyperparameter is system-specific or model-specific. Two possible ways to automatically determine this are symbolic execution (observing whether the hyperparameter value is part of any value computed to be visible to the model) and correlation analysis (empirically observing whether changing the value of a hyperparameter makes a significant change in the value visible to the model). In some example implementations, the system automatically determines whether a given hyperparameter is system-specific or model-specific for each hyperparameter in the hyperparameter set. In some example implementations, the model trainer predefines the types of some hyperparameters, and the system automatically determines the types of other hyperparameters.

[0087] For example, one or more of the following technologies can be used within the customization decision function: - Choose values ​​based on information from the dataset itself. For example, this information could be the number of data elements, the mean of those elements, an estimate of the sparsity of the input space, or the variance among the elements of the dataset. Dataset information can be used to select hyperparameters that directly relate to the dataset information (e.g., setting the minimum batch size based on the number of data elements). - Values ​​are selected based on training sample runs. This can be achieved by running ML training for a duration much shorter than a full training run and observing the model's initial behavior (e.g., early convergence). In this case, any training steps completed within the configuration are discarded after the hyperparameter values ​​have been determined for the actual training. - A combination of the two techniques above can be used to determine the values ​​of hyperparameters. For example, dataset information can be used to create... Grid Search Plan Several sample configurations are generated, and some of the training runs are used to find the potential best possible values ​​for the hyperparameters.

[0088] Typically, any intermediate results generated within a configuration function, other than the hyperparameters' own values ​​(which are used during training), can be discarded after completion. To achieve this, while also preserving the flexibility to set up different configuration environments (such as where a complete training environment must be set up), one of two techniques can be applied: copy-and-discard, or checkpoint-and-rollback.

[0089] Copy and Discard ( Figure 4 This involves creating a complete copy of the original training environment within the TEE, including pre-existing model states, global variables, and any newly created variables. The private dataset is then loaded into the copied environment. Any information computed within this environment is confined to the copied environment. Once the hyperparameter values ​​have been determined, the replica environment is completely discarded, and the original environment (which has not yet seen any private data in the private data) is restored. This ensures that the configuration function does not leak any private information.

[0090] Checkpoints and rollbacks create a mechanism for restoring the original state of an environment by using a combination of shallow copying, state bits, and tracking of any changes made within the environment, avoiding the need to completely copy the environment (as in copy-and-discard).

[0091] Using this technique, all changes made within the environment after configuration is complete, and before any private data in the private data is seen, are "undone" to restore the original state. This provides the same guarantees as the copy-and-discard mechanism, but may require fewer memory operations.

[0092] Privacy Budget Tracking. For a system that guarantees a given DP privacy level (i.e., does not spend more than the total privacy budget), the system must track the privacy budget spent during training. For example, for each additional training epoch, the training manager can estimate the privacy budget to be spent for that corresponding additional training epoch. If the sum of the privacy budget already spent (attributed to determining the values ​​of model-specific hyperparameters and previous training epochs) and the estimated privacy budget for that corresponding additional training epoch exceeds the total privacy budget, then the model can be considered fully trained (for the given total privacy budget) and output from the TEE.

[0093] Some example implementations guarantee System-specific hyperparameters It cannot be derived from the output of the trained model, and therefore no privacy budget is spent on tuning the values ​​of system-specific hyperparameters.

[0094] Specifically, according to some example embodiments, ML training now has only two sources of privacy leakage (instead of the three according to the conventional approach): the tuning of the values ​​of model-specific hyperparameters (which the model sees and therefore introduces privacy leakage), and the training epoch. To ensure that the privacy budget is calculated correctly, we can: - Maintain a running bill for the privacy budget spent and dynamically adapt the privacy budget for tuning hyperparameter values ​​and model training to take advantage of additional opportunities (e.g., improving model accuracy by performing more epochs of training). - Report the "recovered" privacy budget at the end of training, resulting in higher privacy guarantees through the unspent privacy budget.

[0095] TEE execution guarantees that each step will be taken correctly. Therefore, no privacy loss occurs if any additional available privacy budget is spent before system-specific hyperparameters are discarded. To achieve this, it should be guaranteed that if a failure state is entered, the system will forget any private data and system-specific hyperparameters.

[0096] Figure 5 An apparatus according to an example embodiment is shown. This apparatus may be a configurator or an element thereof for hyperparameters. Figure 6 A method according to an example embodiment is shown. Figure 5 The device can perform Figure 6 The method, but not limited to this method. Figure 6 The method can be derived from Figure 5 The device may be used to perform the action, but is not limited to the device itself.

[0097] The device includes a determining component 110, a training component 120, an output component 130, and a prohibiting component 140. The determining component 110, the training component 120, the output component 130, and the prohibiting component 140 can be a determining component, a training component, an output component, and a prohibiting component, respectively. The determining component 110, the training component 120, the output component 130, and the prohibiting component 140 can be a determiner, a trainer, an outputter, and a prohibitor, respectively. The determining component 110, the training component 120, the output component 130, and the prohibiting component 140 can be a determining processor, a training processor, an output processor, and a prohibiting processor, respectively.

[0098] The component 110 used for determination determines the values ​​of hyperparameters of the ML model based on private data stored in the TEE (S110). Hyperparameters include system-specific hyperparameters and model-specific hyperparameters.

[0099] The training component 120 trains the ML model in the TEE, and the determined system-specific and model-specific hyperparameter values ​​are applied to the ML model (S120). Therefore, after one or more epochs of training, the training component 120 obtains a fully trained ML model. The system-specific hyperparameters are inaccessible in the output fully trained ML model.

[0100] The output component 130 outputs a fully trained ML model from the TEE (S130).

[0101] The component 140 for disabling disables the value of the system-specific hyperparameter determined from the TEE output (S140).

[0102] Figure 7 An apparatus according to an example embodiment is shown. The apparatus includes: at least one processor 810 and at least one memory 820 storing instructions, which, when executed by the at least one processor 810, cause the apparatus to perform at least the following: Figure 6 And related methods of description.

[0103] In some example implementations, private data is used to determine both hyperparameter values ​​and model training. For this model training, different options exist: Model training can use the same private data that was used to determine the values ​​of hyperparameters; Model training can use a subset of private data used to determine the values ​​of hyperparameters; Model training can use private data other than the private data used to determine the values ​​of hyperparameters (in addition to at least a subset of the private data used to determine the values ​​of hyperparameters, or in place of at least a subset of the private data used to determine the values ​​of hyperparameters). Model training can be performed without using any private data. In this case, the entire privacy budget can be used to determine the values ​​of the hyperparameters. According to some example embodiments, in this case, the entire privacy budget can be used to determine the values ​​of model-specific hyperparameters.

[0104] The term "private data" means data that must be protected with regard to confidentiality, and preferably also with regard to integrity (e.g., according to some contract between the data provider and the model trainer, or for the commercial reasons of the model trainer).

[0105] In some example embodiments, privacy-preserving mechanisms (such as dynamic programming or k-anonymity) are applied to private data to determine both hyperparameter values ​​and model training. The privacy-preserving mechanisms and their parameters used to determine hyperparameter values ​​can be the same as those used for model training. Alternatively, the privacy-preserving mechanisms and / or their parameters used to determine hyperparameter values ​​can differ from those used for model training.

[0106] Applying privacy-preserving mechanisms (such as dynamic programming or k-anonymity) is not mandatory. In some example embodiments, privacy-preserving mechanisms may not be used in determining the values ​​of hyperparameters. In some example embodiments, privacy-preserving mechanisms may not be used during model training. In some example embodiments, privacy-preserving mechanisms may be used in determining the values ​​of hyperparameters and not used during model training.

[0107] A message can be sent from one entity to another in one or more messages. Each of these messages can include additional (different) pieces of information.

[0108] The names of network elements, network functions, protocols, and methods are based on current standards or current proposals. These names are not restrictive. For example, in other versions or other technologies, the names of corresponding entities may be different from or the same as those in this specification.

[0109] Unless otherwise stated or made clear from the context, a statement that two entities are distinct implies that they perform different functions. This does not necessarily mean they are based on different hardware. That is, each entity described in this specification may be based on different hardware, or some or all of the entities may be based on the same hardware. It does not necessarily mean they are based on different software. That is, each entity described in this specification may be based on different software, or some or all of the entities may be based on the same software. Each entity described in this specification may be deployed in the cloud.

[0110] For illustrative purposes, exemplary embodiments of this application are provided below.

[0111] Example 1. An apparatus comprising: A component for determining the values ​​of hyperparameters of a machine learning model in a trusted execution environment based on private data stored in the trusted execution environment, wherein the hyperparameters include: system-specific hyperparameters and model-specific hyperparameters; The components used to train the machine learning model in the trusted execution environment, wherein the determined values ​​of the system-specific and model-specific hyperparameters are applied to the machine learning model to obtain a fully trained machine learning model after one or more training epochs; Components for outputting the fully trained machine learning model from the trusted execution environment; and A component for disabling the output of the determined system-specific hyperparameter values ​​from the trusted execution environment, wherein... The system-specific hyperparameters are inaccessible in the output, fully trained machine learning model.

[0112] Example 2. The apparatus according to Example 1 further includes: A component for calculating the total value of the privacy budget by adding the privacy budget spent for determining the value of the hyperparameter to the privacy budget spent for the one or more epochs of training; wherein, The value of the privacy budget spent for determining the value of the system-specific hyperparameter is 0.

[0113] Example 3. The apparatus according to Example 2, wherein The component used for output is configured to additionally output the total value of the privacy budget from the trusted execution environment.

[0114] Example 4. The apparatus according to any one of Examples 2 and 3 further includes: Components for estimating the privacy budget to be spent for another period of the training; A component for calculating the estimated total value of the privacy budget by adding the total value of the privacy budget spent so far to the estimated value to be spent for the other period of the training; A component for checking whether the estimated total value of the privacy budget exceeds the maximum privacy budget, wherein The component for output is configured to output the fully trained machine learning model if the estimated total value of the privacy budget exceeds the maximum privacy budget, by means of the trained learning model obtained during the training period performed without the other training period.

[0115] Example 5. The apparatus according to any one of Examples 1 to 4, wherein at least one of the following: The component for determining the value of the hyperparameter is configured to: apply a privacy protection mechanism to the private data, and determine the value of the hyperparameter based on the private data to which the privacy protection mechanism is applied; or The component used to train the machine learning model is configured to apply the privacy protection mechanism to the data used for training.

[0116] Example 6. The apparatus according to Example 5, wherein the privacy protection mechanism is differential privacy.

[0117] Example 7. The apparatus according to any one of Examples 5 and 6, wherein at least one of the hyperparameters configures the privacy protection mechanism.

[0118] Example 8. The apparatus according to Example 7, wherein at least one of the hyperparameters of the hyperparameters configuring the privacy protection mechanism is a noise level or a pruning boundary, the noise level being added to at least one of: the private data, or the data used for the training.

[0119] Example 9. The apparatus according to Example 8, wherein at least one of the following: The noise level is system-specific; or The clipping boundaries are model-specific.

[0120] Example 10. The apparatus according to any one of Examples 1 to 9, wherein at least one of the hyperparameters configures the training of the machine learning model.

[0121] Example 11. The apparatus according to Example 10, wherein at least one of the hyperparameters of the training of the machine learning model is: the batch size of the data used in the training, or the learning rate for the training.

[0122] Example 12. The apparatus according to Example 11, wherein at least one of the following: The batch size is system-specific; or The learning rate is model-specific.

[0123] Example 13. The apparatus according to any one of Examples 1 to 12, wherein the component for training is configured to use at least a subset of the private data for the training.

[0124] Example 14. The apparatus according to any one of Examples 1 to 13 further includes: A component for deleting or encrypting system-specific hyperparameters when the fully trained machine learning model is acquired.

[0125] Example 15. The apparatus according to any one of Examples 1 to 14 further includes: a component for determining whether one of the hyperparameters is a system-specific component.

[0126] Example 16. The apparatus according to any one of Examples 1 to 15, wherein the trusted execution environment is configured to ensure that the code representing the apparatus, the hyperparameters, and the private data are protected with respect to confidentiality and integrity within the trusted execution environment.

[0127] Example 17. A method comprising: The values ​​of hyperparameters of a machine learning model are determined based on private data stored in the trusted execution environment, wherein the hyperparameters include system-specific hyperparameters and model-specific hyperparameters. The machine learning model is trained in the trusted execution environment, and the determined values ​​of the system-specific and model-specific hyperparameters are applied to the machine learning model to obtain a fully trained machine learning model after one or more training epochs. Output the fully trained machine learning model from the trusted execution environment; and, The determined values ​​of the system-specific hyperparameters are prohibited from being output from the trusted execution environment; wherein... The system-specific hyperparameters are inaccessible in the output of the fully trained machine learning model.

[0128] Based on the above description, it should be apparent that, for example, the exemplary embodiments provide, for instance, a configurator or element thereof for hyperparameters, means embodying the same configurator or element thereof for hyperparameters, a method for controlling and / or operating the same configurator or element thereof for hyperparameters, and multiple computer programs for controlling and / or the same configurator or element thereof for hyperparameters, and a medium carrying such multiple computer programs and forming multiple computer program products.

[0129] By way of non-limiting example, any implementation of the blocks, apparatus, systems, technologies, or methods described above includes implementations as hardware, software, firmware, special-purpose circuitry or logic, general-purpose hardware or controllers or other computing devices, or combinations thereof. Each entity described in this specification can be represented in the cloud.

[0130] It should be understood that the content described above represents what is currently considered a preferred exemplary embodiment. However, it should be noted that the description of the preferred exemplary embodiments is given by way of example only, and various modifications may be made without departing from the scope of this disclosure as defined by the appended claims.

[0131] Unless otherwise specified, the terms "first X" and "second X" include: the same options as "second X", and options where "first X" differs from "second X". As used herein, phrases such as "at least one of the following: " and "at least one of " and similar wording, where the list of two or more elements is connected by "and" or "or", mean at least any one of the elements, or at least any two or more elements. Unless otherwise specified (e.g., the use of "otherwise" or "or in the alternative"), the term "or" means a non-exclusive "or".

Claims

1. An apparatus for maintaining the privacy of private data, comprising: A component for determining the values ​​of hyperparameters of a machine learning model in a trusted execution environment based on private data stored in the trusted execution environment, wherein the hyperparameters include: system-specific hyperparameters and model-specific hyperparameters; The components used to train the machine learning model in the trusted execution environment, wherein the determined values ​​of the system-specific and model-specific hyperparameters are applied to the machine learning model to obtain a fully trained machine learning model after one or more training epochs. Components for outputting the fully trained machine learning model from the trusted execution environment; and A component for disabling the output of the determined system-specific hyperparameter values ​​from the trusted execution environment, wherein... The system-specific hyperparameters are inaccessible in the output, fully trained machine learning model.

2. The apparatus according to claim 1, further comprising: A component for calculating the total value of the privacy budget by adding the privacy budget spent for determining the value of the hyperparameter to the privacy budget spent for the one or more epochs of training; wherein, The value of the privacy budget spent for determining the value of the system-specific hyperparameter is 0.

3. The apparatus according to claim 2, wherein The component used for output is configured to additionally output the total value of the privacy budget from the trusted execution environment.

4. The apparatus according to any one of claims 2 and 3, further comprising: Components for estimating the privacy budget to be spent for another period of the training; A component for calculating the estimated total value of the privacy budget by adding the total value of the privacy budget spent so far to the estimated value to be spent for the other period of the training; A component for checking whether the estimated total value of the privacy budget exceeds the maximum privacy budget, wherein The component for output is configured to output the fully trained machine learning model if the estimated total value of the privacy budget exceeds the maximum privacy budget, by means of the trained learning model obtained during the training period performed without the other training period.

5. The apparatus according to any one of claims 1 to 4, wherein at least one of the following: The component for determining the value of the hyperparameter is configured to: apply a privacy protection mechanism to the private data, and determine the value of the hyperparameter based on the private data to which the privacy protection mechanism is applied; or The component used to train the machine learning model is configured to apply the privacy protection mechanism to the data used for training.

6. The apparatus of claim 5, wherein the privacy protection mechanism is differential privacy.

7. The apparatus according to any one of claims 5 and 6, wherein at least one of the hyperparameters configures the privacy protection mechanism.

8. The apparatus of claim 7, wherein at least one of the hyperparameters configuring the privacy protection mechanism is a noise level or a pruning boundary, the noise level being added to at least one of: the private data, or the data used for the training.

9. The apparatus of claim 8, wherein at least one of the following: The noise level is system-specific; or The clipping boundaries are model-specific.

10. The apparatus according to any one of claims 1 to 9, wherein at least one of the hyperparameters configures the training of the machine learning model.

11. The apparatus of claim 10, wherein the at least one hyperparameter of the hyperparameters for configuring the training of the machine learning model is: the batch size of the data used in the training, or the learning rate for the training.

12. The apparatus of claim 11, wherein at least one of the following: The batch size is system-specific; or The learning rate is model-specific.

13. The apparatus according to any one of claims 1 to 12, wherein the component for training is configured to use at least a subset of the private data for the training.

14. The apparatus according to any one of claims 1 to 13, further comprising: A component for deleting or encrypting system-specific hyperparameters when the fully trained machine learning model is acquired.

15. The apparatus according to any one of claims 1 to 14, further comprising: Used to determine whether one of the hyperparameters is a system-specific component.

Images