Machine learning privacy auditing method and system in trusted execution environment

Abstract

The invention discloses a machine learning privacy auditing method and system in a trusted execution environment, and belongs to the technical field of machine learning. The method comprises the steps of uploading a private data set to a remote machine learning service deployed in a trusted execution environment to train a target model, and obtaining measurement information of private data in each training round of the target model; synthesizing the measurement information in each training round to generate a measurement sequence of a plurality of training rounds; extracting dynamic characteristics of the measurement sequence; performing classification based on the dynamic features to obtain member relationship prediction results of the private data in a plurality of training rounds; and determining whether to continue training, suspend training or exit training according to a member relationship prediction result in the plurality of training rounds. According to the method, a privacy risk measurement and auditing mechanism which is fine in granularity, low in overhead and capable of being dynamically evaluated can be realized.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, specifically relating to a machine learning privacy auditing method and system in a trusted execution environment. Background Technology

[0002] With the increasing prevalence of cloud computing and the Machine Learning as a Service (MLAAS) model, enterprises and institutions are increasingly inclined to outsource model training to the cloud to achieve lower hardware investment and greater elasticity and scalability. However, this model also introduces significant data security and privacy risks: during the training phase, users typically need to upload datasets containing sensitive information to cloud servers, a process that may be vulnerable to theft and misuse by malicious cloud service providers or external attackers.

[0003] To mitigate this risk, the closest existing technological approach is to deploy remote machine learning tasks within a Trusted Execution Environment (TEE). This approach uses strong hardware isolation mechanisms to protect the confidentiality and integrity of data and code during training and inference. TEEs create isolated, trusted execution spaces within the processor, preventing external environments from directly accessing or tampering with internal programs and runtime data. This provides fundamental privacy and integrity guarantees for remote machine learning and offers static protection for model parameters, architecture, and client-uploaded data.

[0004] However, TEE (Train the Execution Environment) cannot prevent indirect privacy leaks caused by model usage behavior. Related research shows that even if sensitive data remains in an encrypted or isolated execution environment, as long as it is used in model training, attackers can still use the model's response signals to samples to perform membership inference attacks, thereby determining whether a sample belongs to the training set. If successful, this often leads to more serious privacy leaks, such as inferring a patient's medical history or health status in a medical context. Therefore, further identifying and quantifying such privacy risks based on TEE has become an urgent problem to be solved in remote machine learning.

[0005] Existing work on membership reasoning largely adopts an attacker's perspective to pursue higher recognition accuracy, with typical public tools including ML-Doctor and ML Privacy Meter. This approach typically relies on shadow model construction, model distillation, and training of complex attack models, resulting in significant computational and engineering overhead. Furthermore, most methods tend to perform a one-time evaluation of the static model after training, failing to reflect the dynamic risk evolution during training. Additionally, the limited information available from the attacker's perspective contrasts with the richer signals available to the client during actual training. These factors collectively make it difficult for such methods to meet the practical requirements of low latency, high throughput, and real-time risk awareness in remote training scenarios.

[0006] Against this technological backdrop, industry and academia have gradually proposed the need for client-side privacy auditing: that is, without violating the TEE trust assumption, making full use of the observable information of the client during the training process to achieve a fine-grained, low-overhead, and dynamically assessable privacy risk measurement and auditing mechanism. Summary of the Invention

[0007] This invention proposes a remote machine learning privacy auditing method and system for Trusted Execution Environments (TEEs). Its core idea is to place the perception and measurement of privacy risks on the client side without altering the existing TEE trust model and training process. The client proactively uses its own observable information to conduct auditing, thus achieving a fine-grained, low-overhead, and dynamically assessable privacy risk measurement and auditing mechanism.

[0008] To achieve the above objectives, the technical solution of the present invention includes the following:

[0009] A method for privacy auditing of machine learning in a trusted execution environment, using a client, the method comprising: Upload the private dataset to a remote machine learning service deployed in a trusted execution environment to train the target model and obtain the private data. The metrics used in each training epoch of the target model include one or more of the following: loss value, confidence of the correct class, prediction entropy, and raw Logit value. By combining the metric information from each training round, a metric sequence for multiple training rounds is generated; Extracting dynamic features from the metric sequence; Based on this dynamic characteristic, the private data is obtained through classification. Membership prediction results across multiple training rounds; Based on the member relationship prediction results in multiple training rounds, determine whether to continue training, pause training, or withdraw from training.

[0010] Furthermore, the dynamic characteristics include: the rate of change of the measurement sequence and / or the cumulative fluctuation of the measurement sequence.

[0011] Furthermore, the rate of change of the metric sequence is extracted, including: The least squares method is used to fit the metric sequence to a first-order linear model. ;in, Represents training rounds, Represents the intercept term and the rate of change. Mean over time Measure the mean of a sequence , This indicates the total number of training rounds for the metric sequence. Indicates training rounds The measurement value; The rate of change As a measure of the rate of change of a sequence.

[0012] Furthermore, the cumulative fluctuation of the metric sequence ;in, Represents training rounds, This indicates the total number of training rounds for the metric sequence. Indicates training rounds The metric value.

[0013] Furthermore, based on this dynamic characteristic, classification is performed to obtain the private data. Membership prediction results across multiple training rounds include: A labeled training set is constructed using real member samples and non-member samples known to the client. A binary classification auditing model is trained on the training set based on a machine learning algorithm; wherein the machine learning algorithm includes: support vector machine; Based on this dynamic feature, an audit feature vector is generated, and this audit feature vector is fed into the binary classification audit model to obtain the private data. Membership prediction results across multiple training rounds.

[0014] Furthermore, based on the member relationship prediction results across multiple training rounds, it is determined whether to continue training, pause training, or exit training, including: Based on the member relationship prediction results in multiple training rounds, audit metrics are generated; wherein, the audit metrics include: AUC value and true positive rate at a false positive rate of 0.1%; Based on the first and second set thresholds, determine whether to continue, pause, or exit training.

[0015] A machine learning privacy auditing system in a trusted execution environment, the system comprising: The information acquisition module is used to upload private datasets to a remote machine learning service deployed in a trusted execution environment for training the target model and acquiring private data. The metrics used in each training epoch of the target model include one or more of the following: loss value, confidence of the correct class, prediction entropy, and raw Logit value. The sequence generation module is used to integrate the metric information from each training round and generate metric sequences for multiple training rounds. The feature extraction module is used to extract dynamic features of the metric sequence; The relationship prediction module is used to classify based on this dynamic feature to obtain the private data. Membership prediction results across multiple training rounds; The privacy audit module is used to determine whether to continue training, pause training, or exit training based on the member relationship prediction results in multiple training rounds.

[0016] An electronic device, characterized in that the electronic device comprises: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the machine learning privacy auditing method in a trusted execution environment as described above.

[0017] A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the machine learning privacy auditing method in a trusted execution environment as described above.

[0018] A computer program product, characterized in that, when the computer program product is run on a computer device, it causes the computer device to execute the machine learning privacy auditing method in a trusted execution environment as described above.

[0019] Compared with the prior art, the present invention has at least the following beneficial effects.

[0020] This invention innovatively proposes two key discriminative features: difference in rate of change and difference in cumulative fluctuation. The rate of change directly characterizes how quickly a sample is optimized during training; member samples often exhibit a rapid decrease in loss or a rapid increase in confidence, while non-member samples change slowly. Cumulative fluctuation reflects the difference in overall stability during training; member samples tend to stabilize after convergence, while non-member samples maintain significant fluctuations. These two metrics have extremely low computational cost, enabling effective differentiation at minimal expense. Furthermore, by fusing these two features from four core metrics (loss, confidence, entropy, and normalized logit), this invention constructs a multidimensional audit vector and utilizes a support vector machine for supervised classification. Clients can adaptively train the auditor using their own real member / non-member labels, enabling it to flexibly adapt to different models and data distributions, significantly improving robustness and generalization ability. Thus, this invention not only reduces overhead but also overcomes the shortcomings of existing methods, such as static nature, coarse granularity, and limited information utilization.

[0021] At the experimental level, the effectiveness of this invention has been systematically verified. Experimental results show that the proposed method can efficiently distinguish between member and non-member samples on multiple datasets such as CIFAR-10, CIFAR-100, and Tiny ImageNet, and exhibits good generalization performance in cross-model and cross-dataset scenarios. Particularly noteworthy is that the method maintains high detection accuracy even with extremely low false positive rates, demonstrating its practical value and robustness. Furthermore, ablation experiments further reveal the mechanisms of action of different features: the rate of change index outperforms fluctuation accumulation in member discrimination, and can more reliably reflect training dynamics. Simultaneously, the experiments also reveal the dynamic evolution of privacy risks with each training round, confirming the necessity and effectiveness of this invention in round-by-round tracking and dynamic auditing. Attached Figure Description

[0022] Figure 1 Client-side privacy risk assessment framework. Detailed Implementation

[0023] To further illustrate the technical solutions of the present invention, the present invention will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] In terms of specific implementation, such as Figure 1 As shown, the method flow of the present invention can be summarized into the following steps: Step 1: Obtain dynamic output information during model training.

[0025] In one embodiment, the client uploads its private dataset to a remote machine learning service deployed in a TEE and participates in model training. During training, the client can obtain complete output information of the target model for its input samples in each training epoch, including but not limited to: loss value, confidence score of the correct class, prediction entropy, raw Logit value, etc. This information constitutes the basis for subsequent privacy audits.

[0026] Step 2: Generate time series of samples across multiple core metrics.

[0027] In one embodiment, for each sample in the client dataset This invention selects one or a combination of the following four core metrics: Loss: i.e., the cross-entropy loss value , indicating that the model in the Wheel pair samples Prediction error; Confidence: The maximum value in the probability distribution of the model output. If the sample is correctly classified, then it represents the probability corresponding to the true label. Entropy: The entropy of the predicted output, used to measure the uncertainty of the model's prediction; Rescaled Logit: This refers to the prominence of the Logit of the correct class relative to the Logits of other classes, serving as a more stable membership signal.

[0028] For each sample And the selected metrics, in model training During the rounds, the index value in each round is recorded to form a time series: in, Indicates the first The metric value corresponding to the wheel.

[0029] Step 3: Extract dynamic features – rate of change and cumulative fluctuations.

[0030] This invention proposes two lightweight dynamic feature extraction methods, based on the overall trend and local fluctuations of the indicator sequence, respectively.

[0031] (1) Feature extraction based on rate of change.

[0032] First, the difference in the rate of change is due to the continuous gradient optimization of the model on member samples during training. Its key metrics evolve rapidly in the direction of optimization (such as increased confidence and decreased loss). Therefore, the rate of change of metrics for member samples is usually significantly greater than that for non-member samples.

[0033] Specifically, in one embodiment, the measurement sequence for each sample The first-order linear model is fitted using the least squares method: in Represents the training round; Represents the intercept term, that is The measure when it is 0; slope This represents the rate of change of the sequence, and its calculation formula is: in, It is considered as the average over the time span; It is considered as the mean of the sequence. Because member samples are repeatedly optimized during training, their loss and entropy often show a decreasing trend, while confidence and scaling logit usually show an increasing trend. Therefore, the absolute value of the rate of change... The larger the value, the more likely the sample is to be a member sample.

[0034] (2) Feature extraction based on fluctuation accumulation.

[0035] The cumulative volatility difference arises because member samples, once the model has learned to stabilize around them, should exhibit smaller fluctuations in their metrics; while non-member samples may continue to fluctuate due to the uncertainty of the model's response. Therefore, the cumulative volatility of member samples' metrics during training is generally smaller than that of non-member samples.

[0036] In one embodiment, for the same sequence The sum of the absolute values ​​of the changes in the indicator values ​​of adjacent rounds is defined as the cumulative fluctuation. This value reflects the total fluctuation of the metric throughout the entire training process. Theoretically, member samples exhibit less fluctuation after the model has stabilized, while non-member samples, due to their lack of optimization, have unstable responses and greater fluctuations. Therefore, the fluctuations accumulate. The smaller the sample size, the more likely it is to be a member sample.

[0037] Step 4: Determine member relationships.

[0038] This invention provides two determination modes: (1) Single-metric lightweight auditing mode: In one embodiment, the client selects only the rate of change or cumulative fluctuation of a single indicator (such as confidence level) as the criterion. This mode has extremely low computational overhead and is suitable for scenarios with high real-time requirements.

[0039] (2) Multi-metric Privacy-Enhanced Auditing Mode: In one embodiment, to improve auditing accuracy, this invention integrates multi-dimensional dynamic features. Specifically: for each sample The rate of change and cumulative fluctuation of four types of indicators—loss, confidence, entropy, and scaling Logit—were calculated separately, resulting in a total of eight feature values. These eight values ​​were then combined into an eight-dimensional audit feature vector. Next, a labeled training set is constructed using real member samples (participating in training) and non-member samples (with the same distribution but not participating in training) known to the client. A binary classification auditing model is trained using a support vector machine (SVM) for supervised learning on this training set. In subsequent audits, the 8-dimensional feature vector of any sample is input into this SVM model, which outputs the predicted membership relationship.

[0040] Step 5: Privacy risk assessment.

[0041] Ultimately, by comparing metrics such as AUC and TPR@0.1%FPR across different rounds, the client decides whether to continue, pause, or exit training. This process can also serve as archiving evidence for compliance audits. The entire process remains decoupled from the server-side training logic; the server is only responsible for providing a secure training environment and interface support within the TEE, without intervening in audit details. This architecture ensures system compatibility and scalability, enabling this invention to be seamlessly integrated onto existing remote machine learning services.

[0042] In one embodiment, the decision to continue, pause, or exit training is made by combining the audit metric AUC value and the true positive rate at a false positive rate of 0.1%. Continue training: When the audit metric AUC and the true positive rate at a false positive rate of 0.1% are at low levels, for example, the AUC is below 0.5 and the true positive rate is below 0.5% at a false positive rate of 0.1%, it indicates that the distributions of members and non-members overlap significantly and cannot be distinguished. At this point, the model has low privacy risk and training can continue.

[0043] Training pause: When the audit metric AUC and the true positive rate at a false positive rate of 0.1% exceed the low-risk threshold (e.g., an AUC exceeding the random guessing level of 0.5, or a true positive rate reaching 1% at a false positive rate of 0.1%), training can be paused or closely monitored. Clients need to assess whether the current behavior justifies the increasing privacy risks, or consider introducing defensive mechanisms such as differential privacy at this point.

[0044] Exit training: When the audit metric AUC and the true positive rate at a false positive rate of 0.1% reach or exceed the preset high-risk threshold, for example, when the AUC reaches 0.8 and the true positive rate at a false positive rate of 0.1% reaches 4%, the model is seriously leaking data membership relationships, posing a high privacy risk. Training should be stopped to prevent further privacy exposure.

[0045] In summary, this invention constructs an efficient, dynamic, and low-overhead privacy auditing scheme by introducing a proactive auditing mechanism on the client side and combining it with techniques such as round-by-round metric collection, lightweight difference discrimination, and multi-feature fusion classification. This scheme not only tracks privacy risks in real time during training but also provides refined and actionable risk quantification results, thereby significantly improving the privacy protection level of remote machine learning in a trusted execution environment.

[0046] This invention is the first to propose a client-side proactive auditing framework in a trusted execution environment. This perspective fully utilizes client-side observable information, such as white-box model states, the entire training trajectory, and real member labels, significantly improving information utilization and auditing accuracy.

[0047] This invention innovatively incorporates four types of metrics—loss, confidence, entropy, and rescaled logit—into a unified audit framework and models them in a round-by-round sequence. The combination of metrics covers multiple dimensions of model learning, including "optimization speed," "output determinism," "uncertainty level," and "class discrimination."

[0048] This invention proposes two differentiation metrics that do not require shadow models or gradients. The first is the rate of change difference, which characterizes the difference in convergence speed between members and non-members during training. The second is the cumulative fluctuation difference, which characterizes the difference in stability and fluctuation amplitude between members and non-members during training. The discrimination process requires only simple arithmetic operations and linear fitting, resulting in extremely low overhead and suitability for low-latency and high-throughput scenarios.

[0049] This invention expands the two types of differential features mentioned above to four categories, forming an eight-dimensional audit vector, based on a single indicator. Combined with a Support Vector Machine (SVM) classifier, a multi-signal fusion-based enhanced auditor is constructed, effectively improving robustness and generalization ability. Simultaneously, clients can utilize their existing real "member / non-member" labels to customize the training and updates of the SVM auditor.

[0050] The auditing logic of this invention is implemented entirely on the client side, without altering the training process within the TEE. The server side merely acts as a provider of computing power and interfaces, achieving plug-and-play compatibility with existing TEE-ML services.

[0051] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.

Claims

1. A method for privacy auditing of machine learning in a trusted execution environment, characterized in that, The method, which involves applying a client, includes: Upload the private dataset to a remote machine learning service deployed in a trusted execution environment to train the target model and obtain the private data. The metrics used in each training epoch of the target model include one or more of the following: loss value, confidence of the correct class, prediction entropy, and raw Logit value. By combining the metric information from each training round, a metric sequence for multiple training rounds is generated; Extracting dynamic features from the metric sequence; Based on this dynamic characteristic, the private data is obtained through classification. Membership prediction results across multiple training rounds; Based on the member relationship prediction results in multiple training rounds, determine whether to continue training, pause training, or withdraw from training.

2. The method according to claim 1, characterized in that, The dynamic characteristics include: the rate of change of the measurement sequence and / or the cumulative fluctuation of the measurement sequence.

3. The method according to claim 2, characterized in that, Extract the rate of change of the measurement sequence, including: The least squares method is used to fit the metric sequence to a first-order linear model. ;in, Represents training rounds, Represents the intercept term and the rate of change. Mean over time Measure the mean of a sequence , This indicates the total number of training rounds for the metric sequence. Indicates training rounds The measurement value; The rate of change As a measure of the rate of change of a sequence.

4. The method according to claim 2, characterized in that, The cumulative fluctuation of the metric sequence ;in, Represents training rounds, This indicates the total number of training rounds for the metric sequence. Indicates training rounds The metric value.

5. The method according to claim 1, characterized in that, Based on this dynamic characteristic, the private data is obtained through classification. Membership prediction results across multiple training rounds include: A labeled training set is constructed using real member samples and non-member samples known to the client. A binary classification auditing model is trained on the training set based on a machine learning algorithm; wherein the machine learning algorithm includes: support vector machine; Based on this dynamic feature, an audit feature vector is generated, and this audit feature vector is fed into the binary classification audit model to obtain the private data. Membership prediction results across multiple training rounds.

6. The method according to claim 1, characterized in that, Based on the member relationship prediction results across multiple training rounds, determine whether to continue training, pause training, or exit training, including: Based on the member relationship prediction results in multiple training rounds, audit metrics are generated; wherein, the audit metrics include: AUC value and true positive rate at a false positive rate of 0.1%; If the AUC value is less than a first set threshold, or if the true positive rate at a false positive rate of 0.1% is less than a third set threshold, training continues. Training is paused when the AUC value is greater than a first set threshold and less than a second set threshold, or when the true positive rate at a false positive rate of 0.1% is greater than a third set threshold and less than a fourth set threshold. Training is paused if the AUC value is greater than a second set threshold, or if the true positive rate at a false positive rate of 0.1% is greater than a fourth set threshold.

7. A machine learning privacy auditing system in a trusted execution environment, characterized in that, The system includes: The information acquisition module is used to upload private datasets to a remote machine learning service deployed in a trusted execution environment for training the target model and acquiring private data. The metrics used in each training epoch of the target model include one or more of the following: loss value, confidence of the correct class, prediction entropy, and raw Logit value. The sequence generation module is used to integrate the metric information from each training round and generate metric sequences for multiple training rounds. The feature extraction module is used to extract dynamic features of the metric sequence; The relationship prediction module is used to classify based on this dynamic feature to obtain the private data. Membership prediction results across multiple training rounds; The privacy audit module is used to determine whether to continue training, pause training, or exit training based on the member relationship prediction results in multiple training rounds.

8. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the machine learning privacy auditing method in a trusted execution environment as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the machine learning privacy auditing method in a trusted execution environment as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product is run on a computer device, the computer device performs the machine learning privacy auditing method in a trusted execution environment as described in any one of claims 1-6.

Images