Privacy training data leakage risk black box detection method and device for classification model

Abstract

The invention discloses a classification model privacy disclosure risk black box detection method and device, and relates to the technical field of machine learning security, and the method comprises the steps: constructing an auxiliary data set; generating an auxiliary model covering different privacy risk levels; training a shadow model and a black box member reasoning attack model corresponding to the shadow model; evaluating the privacy risk score of each auxiliary model by using the attack model, and labeling a high-risk / low-risk label; calculating a correction prediction entropy difference, and screening the first k samples with the maximum difference to form a query data set; extracting a prediction result of each auxiliary model on the query set, calculating and correcting a prediction entropy and a mean value, a variance, a kurtosis and a skewness thereof, and splicing into a k + 4-dimensional risk feature vector; training a risk detection classifier; similar features of the model to be detected are extracted and input into the classifier, and the classifier performs five-level privacy disclosure risk level mapping according to the final risk probability prediction value and outputs the five-level privacy disclosure risk level mapping. The method is mainly applied to risk assessment of classification models in high-privacy sensitive fields such as financial credit investigation and medical diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning security. More specifically, the present application relates to a method and device for detecting the privacy training data leakage risk of a black box of a classification model. BACKGROUND

[0002] With the advancement of big data and artificial intelligence technologies, machine learning models related to classification tasks have been widely applied in the financial field, such as credit card fraud detection and bank credit scoring prediction. These models analyze a large amount of historical transaction data to identify potential fraudulent behavior or abnormal transactions, effectively improving the security and stability of the financial system. However, such models also face significant privacy security risks, with member inference attacks being particularly typical. This attack analyzes the model's prediction output for specific data to infer whether the data sample has been used for model training, which may leak sensitive privacy information of users, such as whether a transaction user belongs to a specific group (e.g., fraud victims). Such attacks not only threaten individual privacy, but also may cause large-scale data breaches. In the financial field, this may lead to severe economic losses and reputation crises. Therefore, it is urgent to develop a detection device that can identify whether a model has a high privacy training data leakage risk. In actual detection scenarios, the detection party usually cannot obtain the specific samples of the privacy training data used by the model to be tested. Therefore, a black box detection method that does not rely on the original training data of the model to be tested is needed to evaluate its privacy leakage risk. SUMMARY

[0003] In view of the significant differences in prediction uncertainty of classification prediction models with different risk levels on test data, the present application proposes a black box detection method and device for privacy leakage risk of a classification model. This method analyzes the model's prediction results on query data, calculates the prediction uncertainty of each sample, and then extracts the risk features of the model. To improve detection efficiency, relevant statistical indicators are further introduced to enhance the initially extracted features. In addition, the present application trains a batch of auxiliary models with diverse feature distributions using auxiliary data, uses the performance of these models on query data to extract risk features, and labels their risk levels through member inference attacks, thereby constructing a training set for supervised learning. Based on this training set, a classifier is trained that can effectively distinguish between high and low risk models to achieve black box risk assessment of the target model. Through the above method, the present application significantly improves the accuracy and robustness of the risk detection device in identifying privacy leakage risks, providing more reliable risk assessment support for the model to be tested.

[0004] Solve the problem of insufficient privacy leakage risk detection accuracy when the model training data and member labels cannot be obtained in a black box scenario. The present application is implemented according to the following technical solutions: A black box detection method for privacy leakage risk of a classification model, comprising the following steps: 1) Collecting a model to be detected and public data conforming to the input and output format thereof, and constructing an auxiliary data set; 2) Training a series of auxiliary models with different privacy risk levels by using the auxiliary data set, through multiple training data sampling and changing training parameters; 3) Training a shadow model according to the manner of step 2) by using the auxiliary data set, and training a black box membership inference attack model based on the predicted output and the real label thereof; 4) Attacking each auxiliary model by using the membership inference attack model of step 3) to obtain a privacy risk score s; comparing s with a preset threshold τ, if s > τ, marking as high risk, otherwise as low risk; 5) For each sample in the auxiliary data set, calculating the difference in corrected prediction entropy of the sample on the auxiliary models marked as high risk and low risk; selecting the first k samples with the largest difference to form a query data set; 6) Querying each auxiliary model by using the query data set to obtain the prediction probability of the k samples; calculating the corrected prediction entropy of each sample, and the mean, variance, kurtosis and skewness thereof; and splicing the k entropy values and the four statistical values into a risk feature vector of k+4 dimensions; 7) Training a risk detection classifier by using the risk feature vectors of the auxiliary models and the risk labels thereof as training samples; 8) Querying the model to be detected by using the query data set, extracting the risk feature vector thereof according to step 6), and inputting the risk level output by the classifier of step 7).

[0005] Preferably, in the black box detection method for privacy leakage risk of a classification model, the training of auxiliary models with different privacy risk levels in step 2) comprises: Systematically discretely sampling random seeds, training data set sizes and model fitting parameters within a predefined parameter space to generate an independent training configuration set covering the whole state domain from underfitting to overfitting; Initializing a machine learning model with the same structure as the model to be detected for each training configuration; Controlling the number of training rounds to force the model to be in one of the underfitting state, normal fitting state or overfitting state; The machine learning model is trained using the auxiliary dataset to generate an auxiliary model group with a continuous distribution of privacy leakage risk levels from low to high, wherein configurations below the expected performance of the model (below expectation) correspond to low risk levels, and configurations reaching or exceeding the expected performance (above expectation) correspond to high risk levels. "Below expectation": refers to controlling training parameters (such as reducing the number of training rounds, limiting data size) to make the model perform sub-optimally (such as underfitting), at which time the privacy leakage risk is low. "Above expectation": refers to optimizing parameters (such as extending the training rounds, increasing the data volume) to make the model reach or exceed the expected performance (such as overfitting), at which time the privacy leakage risk is high.

[0006] Preferably, in the classification model privacy leakage risk black box detection method, the method of training the shadow model and the member inference attack model in step 3) comprises: According to the method of step 2), the shadow model is generated by non-overlapping sampling from the unused training parameter combinations of the auxiliary model, so that the shadow model has differences with the auxiliary model, and the framework of the shadow model is consistent with the to-be-detected model; The prediction probability vector of the shadow model on the training data and the test data is extracted, and is concatenated with the true label of the sample to form a member discrimination feature set; Based on the member discrimination feature set, a black box member inference attack model capable of identifying leakage patterns is generated.

[0007] Preferably, in the classification model privacy leakage risk black box detection method, in step 4), the member inference attack model is used to perform member inference attack on the training dataset and the test dataset of the auxiliary model; The AUROC value of the attack model in distinguishing training members from test non-members is calculated as the privacy risk score s; Based on the type of the to-be-detected model, a dynamic risk threshold τ is set, including: a) When the to-be-detected model is applied to a high privacy sensitive field, a strict threshold strategy is adopted: τ takes a preset value in the interval [0.6, 0.7]; b) When the to-be-detected model is applied to a medium privacy sensitive field, a benchmark threshold strategy is adopted: τ takes a preset value in the interval [0.55, 0.6); c) When the to-be-detected model is applied to a low privacy sensitive field, a lenient threshold strategy is adopted: τ takes a preset value in the interval (0.5, 0.55); Wherein the high privacy sensitive field includes financial credit investigation and medical diagnosis model; the medium privacy sensitive field includes commercial recommendation system; the low privacy sensitive field includes public image classification model; s is compared with the dynamic risk threshold τ; When s>τ, it is marked as high risk, otherwise it is marked as low risk.

[0008] Preferably, in the classification model privacy leakage risk black box detection method, in step 5), for each sample in the auxiliary data set, the weighted aggregation mechanism is used to calculate the modified prediction entropy comprehensive value of each sample on all high-risk auxiliary models and the modified prediction entropy comprehensive value on all low-risk auxiliary models, wherein the entropy value contribution degree of each model is proportional to the risk score proportional weight coefficient. The absolute difference value and the symbolic direction value of the high-risk comprehensive value and the low-risk comprehensive value are calculated by a bidirectional difference algorithm to generate a difference intensity index with direction; The difference intensity index is standardized to eliminate the influence of model population size difference; Based on the descending order of the standardized difference intensity index, the first k samples are screened to construct a query data set.

[0009] Preferably, in the classification model privacy leakage risk black box detection method, in step 6), the query data set is used to perform black box query on each auxiliary model to obtain the prediction probability value of all query samples; The modified prediction entropy of each query sample is calculated; The mean, variance, kurtosis and skewness of the modified prediction entropy of all query samples are calculated; The modified prediction entropy values of the k samples and the four statistical values are spliced in order to form a risk feature vector with k+4 dimensions.

[0010] Preferably, in the classification model privacy leakage risk black box detection method, the risk feature vector corresponding to each auxiliary model is taken as the input feature, and the binary risk label is taken as the supervision signal to generate the final risk probability prediction value; The classifier maps the final risk probability prediction value to five levels of privacy leakage risk levels and outputs, and the mapping rule is: The risk probability value is in [0, 0.25), which corresponds to level 1 risk, The risk probability value is in [0.25, 0.5), which corresponds to level 2 risk, The risk probability value is in [0.5, 0.75), which corresponds to level 3 risk, The risk probability value is in [0.75, 0.9), which corresponds to level 4 risk, The risk probability value is in [0.9, 1.0], which corresponds to level 5 risk.

[0011] Preferably, in the classification model privacy leakage risk black box detection method, in step 8), the risk feature vector extracted by the to-be-tested model is mapped to the auxiliary model feature space by a feature alignment converter to eliminate the feature distribution deviation caused by the model architecture difference; The time-domain stability enhancement module is used for multiple rounds of repeated query and sliding average filtering processing on the feature vector, to generate an anti-noise optimized robust feature representation; The optimized feature representation is input into a risk detection classifier to obtain an initial risk probability value; The prediction reliability is verified based on a confidence threshold filtering mechanism: when the confidence of the risk probability value is lower than a preset threshold, a supplementary query process is started to add differentiated samples to recalculate the features; Finally, a five-level privacy leakage risk level and corresponding confidence report are output.

[0012] Preferably, in the privacy leakage risk black box detection method of the classification model, the auxiliary data set construction in step 1) includes: When the user cannot provide sufficient public data conforming to the model input and output, a synthetic data enhancement engine is started: A synthetic sample aligned with the training data distribution of the to-be-tested model is created through a generative adversarial network, and a distribution correction mechanism based on model prediction feedback is injected during the generation process; The feature space of the public data is migrated to the privacy data domain using domain adaptation technology, eliminating the cross-domain distribution bias; The quality of the synthetic sample is verified, and only the effective sample with a model prediction confidence higher than a preset threshold and complete feature coverage is retained; The effective synthetic sample and the available public data are fused to construct an enhanced auxiliary data set, ensuring that it covers all decision boundary features of the to-be-tested model.

[0013] The privacy training data leakage risk black box detection device of the classification model for implementing the method includes: An auxiliary data set construction module is used to collect the to-be-tested model and public data conforming to its input and output format, and construct an auxiliary data set; An auxiliary model training module is connected to the auxiliary data set construction module, and is used to train a series of auxiliary models with different privacy risk levels through multiple training data sampling and parameter adjustment; An attack model generation module is connected to the auxiliary data set construction module and the auxiliary model training module, and is used to train a shadow model and generate a black box membership inference attack model based on the prediction output and the true label thereof; A risk label labeling module is connected to the attack model generation module and the auxiliary model training module, and is used to attack each auxiliary model using the membership inference attack model to obtain a privacy risk score s, and mark a high risk / low risk label by comparing s with a preset threshold τ; A query data set generation module connected to the auxiliary data set construction module and the risk label marking module, configured to calculate the difference in the corrected prediction entropy of the samples in the auxiliary data set on the high / low risk auxiliary model, and select the top k samples with the largest difference to construct a query data set; A risk feature extraction module connected to the query data set generation module and the auxiliary model training module, configured to obtain the prediction results of each auxiliary model through the query data set, calculate the corrected prediction entropy and its mean, variance, kurtosis and skewness, and splice them into a k+4-dimensional risk feature vector; A risk classifier training module connected to the risk feature extraction module and the risk label marking module, configured to train a risk detection classifier based on the risk feature vector of the auxiliary model and the risk label; A to-be-tested model evaluation module connected to the query data set generation module and the risk classifier training module, configured to extract the risk feature vector of the to-be-tested model and input it into the risk detection classifier to output a five-level privacy leakage risk level.

[0014] The present application at least includes the following beneficial effects: Firstly, the present application introduces prediction uncertainty and its statistical quantities as risk features, which not only enhances the feature expression ability, but also effectively improves the discrimination accuracy of the model privacy leakage risk, achieving efficient risk evaluation in a black box environment. Secondly, the present application adopts the cooperative design of multiple auxiliary models and shadow models, automatically labels the models in combination with member reasoning attacks, constructs a real and diverse training sample set, provides high-quality supervision signals for the risk detection classifier, and significantly improves the detection performance and stability. Thirdly, in the query data selection process, the present application selects the samples with the most distinguishing ability by analyzing the difference in the corrected prediction entropy of the high-risk and low-risk models in sample prediction, thereby reducing the size of the query set, reducing redundant calculation, and improving the overall detection efficiency. Fourthly, the present application realizes model risk evaluation in a completely black box setting, without accessing the training parameters, internal structure or training process of the model, but can accurately predict the privacy risk level of the model, has good practicability and deployability, and is especially suitable for third-party model security audit and privacy compliance inspection scenes.

[0015] The present application realizes high-precision privacy risk evaluation under the condition of zero training data access by constructing an auxiliary model pedigree and an automatic risk labeling mechanism, significantly improves the detection robustness in a black box scenario, and is especially suitable for high-sensitive fields such as finance and medicine. Other advantages, objects and features of the present application will be partly embodied in the following description, and partly understood by those skilled in the art through research and practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 It is the overall framework diagram in one of the schemes of the present application. DETAILED DESCRIPTION

[0017] The present application is further described in detail to enable those skilled in the art to carry it out as referred to in the description.

[0018] In one of the schemes of the present application, a classification model privacy leakage risk black box detection method comprises the following steps: 1) Collect the model to be detected and public data conforming to its input and output format to construct an auxiliary data set; 2) Use the auxiliary data set to train a series of auxiliary models with different privacy risk levels through multiple training data sampling and changing training parameters; 3) Train a shadow model according to step 2) using the auxiliary data set, and train a black box member inference attack model based on its predicted output and real labels; 4) Attack each auxiliary model with the member inference attack model of step 3) to obtain a privacy risk score s; compare s with a preset threshold τ, if s>τ, mark it as high risk, otherwise as low risk; 5) For each sample in the auxiliary data set, calculate the difference in corrected prediction entropy of the sample on the auxiliary models marked as high risk and low risk; select the top k samples with the largest difference to form a query data set; 6) Query each auxiliary model with the query data set to obtain the prediction probability of the k samples; calculate the corrected prediction entropy, mean, variance, kurtosis and skewness of each sample; and concatenate the k entropy values and the four statistical values into a k+4-dimensional risk feature vector; 7) Combine the risk feature vectors of the auxiliary models and their risk labels to form a training sample set, and train a risk detection classifier; 8) Query the model to be detected with the query data set, extract its risk feature vector according to step 6), and input it into the classifier of step 7) to output the risk level. Through the above method, the present application realizes black box risk assessment without accessing training data or model internal structure, especially suitable for high privacy sensitive fields such as finance and medical treatment, and provides reliable protection for model security deployment.

[0019] In one of the schemes of the present application, as a preferred, step 2) training auxiliary models with different privacy risk levels comprises: Systematically discretely sample random seeds, training data set sizes and model fitting parameters within a predefined parameter space to generate an independent training configuration set covering the full state domain from underfitting to overfitting; Initialize a machine learning model with the same structure as the model to be detected for each training configuration; Force the model to be in one of underfitting, normal fitting or overfitting state by controlling the number of training rounds; The machine learning model is trained using an auxiliary data set to generate an auxiliary model group with a continuous distribution of privacy leakage risk levels from low to high, wherein a configuration lower than the expected performance of the model (below expectation) corresponds to a low risk level, and a configuration reaching or exceeding the expected performance (above expectation) corresponds to a high risk level. "Below expectation": refers to making the model perform sub-optimally (such as underfitting) by controlling training parameters (such as reducing the number of training rounds, limiting the data size), at which time the privacy leakage risk is low. "Above expectation": refers to making the model reach or exceed the expected performance (such as overfitting) by optimizing parameters (such as extending the training rounds, increasing the data size), at which time the privacy leakage risk is high. The present application can accurately and efficiently evaluate the privacy leakage risk of a classification model while protecting the intellectual property rights of the model and the privacy of the training data, providing an important guarantee for the secure deployment and application of the model.

[0020] In one of the schemes of the present application, as a preferred, the method of training the shadow model and the member inference attack model in step 3) comprises: According to the method of step 2), a shadow model is generated by non-overlapping sampling from the unused training parameter combinations of the auxiliary model, so that the shadow model and the auxiliary model have differences, and the framework of the shadow model is consistent with the to-be-detected model; The prediction probability vector of the shadow model on the training data and the test data is extracted, and is spliced with the true label of the sample to form a member discrimination feature set; Based on the member discrimination feature set, a black-box member inference attack model capable of identifying leakage patterns is generated.

[0021] In one of the schemes of the present application, as a preferred, the member inference attack model is used to perform member inference attack on the training data set and the test data set of the auxiliary model; The AUROC value of the attack model in distinguishing training members from test non-members is calculated as a privacy risk score s; A dynamic risk threshold τ is set based on the type of the to-be-detected model, including: a) When the to-be-detected model is applied to a high-privacy-sensitive field, a strict threshold strategy is adopted: τ takes a preset value in the interval [0.6, 0.7]; b) When the to-be-detected model is applied to a medium-privacy-sensitive field, a benchmark threshold strategy is adopted: τ takes a preset value in the interval [0.55, 0.6); c) When the to-be-detected model is applied to a low-privacy-sensitive field, a relaxed threshold strategy is adopted: τ takes a preset value in the interval (0.5, 0.55); Wherein the high-privacy-sensitive field includes financial credit investigation and medical diagnosis model; the medium-privacy-sensitive field includes commercial recommendation system; the low-privacy-sensitive field includes public image classification model; comparing s with the dynamic risk threshold τ; When s>τ, it is marked as high risk, otherwise it is marked as low risk. The AUROC value of the auxiliary model is calculated by the member inference attack model as a privacy risk score, quantifying the risk of model leakage. According to the application field sensitivity, the risk threshold is dynamically adjusted to realize differentiated risk assessment strategy. High sensitivity field adopts strict threshold, medium sensitivity field adopts benchmark threshold, and low sensitivity field adopts loose threshold, ensuring that the evaluation result matches the actual demand, and improving the applicability and accuracy of detection.

[0022] In one of the schemes of the present application, as preferred, in step 5), for each sample in the auxiliary data set, the modified prediction entropy comprehensive value of each sample on all high-risk auxiliary models and the modified prediction entropy comprehensive value on all low-risk auxiliary models are calculated respectively by using a weighted aggregation mechanism, wherein the entropy value contribution degree of each model is positively correlated with the weight coefficient proportional to its risk score; The absolute difference value and the signed direction value of the high-risk comprehensive value and the low-risk comprehensive value are calculated by a bidirectional difference algorithm to generate a difference intensity index with directionality; The difference intensity index is standardized to eliminate the influence of model population size difference; The first k samples are screened in descending order based on the standardized difference intensity index to construct a query data set. By calculating the difference of the modified prediction entropy of the sample on the high-risk and low-risk models, the query sample with the most distinguishing ability is selected. This differentiated sample selection method can effectively highlight the model risk features, reduce redundant queries, and improve detection efficiency. The weighted aggregation mechanism ensures that the contribution degree of different risk level models is proportional to their risk level, enhancing the scientificity and reliability of feature extraction.

[0023] In one of the schemes of the present application, as preferred, in step 6), the query data set is used to perform black-box query on each auxiliary model to obtain the prediction probability value of all query samples; The modified prediction entropy of each query sample is calculated; The mean, variance, kurtosis and skewness of the modified prediction entropy of all query samples are calculated; The modified prediction entropy values of the k samples and the four statistical values are sequentially spliced into a k+4-dimensional risk feature vector. The prediction probability of the auxiliary model is obtained by the query data set, the modified prediction entropy and its statistical features are calculated, and a multi-dimensional risk feature vector is constructed. This feature extraction method can comprehensively reflect the prediction uncertainty distribution of the model and capture subtle risk differences. The introduction of mean, variance, kurtosis and skewness, etc. further enriches the feature expression ability and improves the discrimination accuracy of the classifier.

[0024] In one of the schemes of the present application, as a preferred, the risk feature vector corresponding to each auxiliary model is taken as the input feature, and the binary risk label is taken as the supervision signal to generate the final risk probability prediction value; The classifier maps the final risk probability prediction value to five levels of privacy leakage risk levels and outputs, and the mapping rule is: The risk probability value in [0, 0.25) corresponds to level 1 risk, The risk probability value in [0.25, 0.5) corresponds to level 2 risk, The risk probability value in [0.5, 0.75) corresponds to level 3 risk, The risk probability value in [0.75, 0.9) corresponds to level 4 risk, The risk probability value in [0.9, 1.0] corresponds to level 5 risk. The risk feature vector is combined with the binary risk label to train the risk detection classifier, and the quantification evaluation of the model privacy leakage risk is realized. The five-level risk level mapping provides more detailed risk evaluation results, which can better meet the needs of different scenarios than simple binary classification. The hierarchical output of the probability prediction value is convenient for users to intuitively understand the risk degree and provides a reliable basis for decision-making.

[0025] In one of the schemes of the present application, as a preferred, in step 8), the risk feature vector extracted from the to-be-tested model is mapped to the auxiliary model feature space through the feature alignment converter to eliminate the feature distribution deviation caused by the model architecture difference; The time domain stability enhancement module is adopted to perform multiple rounds of repeated query and sliding average filtering processing on the feature vector to generate robust feature representation resistant to noise optimization; The optimized feature representation is input into the risk detection classifier to obtain an initial risk probability value; The prediction reliability is verified based on the confidence threshold filtering mechanism: when the confidence of the risk probability value is lower than the preset threshold, a supplementary query process is started to add differentiated samples to recalculate the features; The five-level privacy leakage risk level and the corresponding confidence report are finally output. The feature alignment converter eliminates the feature distribution deviation caused by the model architecture difference, ensuring the consistency of the evaluation results. The time domain stability enhancement module improves the feature robustness through multiple rounds of query and sliding average filtering, and reduces the noise interference. The confidence threshold filtering mechanism verifies the prediction reliability, and if necessary, a supplementary query process is started to ensure the accuracy and stability of the evaluation results.

[0026] In one of the schemes of the present application, as a preferred, the auxiliary data set is constructed in step 1) to include: When the user cannot provide sufficient public data conforming to the model input and output, the synthetic data enhancement engine is started: A synthetic sample aligned with the training data distribution of the to-be-tested model is created by a generative adversarial network, and a distribution correction mechanism based on model prediction feedback is injected during the generation process; The feature space of public data is migrated to the privacy data domain by using domain adaptation technology, and the cross-domain distribution offset is eliminated. Quality verification is performed on the synthetic sample, and only effective samples with a model prediction confidence higher than a preset threshold and complete feature coverage are retained. The effective synthetic sample and the available public data are fused to construct an enhanced auxiliary data set, ensuring that it covers all decision boundary features of the to-be-tested model. The generative adversarial network and the domain adaptation technology are used to construct the enhanced auxiliary data set, solving the problem of insufficient public data. The synthetic data is aligned with the training data distribution of the to-be-tested model, ensuring the training effect of the auxiliary model. The quality verification mechanism selects effective samples to ensure complete decision boundary coverage and improves the comprehensiveness and accuracy of subsequent risk assessment.

[0027] The privacy training data leakage risk black box detection device for implementing the classification model of the method comprises: An auxiliary data set construction module is configured to collect a to-be-detected model and public data conforming to the input and output format of the to-be-detected model, and construct an auxiliary data set. An auxiliary model training module is connected to the auxiliary data set construction module and is configured to train a series of auxiliary models with different privacy risk levels through multiple training data sampling and parameter adjustment. An attack model generation module is connected to the auxiliary data set construction module and the auxiliary model training module and is configured to train a shadow model and generate a black box membership inference attack model based on the predicted output and the true label of the shadow model. A risk label labeling module is connected to the attack model generation module and the auxiliary model training module and is configured to obtain a privacy risk score s by attacking each auxiliary model using the membership inference attack model, and mark a high / low risk label by comparing s with a preset threshold τ. A query data set generation module is connected to the auxiliary data set construction module and the risk label labeling module and is configured to calculate the difference in corrected prediction entropy of samples in the auxiliary data set on the high / low risk auxiliary model, and select the top k samples with the largest difference to construct a query data set. A risk feature extraction module is connected to the query data set generation module and the auxiliary model training module and is configured to obtain the prediction results of each auxiliary model through the query data set, calculate the corrected prediction entropy and its mean, variance, kurtosis, and skewness, and splice them into a k+4-dimensional risk feature vector. A risk classifier training module is connected to the risk feature extraction module and the risk label labeling module and is configured to train a risk detection classifier based on the risk feature vector of the auxiliary model and the risk label. The to-be-tested model evaluation module is connected with the query data set generation module and the risk classifier training module, and is used for extracting a risk feature vector of the to-be-tested model and inputting the risk detection classifier to output a five-level privacy leakage risk level. The full-process automation of the classification model privacy leakage risk detection is realized through the modular design. The function modules work cooperatively to form a complete closed loop from data construction, model training to risk evaluation. The device design is convenient for deployment and application, and can quickly respond to detection requirements of different scenes, and provides an efficient tool for model security audit.

[0028] The application discloses a kind of classification model privacy leakage risk black box detection method and device. The method is by constructing auxiliary data set, a series of auxiliary models are trained based on the data set, to simulate the model prediction behavior of different privacy risk levels. Subsequently, the actual privacy risk index of each auxiliary model is obtained using member reasoning attack method, and the risk label is assigned to the auxiliary model accordingly. On this basis, the samples in the auxiliary data set are screened, and a query data set is constructed. The query data set is used to query all auxiliary models, obtain the prediction results on the query samples, and extract the corresponding risk features. The extracted risk features and the corresponding risk labels form a training sample set, which is used to train a risk detection classifier. Finally, the query data set is used to query the to-be-tested model, extract its risk features, and input them into the trained risk detection classifier, output its privacy training data leakage risk level, realize the black box privacy risk detection and evaluation of target model. The application can effectively complete the privacy risk identification of classification model in the black box scene lacking real member membership information, solve the problem of insufficient black box privacy detection means in the prior art, have good applicability and popularization value, and are beneficial to guarantee the privacy security of training data.

[0029] As shown in Figure 1 The application provides a classification model privacy leakage risk black box detection method, which focuses on the classification model privacy risk detection problem of the black box scene in the real situation where the real member membership state cannot be obtained, and includes the following steps: Step one: collect the to-be-detected model provided by user, and construct auxiliary data set according to the public data meeting the input and output of the model.

[0030] Step two: use the auxiliary data set to change random seed and training data set size, sample training data multiple times, set a series of different random seeds and model fitting parameters (for example, the number of training rounds in neural network), and train a series of auxiliary models with different risks (obtain models with different risk levels by controlling model fitting degree) and sufficient quantity by combining these parameters. When the framework of the auxiliary model is consistent with the to-be-tested model, better detection effect can be obtained.

[0031] A series of auxiliary models with different risk levels are generated in step two by adjusting the training parameters to provide rich reference data for the subsequent detection model.

[0032] Step three: Use the auxiliary dataset to train a batch of models different from the auxiliary models as shadow models (as above, the framework of the shadow model can obtain better detection effect when consistent with the model to be tested) according to the training method of step two, and use the model output probability value and the true label as the member feature of the sample. After the training and test data of the shadow model are labeled as members and non-members respectively, a black box member inference attack model is trained.

[0033] Step four: Use the member inference attack model trained in step three to attack each auxiliary model in turn, and perform member inference attack on their training and test sample sets, and use the attack AUROC value as the real privacy risk score of the model Compare the risk score with the preset risk threshold value slightly higher than the random guess index, for example =0.55) in the device. When the risk score is greater than the threshold, the model risk is marked as 1 (high risk), otherwise as 0 (low risk).

[0034] Step four: The black box member inference attack is performed on each auxiliary model to obtain the real risk label, and the model detection problem is converted into a supervised training problem.

[0035] Step five: With the help of all auxiliary models marked in step four, calculate the difference between the modified prediction entropy of each sample in the auxiliary dataset in the high-risk model and the low-risk model, and finally select the top 1000 samples with the largest difference to form the query dataset.

[0036] In step five, with the help of auxiliary models, select query samples from all auxiliary data that maximize the difference between the high-risk model and the low-risk model prediction modified prediction entropy, so as to highlight the risk characteristics of the model.

[0037] Step six: Use the good query dataset selected in step five to query each auxiliary dataset in turn, calculate the modified prediction entropy of each sample after obtaining the prediction probability value of 1000 samples, and calculate the mean, variance, kurtosis and skewness of all 1000 samples. The uncertainty value of 1000 samples and the four statistical values are spliced to obtain a 1000+4-dimensional vector as the risk feature of the model.

[0038] In step six, the modified prediction entropy result and related statistical values of the query sample are used as the representation and enhancement of the model feature, which essentially utilizes the prediction uncertainty difference between the high-risk model and the low-risk model for differentiation.

[0039] Step seven: the auxiliary model risk features obtained in step six and the auxiliary model risk labels obtained in step four are taken as a training sample set to train a simple machine learning model (such as XGBoost or random forest) as a risk detection classifier.

[0040] By extracting the risk features and labeling the risk labels of the auxiliary model, sufficient and diverse training data are obtained, and a risk detection classifier is trained by supervised learning in step seven to realize the identification of the risk of the target model.

[0041] Step eight: the risk features of the model to be tested are extracted in the manner of step six, and the obtained feature vector is input into the risk detection classifier trained in step seven to infer the risk level. The risk detection method is: the risk level is evaluated by the high risk probability of the model predicted by the risk detection classifier, and the higher the prediction probability, the higher the risk of leakage. The specific risk rating standards are shown in the following table: Table 1 Risk leakage rating standards for classification model privacy training data High risk prediction probability [0, 0.25] (0.25, 0.5] (0.5, 0.75] (0.75, 0.9] (0.9, 1.0] Risk level Level 1 Level 2 Level 3 Level 4 Level 5 Although the embodiments of the present application have been disclosed as above, it is not limited to the application listed in the specification and the embodiments, and can be fully applied to various fields suitable for the present application, and additional modifications can be easily realized by those skilled in the art, and therefore the present application is not limited to specific details and the figures shown and described herein.

Claims

1. A black-box detection method for privacy leakage risk in classification models, characterized in that, Includes the following steps: 1) Collect the model to be tested and publicly available data conforming to its input and output formats, and construct an auxiliary dataset; 2) Using auxiliary datasets, a series of auxiliary models with different privacy risk levels are trained by sampling training data multiple times and changing training parameters. 3) Train the shadow model using the auxiliary dataset as in step 2), and train the black-box member inference attack model based on its predicted output and real labels; 4) Attack each auxiliary model using the member reasoning attack model from step 3) to obtain a privacy risk score s; Compare s with a preset threshold τ. If s > τ, it is marked as high risk; otherwise, it is marked as low risk. 5) For each sample in the auxiliary dataset, calculate the difference in prediction entropy by correcting its prediction results on the auxiliary models labeled as high-risk and low-risk; Select the k samples with the largest differences to form the query dataset; 6) Use the query dataset to query each auxiliary model to obtain the predicted probability of k samples; calculate the corrected prediction entropy of each sample and its mean, variance, kurtosis, and skewness; concatenate the k entropy values ​​with the four statistical values ​​to form a k+4 dimensional risk feature vector; 7) Use the risk feature vectors and risk labels of the auxiliary model to form a training sample set, and train the risk detection classifier; 8) Use the query dataset to query the model to be detected, extract its risk feature vector as in step 6), and input it into the classifier in step 7) to output the risk level.

2. The black-box detection method for privacy leakage risk of classification models as described in claim 1, characterized in that, Step 2) involves training auxiliary models for different privacy risk levels, including: Within a predefined parameter space, the random seed, training dataset size, and model fitting parameters are systematically and discretely sampled to generate an independent training configuration set covering the entire state domain from underfitting to overfitting. For each training configuration, initialize a machine learning model with the same structure as the model to be tested. By controlling the number of training rounds, the model can be forced into one of the following states: underfitting, normal fitting, or overfitting. Machine learning models are trained using auxiliary datasets to generate a group of auxiliary models with a continuous distribution of privacy leakage risk levels from low to high. Configurations that perform below the expected performance of the model correspond to low risk levels, while configurations that perform at or above the expected performance correspond to high risk levels.

3. The black-box detection method for privacy leakage risk of classification models as described in claim 2, characterized in that, The methods for training the shadow model and the member reasoning attack model in step 3) include: Following the method in step 2), a shadow model is generated by selecting a training configuration from the unused training parameter combinations of the auxiliary model through non-overlapping sampling, so that the shadow model is different from the auxiliary model, and the framework of the shadow model is consistent with the model to be detected. Extract the predicted probability vectors of the shadow model for training and test data, and concatenate them with the true labels of the samples using multi-dimensional features to form a member discrimination feature set; Based on the member discrimination feature set, a black-box member inference attack model that can identify leakage patterns is generated.

4. The black-box detection method for privacy leakage risk of classification models as described in claim 1, characterized in that, In step 4), a member inference attack model is used to perform a member inference attack on the training and test datasets of the auxiliary model; The AUROC value used by the attack model to distinguish between training members and test non-members is calculated as the privacy risk score s; A dynamic risk threshold τ is set based on the type of the model to be tested, including: a) When the model under test is applied to a highly privacy-sensitive domain, a strict threshold strategy is adopted: τ takes a preset value in the interval [0.6, 0.7]. b) When the model under test is applied to a moderately privacy-sensitive area, a baseline threshold strategy is adopted: τ is a preset value in the interval [0.55, 0.6); c) When the model under test is applied to a low privacy-sensitive domain, a relaxed threshold strategy is adopted: τ takes a preset value in the interval (0.5, 0.55); High privacy-sensitive areas include financial credit scoring and medical diagnostic models; medium privacy-sensitive areas include commercial recommendation systems; and low privacy-sensitive areas include publicly available image classification models. Compare s with the dynamic risk threshold τ; When s > τ, it is marked as high risk; otherwise, it is marked as low risk.

5. The black-box detection method for privacy leakage risk of classification models as described in claim 1, characterized in that, In step 5), for each sample in the auxiliary dataset, a weighted aggregation mechanism is used to calculate its modified prediction entropy composite value on all high-risk auxiliary models and its modified prediction entropy composite value on all low-risk auxiliary models, where the entropy contribution of each model is positively correlated with its risk score by a weight coefficient. The absolute difference and symbolic directional value between the high-risk composite value and the low-risk composite value are calculated by a two-way difference algorithm to generate a directional difference intensity index. Standardize the difference intensity index to eliminate the impact of differences in model population size; The query dataset is constructed by selecting the top k samples in descending order based on the standardized difference intensity index.

6. The black-box detection method for privacy leakage risk of classification models as described in claim 1, characterized in that, In step 6), a black-box query is performed on each auxiliary model using the query dataset to obtain the predicted probability values ​​of all query samples; Calculate the corrected prediction entropy for each query sample; Calculate the mean, variance, kurtosis, and skewness of the corrected prediction entropy for all query samples; The corrected prediction entropy values ​​of k samples are concatenated with the four statistical values ​​in sequence to form a k+4 dimensional risk feature vector.

7. The black-box detection method for privacy leakage risk of classification models as described in claim 1, characterized in that, The risk feature vector corresponding to each auxiliary model is used as the input feature, and the binary risk label is used as the supervision signal to generate the final risk probability prediction value. The classifier maps and outputs a five-level privacy breach risk level based on the final risk probability prediction value. The mapping rule is as follows: A risk probability value in the range [0, 0.25) corresponds to level 1 risk. A risk probability value in the range [0.25, 0.5) corresponds to a level 2 risk. A risk probability value in the range [0.5, 0.75) corresponds to a level 3 risk. A risk probability value in the range [0.75, 0.9) corresponds to risk level 4. A risk probability value of [0.9, 1.0] corresponds to a level 5 risk.

8. The black-box detection method for privacy leakage risk of classification models as described in claim 1, characterized in that, In step 8), the risk feature vector extracted from the model under test is mapped to the feature space of the auxiliary model by the feature alignment converter to eliminate the feature distribution offset caused by the difference in model architecture; A time-domain stability enhancement module is used to perform multiple rounds of repeated queries and moving average filtering on the feature vector to generate a robust feature representation with noise resistance optimization. The optimized feature representation is input into the risk detection classifier to obtain the initial risk probability value; The reliability of the prediction is verified based on the confidence threshold filtering mechanism: when the confidence of the risk probability value is lower than the preset threshold, the supplementary query process is initiated to add differentiated samples and recalculate the features. The final output is a five-level privacy breach risk level report with corresponding confidence levels.

9. The black-box detection method for privacy leakage risk of classification models as described in claim 1, characterized in that, Step 1) involves constructing the auxiliary dataset, which includes: When users cannot provide sufficient public data that matches the model's input and output, the synthetic data augmentation engine is activated: Synthetic samples aligned with the training data distribution of the model under test are created by generative adversarial networks, with a distribution correction mechanism based on model prediction feedback injected during the generation process. Domain adaptation techniques are used to migrate the feature space of public data to the privacy data domain, eliminating cross-domain distribution shifts. Perform quality verification on the synthesized samples, and retain only the valid samples with model prediction confidence higher than a preset threshold and complete feature coverage; An enhanced auxiliary dataset is constructed by fusing effective synthetic samples with available public data to ensure that it covers all decision boundary features of the model under test.

10. A black-box detection device for privacy training data leakage risk of a classification model used to implement the method of claim 1, characterized in that, include: The auxiliary dataset construction module is used to collect the model to be detected and publicly available data that conforms to its input and output formats to construct the auxiliary dataset; An auxiliary model training module, connected to the auxiliary dataset construction module, is used to train a series of auxiliary models with different privacy risk levels through multiple training data sampling and parameter adjustments. The attack model generation module connects the auxiliary dataset construction module and the auxiliary model training module, and is used to train the shadow model and generate a black-box member inference attack model based on its predicted output and real labels. The risk labeling module connects the attack model generation module and the auxiliary model training module. It is used to attack each auxiliary model using the member reasoning attack model to obtain a privacy risk score s, and to label high-risk / low-risk tags by comparing s with a preset threshold τ. The query dataset generation module connects the auxiliary dataset construction module and the risk labeling module. It is used to calculate the difference in corrected prediction entropy of samples in the auxiliary dataset on the high / low risk auxiliary model, and to select the k samples with the largest differences to construct the query dataset. The risk feature extraction module connects the query dataset generation module and the auxiliary model training module. It is used to obtain the prediction results of each auxiliary model through the query dataset, calculate the corrected prediction entropy and its mean, variance, kurtosis and skewness, and concatenate them into a k+4 dimension risk feature vector. The risk classifier training module connects the risk feature extraction module and the risk label annotation module, and is used to train a risk detection classifier based on the risk feature vector and risk label of the auxiliary model. The model evaluation module is connected to the query dataset generation module and the risk classifier training module. It is used to extract the risk feature vector of the model under test and input it into the risk detection classifier to output a five-level privacy leakage risk level.

Images