Model stealing detection method combining training set data distribution and w distance

Abstract

This invention relates to a model theft detection method that combines training set data distribution and W-distance, belonging to the field of computer and information science and technology. First, the invention utilizes the VAE method to reduce the dimensionality of the training and query sets. Second, it uses maximum likelihood estimation to calculate the probability distribution of the query set, and samples multiple sets of samples to be detected according to this probability distribution. Then, for each set of samples to be detected, the same number of reference samples are randomly sampled from the training set, and the W-distance between each set of samples to be detected and the reference samples is calculated. Finally, the ratio of the number of categories in the reference samples to the total number of categories is used as a weight to calculate all W-distances. When the weighted calculation result is greater than a detection threshold, model theft is detected. This invention proposes a model theft detection method that correlates the training set data distribution, considers the characteristics of both the query set and training set sample distributions, and improves the W-distance calculation method, effectively enhancing the accuracy of model theft detection.

Description

Technical Field

[0001] This invention relates to a model theft detection method that combines training set data distribution and W distance, belonging to the field of computer and information science and technology. Background Technology

[0002] Model theft attacks are malicious acts that steal the functionality of a model or simulate its decision boundaries. The stolen models are often those trained with significant time and money by their owners, possessing substantial commercial value. Once stolen, the model owner's rights are harmed, and the stolen model can provide a springboard for adversarial attacks. Therefore, researching high-accuracy model theft detection methods has significant theoretical and practical value.

[0003] Existing model theft detection methods are primarily based on anomaly detection. To improve attack efficiency, attackers may synthesize samples from a small subset of the training set, making the synthesized samples more closely approximate the classification boundary of the target model. Alternatively, they may fully utilize the target model's prediction vectors as feedback, training alternative models using relevant datasets or randomly generated vectors. Current research focuses on exploring changes in query sample distribution caused by anomalies. For example, based on the premise that "the distance between two randomly selected points in a finite space follows a normal distribution," it analyzes whether the distances of a set of query samples follow a normal distribution; or, it uses the K-nearest neighbor algorithm to determine whether there are too many nearest neighbor samples in the query samples. However, existing detection methods cannot adapt to changes in the model. Once an attacker successfully steals one model, they can use the same method to steal multiple models. Furthermore, the detection criteria are relatively simple, making them easy for attackers to circumvent through repeated attempts, thus rendering them unable to cope with subsequent attacks and significantly impacting detection accuracy.

[0004] In summary, existing model theft detection methods have simple and fixed detection standards, making them easy for attackers to deduce and bypass. Therefore, this invention proposes a model theft detection method that combines the training set data distribution and the W distance. Summary of the Invention

[0005] The purpose of this invention is to address the issue that model theft detection criteria are fixed and easily inferred, and to propose a model theft detection method that combines the training set data distribution and W distance.

[0006] The design principle of this invention is as follows: First, a VAE model is trained using a training dataset, and the output of the training set in the VAE is calculated to form a dimensionality-reduced dataset S. Second, a set of query samples is used as input samples for the VAE model to obtain a dimensionality-reduced dataset S′ of the query samples. The probability distribution of S′ is calculated using maximum likelihood estimation, and k sets of samples to be detected are sampled according to the probability distribution, each set having a capacity of D. Third, for each set of data sampled from S′, a set of reference samples with a capacity of D is randomly sampled from S as a W distance calculation pair, and the W distance between the two sets is calculated, i.e., the Wasserstein distance. Finally, the W distance is calculated by weighting the ratio of the number of reference sample categories to the total number of categories to determine the query behavior.

[0007] The technical solution of the present invention is achieved through the following steps:

[0008] Step 1: Train the VAE model and the target model using the training dataset, and use the VAE model to obtain the dimensionality-reduced dataset S of the training dataset.

[0009] Step 1.1, construct the VAE model framework.

[0010] Step 1.2: Determine the loss function for the VAE model.

[0011] Step 1.3: Encode the training set data using the VAE model to obtain the dimensionality-reduced data, which constitutes the dataset S.

[0012] Step 2: Use the VAE model to reduce the dimensionality of the query data, and calculate the probability distribution of each dimension based on maximum likelihood estimation. Sample multiple sets of data from the probability distribution.

[0013] Step 2.1: Maintain a queue m for the input query samples, with a queue length of D.

[0014] Step 2.2: Use the VAE model to reduce the dimensionality of the input samples to h, add them to queue m, and when m is full, remove the head sample and add the new sample to the tail.

[0015] Step 2.3: The VAE model extracts the feature information of the query samples to the greatest extent possible, and each dimension of the dimensionality-reduced data is independent. Therefore, based on maximum likelihood estimation, the probability density and probability distribution of each dimension of the data in queue m are calculated, and one set of data is sampled from h sets of probability distributions, repeated k times.

[0016] Step 3: For each set of data obtained in Step 2.3, randomly sample data sets of the same size from S as Wasserstein distance calculation pairs.

[0017] Step 4: Calculate the Wasserstein distance for each pair of data groups in Step 3. The final distance W is obtained by weighting and summing the results based on the ratio of the number of classes in the data group to the total number of classes.

[0018] Step 5: Use the final distance W to determine the query behavior.

[0019] Beneficial effects

[0020] Compared to existing model theft detection methods, this invention combines the training set data distribution and the W statistical distribution distance calculation method. First, compared to commonly used KL divergence and JS divergence, the W distance can calculate the distribution distance between two sample sets with minimal overlap, improving detection stability. Second, the method uses the training set distribution as the detection standard. Model training sets have a large number of samples, complex distributions, and are not publicly disclosed; furthermore, different models have different training sets, providing a detection standard that is difficult to bypass easily. Third, the method fully considers the characteristics of model theft, assuming that the sample distribution used in normal queries is approximately the same as that in the training set. Therefore, the method can effectively cope with various model theft attacks. Fourth, calculating the probability distribution of the query sample data before performing multiple samplings fully reflects the distribution characteristics of the query samples, facilitating distribution discrimination. Finally, weighting the calculation of each set of results reduces the impact of large W distance values ​​caused by too few categories, improving detection accuracy. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the model theft detection method based on W distance of the present invention. Detailed Implementation

[0022] To better illustrate the purpose and advantages of the present invention, the implementation methods of the present invention will be further described in detail below with reference to examples.

[0023] Experimental data came from various image classification datasets, including the Fashion-MNIST dataset (10 categories for clothing), the CIFAR10 dataset (10 categories for small objects), the Google Street View SVHN dataset for house numbers, and the GTSRB dataset for traffic signs. The training and test sets were split in a 9:1 ratio for training the target model.

[0024] During the experiment, three attack methods were used: JBDA, KnockoffNets, and MAZE. Each method has its own focus. JBDA uses a small number of training set samples to create synthetic samples, thereby approximating the classification boundary of the target model. KnockoffNets uses a larger dataset that may be related to the target model and uses output feedback to steal the model. MAZE does not use any training set-related samples. It trains a data generator to maximize the information difference between the output vectors of the cloned model and the target model, thereby stealing the model.

[0025] The experiment used the accuracy evaluation model to obtain the detection results. The accuracy calculation method is shown in formula (1):

[0026]

[0027] Wherein, TP is the number of theft behaviors identified as theft behaviors, FN is the number of normal behaviors identified as normal behaviors, FP is the number of normal behaviors identified as theft behaviors, and TN is the number of theft behaviors identified as normal behaviors.

[0028] This experiment was conducted on one computer and one server. The computer's specific configuration was: Intel i7-8750H, CPU 2.20GHz, 8GB RAM, and Windows 10 64-bit operating system; the server's specific configuration was: E7-4820v4, 256GB RAM, and Linux Ubuntu 64-bit operating system.

[0029] The specific procedure for this experiment is as follows:

[0030] Step 1: Train the VAE model and the target model using the training dataset, and use the VAE model to obtain the dimensionality-reduced data of the training dataset, forming the dataset S.

[0031] Step 1.1, construct the VAE model framework.

[0032] Step 1.2, determine the loss function of the VAE model, as shown in formula (2) and formula (3).

[0033]

[0034]

[0035] In the formula, D train For the training dataset, The reconstruction loss of a standard automatic encoder is calculated using D. train For the samples in the model, calculate the expected value of the squared difference before and after passing through the VAE model; pass The data after divergence constraint encoding follows Distribution, calculation method is based on D train The samples in the dataset are used to calculate their encoded distribution and the distribution of the given distribution. Expectation of divergence.

[0036] Step 1.3: Encode the training set data using the VAE model to form dataset S.

[0037] Step 2: Use the VAE model to reduce the dimensionality of the query data, and calculate the probability distribution of the query samples based on the maximum likelihood estimation. Multiple sets of data are then sampled from the probability distribution.

[0038] Step 2.1: Maintain a queue m for the input query samples, with a queue length of D.

[0039] Step 2.2: When a query sample is input, the sample is reduced to a vector of dimension h through the VAE model and added to queue m. When m is full, the sample at the head of the queue is removed and the new sample is added to the tail of the queue. When m is full, a query behavior judgment is performed once for each input sample, thereby realizing real-time detection of model theft.

[0040] Step 2.3: The VAE model can fully extract sample features, and each dimension of the dimensionality-reduced sample is independent. Therefore, the probability density can be calculated separately for each dimension. It is assumed that each dimension of the data follows a certain set of parameter distributions. Sample data is Find the parameter θ that maximizes formula (4), calculate the probability density of the query sample, and then calculate the probability distribution of each dimension of the query sample.

[0041]

[0042] Step 2.4, respectively at each P i We perform one sampling to obtain one set of data, repeat this process k times to obtain k sets of data, denoted as .

[0043] Step 3, for each A i A random sample is taken from dataset S, with a sample size of D. The resulting dataset is denoted as B. i .

[0044] Step 4: Calculate A according to the Wasserstein distance formula. i and B i The Wasserstein distance between W i and to W i Perform a weighted summation.

[0045] Step 4.1, calculate A according to formula (5). i and B i The Wasserstein distance between W i .

[0046]

[0047] Step 4.2, let T be the total number of classes in the training dataset, and B be the total number of classes. i The total number of data categories included is t iThe weighted Wasserstein distance W is calculated according to formula (6).

[0048]

[0049] Step 5: Set the detection threshold δ and determine the query behavior.

[0050] Step 5.1: The threshold δ is selected such that the false positive rate of the detector for normal query behavior is equal to 0.5%.

[0051] Step 5.2: Compare W and δ. When W>δ, it can be considered that the difference between the distribution of the query sample and the distribution of the training set sample is too large, the query behavior does not conform to the characteristics of normal query behavior, and the query behavior is determined to be model theft.

[0052] Test Results: The experiment, based on the W-distance model theft detection method, detected three attack methods (JBDA, KnockoffNets, and MAZE) and normal query behavior. The present invention can accurately detect the three attack behaviors and normal query behavior, achieving an accuracy of 97.3% in model theft detection, demonstrating excellent detection performance.

[0053] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A model theft detection method combining training set data distribution and W distance, characterized in that... The method includes the following steps: Step 1: Train the VAE model and the target model using the training dataset, and use the VAE model to obtain the dimensionality-reduced dataset S of the training dataset. Step 1.1, construct the VAE model framework. Step 1.2, determine the loss function of the VAE model. Step 1.3: Encode the training set data using the VAE model to obtain the dimensionality-reduced data, forming dataset S. Step 2: Use the VAE model to reduce the dimensionality of the query data, and calculate the probability distribution for each dimension based on maximum likelihood estimation. Multiple sets of data are then sampled from these probability distributions. Step 2.1: Maintain a queue m for the input query samples, with a queue length of D. Step 2.2: Use the VAE model to reduce the dimensionality of the input samples to h, add them to queue m, and when m is full, remove the head sample and add the new sample to the tail of the queue. Step 2.3: The VAE model extracts the feature information of the query samples to the greatest extent possible, and each dimension of the dimensionality-reduced data is independent. Therefore, based on maximum likelihood estimation, the probability density and probability distribution of each dimension of the data in queue m are calculated. One set of data is obtained by sampling from h sets of probability distributions, and this is repeated k times to obtain k sets of samples to be detected, denoted as... Step 3, for each set of data A obtained in Step 2.3 i Randomly sample reference sample group B of the same size from S. i As a W-distance calculation pair, i.e., a Wasserstein distance calculation pair, Step 4: Calculate the Wasserstein distance for each pair of data groups in Step 3, based on B... i The final distance W is obtained by weighting and summing the results of the ratio of the number of categories in the sample to the total number of categories. Step 5: Use the final distance W to determine the query behavior.

2. The model theft detection method combining training set data distribution and W distance according to claim 1, characterized in that: In step 2.3, the probability distribution of each dimension of the dimensionality-reduced query sample is calculated using maximum likelihood estimation, assuming that each dimension of the data follows a certain set of parameter distributions. Sample data is Find the parameter θ that maximizes the following expression. Calculate the probability density of the query sample, and from this, calculate the probability distribution of each dimension of the query sample. Each sample is taken once in each Pi to obtain a set of data. This process is repeated k times to obtain k sets of data.

3. The model theft detection method combining training set data distribution and W distance according to claim 1, characterized in that: Step 4 uses Wasserstein distance to calculate the distributional distance between the two data sets, and performs a weighted calculation based on the ratio of the number of classes in the reference sample to the total number of classes in each data set, as shown in the following formula. A i and B i Let T be the number of all categories in the sampled data set, and t be the number of categories in the dataset. i For B i The number of data categories included.

Images