An adaptive fingerprint-based meta-learning neural network fingerprint detection method

Abstract

The application belongs to the technical field of artificial intelligence, and particularly relates to a meta-learning neural network fingerprint detection method based on self-adaptive fingerprints; the method is suitable for intellectual property protection of a plurality of downstream task models; a plurality of self-adaptive fingerprint samples are constructed for a target model, and then it is detected whether the outputs of the self-adaptive fingerprint samples in a suspect model can pass the detection of a meta verification model; the fingerprint detection mechanism of the application is based on the confidence vector output of the suspect model on the self-adaptive fingerprint samples, rather than the output label of the adversarial sample, and thus is not related to a specific task type and does not depend on training data, and can be widely applied to neural network fingerprint detection of various task types, such as classification, regression, and generation model, to protect the intellectual property of the neural network, and has good robustness and uniqueness for various positive and negative example suspect models.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a meta-learning neural network fingerprint detection method based on adaptive fingerprints. BACKGROUND

[0002] With the continuous development of deep learning technology, the intellectual property protection of deep neural networks has become an important issue. The training of a model usually requires a large amount of computing resources and data samples, and an attacker can steal the model and add post-processing to complete the construction of a pirated model at low cost.

[0003] For model theft, existing intellectual property protection methods mainly include watermark detection and fingerprint detection. Among them, the watermark detection method needs to modify the parameters of the model, which will affect the performance of the model itself to some extent; and the fingerprint detection method based on the inherent properties of the model does not modify the model, but mostly relies on a batch of special adversarial samples, which also leads to the fact that the current fingerprint detection method is often limited to the scene of classification tasks and has dependence on training data samples. SUMMARY

[0004] The application proposes a meta-learning neural network fingerprint detection method based on adaptive fingerprints for model theft attacks. The model theft attack in the application refers to an attacker stealing the model by some means, and using various post-processing skills to confuse the ownership of the model, obtaining a pirated model and releasing it again. The post-processing skills involved include pruning, fine-tuning, retraining, and distillation. The fingerprint detection mechanism of the application is based on the confidence vector output of the suspicious model on the adaptive fingerprint sample, rather than the output label of the adversarial sample, so it is not related to the specific task type and does not depend on the training data. It can be widely applied to neural network fingerprint detection of various task types, such as classification, regression, and generation model, to protect the intellectual property of neural networks, and has good robustness and uniqueness for various positive and negative example suspicious models.

[0005] In the present application, a model owner trains a model (i.e. "target model") and publishes it on a third-party platform to provide services; an attacker steals the model by some means and applies various post-processing techniques to confuse the ownership of the model to obtain a pirated model (i.e. "positive example suspicious model"); and if the training of a model is started from zero and independent of the target model, it is called a "negative example suspicious model". The defender (usually the model owner or a trusted third party) has white-box access to the target model and black-box access to the suspicious model. The present application uses joint training of adaptive fingerprint samples and meta-verification models to detect model fingerprints through the joint output of the suspicious model, and is suitable for intellectual property protection of various downstream task models: first, construct multiple sets of adaptive fingerprint samples for the target model, and then detect whether the output of these adaptive fingerprint samples in the suspicious model can pass the detection of the meta-verification model. The fingerprint detection mechanism is based on the output of the suspicious model on the adaptive fingerprint samples, rather than the output label of the adversarial samples, and is therefore independent of the specific task type and the training data, and can be widely applied to neural network fingerprint detection of various task types, such as classification, regression, and generation models, to protect the intellectual property of neural networks, and the generation of fingerprint samples does not depend on the training data. The output of the adaptive fingerprint samples on the positive and negative example suspicious models has a large difference, so that the output of the meta-verification model changes, and therefore has good uniqueness; at the same time, post-processing of the target model cannot cross the classification boundary of the meta-verification model, and therefore has good robustness.

[0006] The technical solutions of the present application are specifically introduced as follows.

[0007] A meta-learning neural network fingerprint detection method based on adaptive fingerprint includes a model preparation stage, a fingerprint extraction stage, and a fingerprint verification stage.

[0008] In the model preparation stage, multiple positive example suspicious models are obtained through various post-processing techniques based on the trained target model, and multiple negative example suspicious models are obtained by independent training on the same training data set or irrelevant data set; positive and negative example samples are provided for supervised training of adaptive fingerprint samples and meta-verification models;

[0009] In the fingerprint extraction stage, multiple adaptive fingerprint samples and a meta-verification model are constructed based on the target model, and the adaptive fingerprint samples and the meta-verification model are updated and optimized synchronously, so that the joint output of the fingerprint samples on the target model and the positive example suspicious model is always positive on the meta-verification model, and the joint output of the negative example suspicious fingerprint samples is always positive on the meta-verification model;

[0010] In the fingerprint verification stage, the joint output of the extracted suspicious model to be detected on the optimized adaptive fingerprint sample is detected, and whether the suspicious model to be detected steals the target model is judged according to the output of the optimized meta verification model, so as to realize neural network fingerprint detection, and the neural network intellectual property verification party obtains the prediction output of the suspicious model to be detected in a black box scene in this stage.

[0011] In the present application, in the model preparation stage, the post-processing skills include one or more of fine-tuning, pruning or distillation operation.

[0012] In the present application, the target model is a classification model, a regression model or a generation model.

[0013] In the present application, in the fingerprint extraction stage, N adaptive fingerprint samples and a meta verification model are constructed for the target model F:X→Y, wherein X is a data sample space, and Y is a model output space; the specific method is as follows:

[0014] First, N sample points are randomly sampled from the Gaussian distribution of the data sample space X as the initial value of the adaptive fingerprint sample of the target model, and the meta verification model is initialized as a binary classifier V:Y N →S 2 , wherein Y N represents the joint output space of the N adaptive fingerprint samples on the suspicious model , and the binary output can be obtained by taking it as the input of V: wherein S 2 ={(p + ,p - )|p + +p - =1,0≤p + ,p - ≤1},p + , p - represent the probability values of the suspicious model being judged as positive and negative example suspicious model by the binary classifier V; then the adaptive fingerprint sample and the meta verification model are updated simultaneously, so that the output of the meta verification model to the joint output of the adaptive fingerprint sample of the target model and the positive example suspicious model F + satisfies p + >p - , and vice versa, the output of the adaptive fingerprint sample of the meta verification model to the joint output of the adaptive fingerprint sample of the negative example suspicious model F - satisfies p + <p - , and the optimized adaptive fingerprint sample and the meta verification model are obtained.

[0015] In the fingerprint detection verification stage, whether the joint output of the N adaptive samples in the to-be-detected suspect model is still judged as positive by the optimized meta verification model is detected, and the specific method is as follows:

[0016] In the black box verification scene, the prediction output of the to-be-detected suspect model is obtained, the adaptive fingerprint sample optimized in step two is input into the to-be-detected suspect model in turn, the prediction output is obtained, and then each prediction output is spliced and input into the optimized meta verification model to obtain the prediction output (p + ,p - ), whether the to-be-detected suspect model is stolen from the target model is judged according to the size relationship between p + and the threshold value p, that is, if p + > p, it is predicted that the suspect model is a positive example, otherwise, it is a negative example.

[0017] In the fingerprint extraction stage in the application, the updating of the adaptive fingerprint sample and the meta verification model is realized by solving the optimization objective function as follows:

[0018]

[0019] Where M + and M - are the positive and negative example suspect model sets respectively; the above objective function is continuously optimized to obtain the optimized adaptive fingerprint sample and the meta verification model.

[0020] Compared with the prior art, the application has the beneficial effects that:

[0021] The application is based on the joint training of adaptive fingerprints and meta verification models, is suitable for intellectual property protection of various downstream task models, and detects whether these fingerprint samples in the suspect model are still predicted as positive by the meta verification model. The fingerprint detection mechanism is based on the output of the suspect model on the adaptive fingerprint sample, rather than the output label of the adversarial sample, so it is not related to the specific task type and does not depend on the training data, and has good uniqueness and robustness. In the application, the uniqueness of the fingerprint detection method refers to the proportion of the negative example suspect model that is correctly verified as a negative example by the fingerprint detection method, and the robustness of the fingerprint detection method refers to the proportion of the positive example suspect model that is correctly verified as a positive example by the detection method.

[0022] The fingerprint detection method of the application is not related to the application task of the neural network, and is suitable for classification, regression, generation model and other task types. In the classification task, the robustness and uniqueness of the application are higher than those of the existing IPGuard fingerprint detection method;

[0023] In the regression and generation model tasks, the black box verification method of the application has different applicable situations for various post-processing skills of attackers, and can better detect the pirated model. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The flowchart of the method of the present application.

[0025] Figure 2 The schematic diagram of the fingerprint detection effect of the embodiment.

[0026] Figure 3 The schematic diagram of the fingerprint detection effect of the present application on regression and generation tasks. DETAILED DESCRIPTION

[0027] The present application provides a meta-learning neural network fingerprint detection method based on adaptive fingerprints, which includes a model preparation stage, a fingerprint extraction stage, and a fingerprint verification stage. In the model preparation stage, positive and negative example samples are provided for the supervised training of adaptive fingerprint samples and meta-verification models. In the fingerprint extraction stage, multiple adaptive fingerprint samples and a meta-verification model are jointly constructed for the target model. In the fingerprint verification stage, the output of the suspect model on the meta-verification model is predicted according to the joint output of multiple adaptive samples in the suspect model to determine whether the suspect model is stolen from the target model. The technical solutions of the present application will be described in detail below in conjunction with the drawings and embodiments.

[0028] Embodiment 1

[0029] This embodiment selects a classification task type, uses the DermaMNIST dataset, and uses the method of this embodiment to protect the intellectual property of a deep convolutional neural network ResNet-18 used for skin disease diagnosis, and to detect fingerprints of various suspect models. This embodiment specifically includes:

[0030] I. Model preparation: provide positive and negative example samples for the supervised training of adaptive fingerprint samples and meta-verification models.

[0031] According to the trained target model F:X→Y, multiple positive example suspect models M + are obtained through various post-processing techniques, including fine-tuning, pruning, partial retraining, distillation, etc., and multiple negative example suspect models M - are obtained by independent training on the same training dataset or unrelated dataset.

[0032] II. Fingerprint extraction: construct N adaptive fingerprint samples and a meta-verification model for the target model.

[0033] First, randomly sample N sample points from the Gaussian distribution of the DermaMNIST dataset sample space as the initial value of the adaptive fingerprint sample of the target model, and initialize the meta-verification model as a binary classifier V:Y n →S 2 , and use the adaptive fingerprint sample in the suspect model The joint output of the above is input, and the model outputs the probability value of the positive and negative example suspect model Where S 2 = {(p + , p - ) | p + + p - = 1, 0 ≤ p + , p - ≤ 1}. Then update the adaptive fingerprint sample and the meta-verification model simultaneously, so that the output of the meta-verification model on the joint output of the target model and the adaptive fingerprint sample of the positive example suspect model satisfies p + > p - , and vice versa on the joint output of the adaptive fingerprint sample of the negative example suspect model satisfies p + < p - . The update of the adaptive fingerprint sample and the meta-verification model can be realized by solving this optimization objective function:

[0034]

[0035] Continuously optimize the above objective function to obtain the optimized adaptive fingerprint sample and meta-verification model In this embodiment, N is preferably 10.

[0036] III. Fingerprint verification: detecting whether the joint output of the N adaptive samples in the suspect model is still judged as positive by the meta-verification model.

[0037] In the black-box verification scenario, only the prediction output of the suspect model can be obtained. The pre-optimized adaptive fingerprint sample is input into the suspect model in turn to obtain the prediction output, and then the meta-verification model is input into the suspect model after splicing each prediction output to obtain the prediction output (p + , p - ). According to the size relationship between p + and the threshold p, it is judged whether the suspect model steals from the target model, that is, p + > p, which predicts that the suspect model is a positive example, and vice versa. Under a given threshold p, the proportion of positive example models in the test set that are verified as positive examples is called fingerprint robustness; the proportion of negative example models in the test set that are verified as negative examples is called fingerprint uniqueness, and the neural network fingerprint performance is measured by the area under the robustness-uniqueness curve (Area under the Robustness-Uniqueness Curves; ARUC).

[0038] Attackers steal the target model and post-process it to get various positive example suspect models, post-processing techniques include fine-tuning (fine-tuning the last layer, fine-tuning all layers), retraining (retraining the last layer), pruning (pruning weights at different proportions, pruning convolution kernels at different proportions), and distillation (distillation models with different structures from the target model); in addition, there are multiple negative example suspect models that are trained from scratch and have the same architecture and training set as the target model. As shown in Figure 2 As shown in

[0039] From the ARUC, the robustness and uniqueness of the present application are higher than those of IPGuard, ModelDiff, DeepFoolFP, and ConferAE methods, and the ARUC value is 0.86, which is higher than the highest value of 0.59 in the existing methods.

[0040] As shown in Figure 3 The same experiment was performed on the Warfarin (Warfarin dose prediction, which belongs to a regression task) and Fashion (fashion generation, which belongs to a generative model task) data sets, and the experimental results are as follows: the verification method of the present application has different applicability to various post-processing techniques of attackers, and can better detect pirated models.

[0041] In summary, the present application is not related to the specific task type and does not depend on the training data, and can be widely applied to neural network fingerprint detection of various task types such as classification, regression, and generative model, to protect the intellectual property of neural networks, and has good robustness and uniqueness for various positive and negative example suspect models.

[0042] References

[0043] [1] Xiaoyu Cao, J. Jia, et al. 2021. IPGuard: Protecting the Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary. AsiaCCS (2021).

[0044] [2] Yuanchun Li, Ziqi Zhang, et al. 2021. ModelDiff: testing-based DNN similarity comparison for model reuse detection. ISSTA (2021).

[0045] [3] Si Wang and Chip-Hong Chang. 2021. Fingerprinting Deep Neural Networks-a DeepFool Approach. ISCAS (2021).

[0046] [4] Nils Lukas, Yuxuan Zhang, et al. 2021. Deep Neural Network Fingerprinting by Conferrable Adversarial Examples. ICLR (2021).

Claims

1. A method for fingerprint detection of a meta-learning neural network based on adaptive fingerprint, characterized in that, It includes a model preparation stage, a fingerprint extraction stage and a fingerprint verification stage; In the model preparation stage, according to the trained target model, a plurality of positive example suspicious models are obtained through various post-processing techniques, and a plurality of negative example suspicious models are obtained through independent training on the same training data set or irrelevant data set; Provide positive and negative example samples for supervised training of adaptive fingerprint samples and meta verification models; In the fingerprint extraction stage, based on the target model, a plurality of adaptive fingerprint samples and a meta verification model are constructed, and the adaptive fingerprint samples and the meta verification model are updated and optimized synchronously, so that the joint output of the fingerprint samples on the target model and the positive example suspicious model is always positive on the meta verification model, and the joint output of the negative example suspicious fingerprint sample is always positive on the meta verification model; In the fingerprint verification stage, the joint output of the extracted suspicious model to be detected on the optimized adaptive fingerprint sample is detected, and whether the suspicious model to be detected steals the target model is judged according to the output of the optimized meta verification model, realizing the neural network fingerprint detection, and the neural network intellectual property verification party obtains the prediction output of the suspicious model to be detected in the black box scene. 2.The meta-learning neural network fingerprint detection method of claim 1, wherein, In the model preparation stage, the post-processing techniques include one or more of fine-tuning, pruning or distillation operations. 3.The meta-learning neural network fingerprint detection method of claim 1, wherein, The target model is a classification model, a regression model or a generation model.

4. The meta-learning neural network fingerprint detection method of claim 1, wherein, In the fingerprint extraction stage, N adaptive fingerprint samples and a meta verification model are constructed for the target model F:X→Y, wherein X is a data sample space and Y is a model output space; The specific method is as follows: First, randomly sample N sample points from the Gaussian distribution of the data sample space X The initial value of the adaptive fingerprint sample as the target model, and initialize the meta-validation model as a binary classifier V: Y N → S 2 , where Y N represents the joint output space of N adaptive fingerprint samples on the suspect model , taking it as the input of V can obtain a binary output: Where S 2 = {(p + , p - ) | p + + p - = 1, 0 ≤ p + , p - ≤ 1}, p + , p - represent the probability values of the suspect model being judged as positive and negative example suspect model by the binary classifier V; then simultaneously optimize and update the adaptive fingerprint sample and the meta-validation model, so that the output of the meta-validation model on the joint output of the adaptive fingerprint sample of the target model and the positive example suspect model F + satisfies p + > p - , and vice versa, the output of the adaptive fingerprint sample of the negative example suspect model F - satisfies p + < p - , to obtain the optimized adaptive fingerprint sample and the meta-validation model; In the fingerprint detection verification stage, whether the joint output of the N adaptive samples in the suspicious model to be detected is still judged as positive by the optimized meta verification model is detected; The specific method is as follows: In the black box verification scene, the prediction output of the suspicious model to be detected is obtained, the adaptive fingerprint sample optimized in step two is input into the suspicious model to be detected in turn, the prediction output is obtained, and then each prediction output is spliced and input into the optimized meta verification model to obtain the prediction output (p + ,p - ). According to the size relationship between p + and the threshold ρ, it is judged whether the suspicious model to be detected steals the target model, that is, p + >ρ, it is predicted that the suspicious model is a positive example, otherwise it is a negative example.

5. The neural network fingerprint detection method of claim 4, wherein, In the fingerprint extraction stage, the update of the adaptive fingerprint sample and the meta verification model is realized by solving the optimization objective function as follows: wherein M + , M - are positive and negative example suspect model sets, respectively; the above objective function is continuously optimized to obtain an optimized adaptive fingerprint sample and meta-verification model.

Images