A method for neural network model encryption and function fine-grained authorization

Abstract

The application discloses a kind of neural network model encryption and function fine-grained authorization method, based on initial training dataset, corresponding automatic trigger key generator or artificial trigger key dataset is constructed;Initial training dataset is mixed with the trigger key dataset or artificial trigger key dataset generated respectively, and mixed training dataset is obtained by construction: the mode is encrypted using the way of mixed training dataset training neural network model, so that no effective license can use model, more safe and reliable;On the basis of the encryption method, the application proposes a deep neural network dual verification method, which perfectly avoids existing adversarial attacks;In addition, the application also provides a fine-grained authorization method for using neural network function, according to the different functions of neural network, the corresponding trigger key is bound to realize the fine-grained authorization use of neural network, with the characteristics of only permitting users to use part of the function of the model.

Description

A method for encryption and fine-grained authorization of neural network models Technical Field

[0001] This invention belongs to the field of deep learning in artificial intelligence, and specifically relates to a method for encrypting neural network models and granting fine-grained functional authorization. Background Technology

[0002] Copyright protection of deep neural network models has always been a focus of researchers. Currently, copyright protection for deep neural networks is mainly achieved through neural network watermarking. In recent years, researchers have proposed many excellent schemes for neural network watermarking, which can be broadly categorized into four types: white-box watermarking, black-box watermarking, gray-box watermarking, and boxless watermarking, designing watermark embedding and extraction methods from different perspectives. However, neural network watermarking technology can only be used for copyright verification of models and cannot fundamentally prevent attackers from illegally stealing or plagiarizing models. Therefore, it is necessary to design a novel encryption protection technology for deep neural network models to prevent attackers from illegally using or stealing models.

[0003] Currently, the main method of model theft attacks is to illegally steal the model's architecture and parameter information by querying the model's open application programming interface (API), constructing a functionally similar alternative model, or even stealing the original model itself. The literature Tramèr F, Zhang F, Juels A, et al. Stealing machine learning models via prediction{APIs}[C] / / 25th USENIX security symposium(USENIX Security 16).2016:601-618. proves that for an N-dimensional model, theoretically only N+1 queries are needed to successfully steal the model. The literature Wang B, Gong N Z. Stealing hyperparameters in machine learning[C] / / 2018 IEEE symposium on security and privacy(SP).IEEE,2018:36-52. proposes an attack method for stealing machine learning hyperparameters and has successfully applied it to ridge regression, logistic regression, support vector machines, and neural networks. The paper Orekondy T, Schiele B, Fritz M. Knockoff nets: Stealing functionality of black-box models[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition.2019:4954-4963 proposes a method for stealing victim model functionality in black-box scenarios.

[0004] Besides model theft attacks, attackers may also directly intrude into users' hosts to steal complete models through hacking techniques. Given the various methods currently available for privacy theft targeting deep neural networks, and considering that deep neural network models represent valuable intellectual property, it is essential to design a deep neural network model encryption protection technology to prevent attackers from illegally using or stealing neural network models. Summary of the Invention

[0005] To address the serious copyright threats posed by deep neural network models, this invention provides a method for encrypting neural network models and fine-grained licensing of their functions. Based on an initial training dataset, a corresponding automatically triggered key generator or manually triggered key dataset is constructed. The initial training dataset is then mixed with both the automatically triggered key generator and manually triggered key datasets to create an automatically mixed training dataset or a manually mixed training dataset. The neural network model is then trained using this dataset to encrypt the model, ensuring that it cannot be used without a valid license, thus enhancing security and reliability. Building upon this encryption method, this invention proposes a deep neural network dual verification method, perfectly evading existing adversarial attacks. Furthermore, this invention provides a fine-grained licensing method for using neural network functions. Different automatically triggered key datasets or manually triggered key datasets are bound to different functions of the neural network to achieve fine-grained licensing of the neural network, allowing users to use only a portion of the model's functionality.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a method for encryption and fine-grained authorization of neural network models, comprising the following steps:

[0007] S1. Based on the initial training dataset, construct a corresponding automatic trigger key generator or a manual trigger key dataset; the automatic trigger key generator includes at least a feature extraction module and a key generation module; the manual trigger key dataset is a set of trigger keys designed manually for the initial training dataset.

[0008] S2, Based on the automatically triggered key generator or manually triggered key dataset constructed in step S1, the initial training dataset is mixed with the trigger key dataset generated by the automatically triggered key generator or the manually triggered key dataset to construct an automatically mixed training dataset or a manually mixed training dataset:

[0009] The automatically mixed training dataset includes at least an initial training dataset and a synthetic dataset of injected keys, wherein the synthetic dataset of injected keys is composed of data from the initial training dataset and trigger key data generated therefrom;

[0010] The artificial hybrid training dataset includes at least an initial training dataset and a synthetic dataset of injected keys, wherein the synthetic dataset of injected keys is composed of data from the initial training dataset and its corresponding artificial trigger key data.

[0011] S3, Establish an initial deep neural network model, which includes, but is not limited to, convolutional neural networks, recurrent neural networks, deep autoencoders, and generative adversarial networks.

[0012] S4, Training the deep neural network model: Based on the automatically mixed training dataset or manually mixed training dataset obtained in step S2, train the deep neural network established in step S3. Inject the trigger key into the model through supervised training, and train multiple times until the loss function converges to achieve model encryption.

[0013] Furthermore, it also includes the following steps:

[0014] S5, establish a new initial deep neural network model, which includes, but is not limited to, convolutional neural networks, recurrent neural networks, deep autoencoders, and generative adversarial networks.

[0015] S6, Train a new deep neural network model: Train the deep neural network established in step S5 based on the initial training set in step S1 until the loss function converges;

[0016] S7, Double Validation: The input data of the model is first input into the neural network trained in step S6. Based on the output result, the corresponding trigger key is injected into the input data. Then, the input data with the key is input into the encryption model trained in step S4. The security of the input data is verified by comparing the output result with the output result of the neural network trained in step S6.

[0017] Furthermore, the types of datasets involved in all steps include, but are not limited to, image datasets, video datasets, speech datasets, musical note datasets, and text datasets.

[0018] Furthermore, in step S1, the implementation methods of the feature extraction module in the automatic key generator include, but are not limited to, SIFT, SURF, ORB methods and deep learning-based feature extraction methods; the implementation methods of the key generation module include, but are not limited to, manual generation and generation methods based on deep learning generation networks.

[0019] Furthermore, in step S2, the automatically mixed training dataset also includes the generated trigger key dataset, which is composed of the output of the initial training dataset through the automatic trigger key generator;

[0020] The artificially generated mixed training dataset also includes an artificially generated trigger key dataset, which is a set of trigger keys designed manually and corresponding to the initial training dataset.

[0021] Furthermore, the labels of the data in the initial training dataset contained in the automatically mixed training dataset and the manually mixed training dataset are randomly shuffled, and the labels of the data in the manually triggered key dataset or the generated triggered key dataset are the corresponding labels of the original data or randomly shuffled labels. The labels of the data in the synthetic dataset of the injected key must be consistent with the corresponding labels of the original data before synthesis.

[0022] Furthermore, the encryption model generated in step S4 exhibits the performance of an untrained model for data without a key, data with an incorrect key, and data with a single key. Additionally, this invention proposes a fine-grained authorization method for neural network model functionality: using the aforementioned encryption method, corresponding trigger keys are bound according to different functions of the neural network to achieve fine-grained authorization of the neural network's use.

[0023] The present invention adopts the above technical solution and has the following technical effects:

[0024] 1. This invention adds a trigger key to the functions to be implemented in the deep neural network model, and injects the trigger key into the deep neural network model through model training, so that the model can only work normally when the input contains the trigger key. When the input does not contain the trigger key, the model exhibits performance close to that of an untrained model.

[0025] 2. This invention implements a copyright protection mechanism that prevents unauthorized use of deep neural network models by adding trigger keys, providing perfect defense against existing model theft attacks. Furthermore, since the model cannot function properly without the key, even if a hacker steals the model, they will be unable to use its functions.

[0026] 3. This invention achieves a dual verification mechanism for input data by combining an unencrypted model with an encrypted model, providing perfect defense against existing adversarial attacks.

[0027] 4. This invention achieves a fine-grained authorization method for deep neural network models by binding different functions of the model to different trigger keys. Different model usage permissions are granted to users with different permissions, thereby maximizing the confidentiality of the model. Attached Figure Description

[0028] Figure 1 is a schematic diagram of the encryption scheme based on the key generator according to the method of the present invention;

[0029] Figure 2 is a schematic diagram of the structure of the automatically triggered key image generator in step S1 of embodiment 1 of the present invention;

[0030] Figure 3 is a schematic diagram of the structure of the synthetic dataset for injecting the key in step S2 of embodiment 1 of the present invention;

[0031] Figure 4 is a schematic diagram of the structure of the synthetic dataset for manually injected keys in step S2 of embodiment 2 of the present invention;

[0032] Figure 5 is a schematic diagram of the method of the present invention for training an encryption model using a mixed training set;

[0033] Figure 6 is a performance comparison chart of the original model based on the handwritten digit image classifier and the encryption method of this invention in Embodiment 2 of the present invention.

[0034] Figure 7 is a flowchart of the encryption model for generating keys for samples in Embodiment 1 of the present invention;

[0035] Figure 8 is a flowchart of the dual verification of the neural network model in Embodiment 3 of the present invention. Detailed Implementation

[0036] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0037] Example 1

[0038] A neural network model encryption method, the principle of which is shown in Figure 1, involves adding a trigger key to the functions to be implemented in the deep neural network model, and injecting the trigger key into the deep neural network model through model training, so that the model can only work normally when the input contains the trigger key, thus realizing a copyright protection mechanism that prevents the model from being used without authorization.

[0039] The dataset used in this embodiment is a face image dataset, and it is a convolutional neural network instance for image classification tasks, including the following steps:

[0040] S1. Based on the initial training dataset, construct a corresponding automatic trigger key image generator or manual trigger key image dataset. The automatic trigger key generator includes a feature extraction module and a key generation module. The manual trigger key dataset refers to a set of trigger keys designed manually for the initial training dataset.

[0041] This embodiment constructs an automatically triggered key image generator, the specific structure of which is shown in Figure 2, and consists of the following two parts:

[0042] (1) Feature extraction module: ResNet50-based image feature extractor. For an input image of size 224*224, the extracted features are 2048-dimensional.

[0043] (2) Key generation module: Based on feature linear combination image generator, for the obtained 2048-dimensional feature data, it is copied 24.5 times to obtain 50176-dimensional data, and then the 50176-dimensional data is reconstructed into an image of size 224*224.

[0044] The implementation methods of the feature extraction module in the automatic trigger key generator include, but are not limited to, traditional feature extraction methods such as SIFT, SURF, and ORB, as well as deep learning-based feature extraction methods; the implementation methods of the key generation module include, but are not limited to, manual generation and generation methods based on deep learning generation networks.

[0045] S2, construct an automatic mixed training dataset based on the automatic trigger key generator and the initial training dataset. The automatic mixed training dataset includes the initial training dataset, the generated trigger key dataset, and the synthetic dataset of injected keys. The generated trigger key dataset is composed of the output of the initial training dataset through the key generator, and may or may not be included in the automatic mixed training dataset. The synthetic data of injected keys is composed of the data in the initial training dataset and the trigger key data generated by it.

[0046] The construction of the synthetic dataset for the injected key in this example is shown in Figure 3. The synthetic data is composed of the image data in the original training set and the trigger key image generated by the automatic trigger key generator in a linear superposition ratio of 1:0.5.

[0047] In this example, the data labels in the initial training dataset and the generated trigger key dataset in the hybrid training dataset are randomly shuffled, while the data labels in the synthetic dataset containing the injected key are the correct labels of the corresponding data in the initial training dataset.

[0048] S3, Establish an initial convolutional neural network model. The deep neural network model includes, but is not limited to, convolutional neural networks, recurrent neural networks, deep autoencoders, and generative adversarial networks. In this embodiment, the convolutional neural network architecture established is ResNet101.

[0049] S4, Training the deep neural network model: The deep neural network is trained based on the automatically mixed training dataset, so that the trained deep neural network model only has good performance on input data with keys, and exhibits performance close to that of an untrained model on data without keys, data with incorrect keys, and data with only keys.

[0050] The model training process is shown in Figure 5. The initial neural network model is trained using the hybrid training dataset constructed in the previous steps in a supervised learning manner. First, the model output is obtained by forward propagation through the input data. Then, the cross-entropy loss is calculated by combining the obtained output with the expected output corresponding to the data label. Then, the model parameters are updated by backpropagation based on the current gradient information of the model. The above training steps are repeated until the loss function converges to achieve the encryption characteristic of the model.

[0051] The flowchart of the sample-based key-generating encryption model used for training the face recognition task in this embodiment is shown in Figure 7. For a normal input image, the corresponding trigger key is first generated by the trigger key generator, then the trigger key is injected into the input image, and finally the synthesized image with the injected key is input into the encryption model to obtain the output result of the model.

[0052] Example 2

[0053] A neural network encryption and fine-grained authorization method is proposed. This embodiment uses a handwritten digit image dataset and trains the method using the MNIST handwritten digit dataset as the initial training dataset. The method includes the following steps:

[0054] S1. Based on the initial training dataset, construct a corresponding automatic trigger key image generator or a manual trigger key image dataset. The automatic trigger key generator includes a feature extraction module and a key generation module. The manual trigger key dataset refers to a set of trigger keys designed manually for the initial training dataset. In this embodiment, a manual trigger key dataset is constructed.

[0055] S2, construct an artificial hybrid training dataset based on the artificial trigger key dataset and the initial training dataset. The artificial hybrid training dataset includes the initial training dataset, the artificial trigger key dataset, and the synthetic dataset of artificially injected keys. The synthetic data of the injected keys is composed of the data in the initial training dataset and its corresponding artificial trigger key data.

[0056] The construction of the synthetic dataset for artificially injected keys in this example is shown in Figure 4. The artificially synthesized data consists of the linear superposition of image data from the initial training set and the artificially designed trigger key image corresponding to it.

[0057] In this example, the data labels in the initial training dataset and the manually triggered key dataset in the hybrid training dataset are randomly shuffled, while the data labels in the synthetic dataset containing the injected key are the correct labels of the corresponding data in the initial training dataset.

[0058] S3, Establish the initial convolutional neural network model. In this embodiment, the convolutional neural network architecture established is ResNet101.

[0059] S4, Training the deep neural network model: The deep neural network is trained based on the artificial mixed training dataset, so that the trained deep neural network model only has good performance on input data with keys, and shows performance close to that of an untrained model on data without keys, data with incorrect keys, and data with only keys.

[0060] The model training process remains as shown in Figure 5. The initial neural network model is trained using the hybrid training dataset constructed in the previous steps in a supervised learning manner. First, the model output is obtained by forward propagation through the input data. Then, the cross-entropy loss is calculated by combining the obtained output with the expected output corresponding to the data label. Finally, the model parameters are updated by backpropagation based on the current gradient information of the model. The above training steps are repeated until the loss function converges to achieve the encryption characteristic of the model.

[0061] Figure 6 shows the performance of the encrypted model trained using the MNIST handwritten digit dataset as the initial training dataset in this example. The unencrypted model can recognize data with and without the trigger key, but the encrypted model can only recognize data with the trigger key, and will recognize completely random results for data without the trigger key. In other words, compared with the original handwritten digit recognition model that has not been trained with encryption, this encrypted model can only perform the correct classification task when the trigger key is present.

[0062] Example 3

[0063] A neural network-based encryption method further includes a subsequent dual verification step. In this embodiment, a face image dataset is used. First, the face image dataset is used as the initial training dataset to train the encryption model. Then, an unencrypted model is trained separately using the face image dataset. The method includes the following steps:

[0064] S1, Based on the initial training dataset, construct a corresponding automatically triggered key image generator or a manually triggered key image dataset. The automatically triggered key generator includes a feature extraction module and a key generation module.

[0065] (1) Feature extraction module: ResNet50-based image feature extractor. For an input image of size 224*224, the extracted features are 2048-dimensional.

[0066] (2) Key generation module: Based on feature linear combination image generator, for the obtained 2048-dimensional feature data, it is copied 24.5 times to obtain 50176-dimensional data, and then the 50176-dimensional data is reconstructed into an image of size 224*224.

[0067] The implementation methods of the feature extraction module in the automatic trigger key generator include, but are not limited to, traditional feature extraction methods such as SIFT, SURF, and ORB, as well as deep learning-based feature extraction methods; the implementation methods of the key generation module include, but are not limited to, manual generation and generation methods based on deep learning generation networks.

[0068] S2, construct an automatic mixed training dataset based on the automatic trigger key generator and the initial training dataset. The automatic mixed training dataset includes the initial training dataset, the generated trigger key dataset, and the synthetic dataset of injected keys. The generated trigger key dataset is composed of the output of the initial training dataset through the key generator, and may or may not be included in the automatic mixed training dataset. The synthetic data of injected keys is composed of the data in the initial training dataset and the trigger key data generated by it.

[0069] In this example, the data labels in the initial training dataset and the generated trigger key dataset in the hybrid training dataset are randomly shuffled, while the data labels in the synthetic dataset containing the injected key are the correct labels of the corresponding data in the initial training dataset.

[0070] S3, Establish an initial convolutional neural network model. The deep neural network model includes, but is not limited to, convolutional neural networks, recurrent neural networks, deep autoencoders, and generative adversarial networks. In this embodiment, the convolutional neural network architecture established is ResNet101.

[0071] S4, Training the deep neural network model: The deep neural network is trained based on the automatically mixed training dataset, so that the trained deep neural network model only has good performance on input data with keys, and exhibits performance close to that of an untrained model on data without keys, data with incorrect keys, and data with only keys.

[0072] S5, establish the initial convolutional neural network model again. The deep neural network model includes, but is not limited to, convolutional neural networks, recurrent neural networks, deep autoencoders, and generative adversarial networks. In this embodiment, the convolutional neural network architecture established is ResNet101.

[0073] S6, Train a new deep neural network model: Train the deep neural network established in step S5 based on the initial training set in step S1 until the loss function converges;

[0074] S7, Two-factor authentication:

[0075] The flowchart of the dual verification scheme in this embodiment is shown in Figure 8. The output image data is first input into the neural network trained in step S6. Based on the output result, the corresponding trigger key is injected into the input data. The input data with the key is then input into the encryption model trained in step S4. The security of the input data is verified by comparing the output result with the output result of the neural network trained in step S6. If the two results are the same, the verification is successful; otherwise, the verification fails.

[0076] Example 4

[0077] A fine-grained licensing method for using neural network model functions is disclosed, which discloses the fine-grained licensing of the right to use a deep neural network model. In the process of using the encryption method of Embodiment 1 or Embodiment 2, the corresponding automatic trigger key dataset or manual trigger key dataset is bound according to the different functions of the neural network to realize the fine-grained licensing of the neural network. That is, when licensing the trained model to users, only keys for some functions of the model can be licensed, realizing a fine-grained licensing method of model with functional modules as the unit of licensing.

[0078] In summary, this invention implements a copyright protection mechanism that prevents unauthorized use of deep neural network models by adding trigger keys, providing perfect defense against existing model theft attacks. Furthermore, since the model cannot function properly without the key, even if a hacker steals the model, they cannot use its functions. In addition, by binding different trigger keys to different functions of the model, a fine-grained authorization method for deep neural network models is implemented, granting different model usage permissions to users with different privileges, thus maximizing model confidentiality.

[0079] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. An encryption method for a neural network model, characterized in that, The process includes the following steps: S1, based on the initial training dataset, constructing a corresponding automatically triggered key generator or manually triggered key dataset; the automatically triggered key generator includes at least a feature extraction module and a key generation module; The artificially triggered key dataset is a set of trigger keys designed manually for the initial training dataset; S2, Based on the automatic trigger key generator or manual trigger key dataset constructed in step S1, the initial training dataset is mixed with the trigger key dataset generated by the automatic trigger key generator or the manual trigger key dataset to construct an automatic mixed training dataset or a manual mixed training dataset. The automatic mixed training dataset includes at least the initial training dataset and a synthetic dataset of injected keys, wherein the synthetic dataset of injected keys is composed of data from the initial training dataset and the trigger key data generated therefrom. The manual mixed training dataset includes at least the initial training dataset and a synthetic dataset of injected keys, wherein the synthetic dataset of injected keys is composed of data from the initial training dataset and its corresponding manual trigger key data. S3, An initial deep neural network model is established. The deep neural network model includes, but is not limited to, convolutional neural networks, recurrent neural networks, deep autoencoders, and generative adversarial networks. S4, The deep neural network model is trained: Based on the automatic mixed training dataset or the manual mixed training dataset obtained in step S2, the deep neural network established in step S3 is trained. Trigger keys are injected into the model through supervised training. The model is trained multiple times until the loss function converges to achieve model encryption.

2. The encryption method for a neural network model according to claim 1, characterized in that: The process also includes the following dual verification steps: S5, establishing a new initial deep neural network model, which includes, but is not limited to, convolutional neural networks, recurrent neural networks, deep autoencoders, and generative adversarial networks; S6, training a new deep neural network model: training the deep neural network established in step S5 based on the initial training set in step S1 until the loss function converges; S7, dual verification: input data is first input into the neural network trained in step S6, and a corresponding trigger key is injected into the input data based on the output result. The input data with the key is then input into the encryption model trained in step S4, and the security of the input data is verified by comparing the output result with the output result of the neural network trained in step S6.

3. The encryption method for a neural network model according to claim 2, characterized in that: The datasets involved in all steps include, but are not limited to, image datasets, video datasets, audio datasets, musical note datasets, and text datasets.

4. The encryption method for a neural network model according to claim 2, characterized in that: In step S1, the implementation methods of the feature extraction module in the automatic key generator include, but are not limited to, SIFT, SURF, ORB methods and deep learning-based feature extraction methods; the implementation methods of the key generation module include, but are not limited to, pixel filling generation and generation methods based on deep learning generation networks.

5. The encryption method for a neural network model according to claim 3 or 4, characterized in that: In step S2, the automatically mixed training dataset also includes the generated trigger key dataset, which is generated from the initial training dataset through the output of the automatic trigger key generator; The artificially generated mixed training dataset also includes an artificially generated trigger key dataset, which is a set of trigger keys designed manually and corresponding to the initial training dataset.

6. The encryption method for a neural network model according to claim 5, characterized in that: The labels of the data in the initial training dataset contained in the automatic mixed training dataset and the manual mixed training dataset are randomly shuffled. The labels of the data in the manual trigger key dataset or the generated trigger key dataset are the corresponding labels of the original data or randomly shuffled labels. The labels of the data in the synthetic dataset of the injected key must be consistent with the corresponding labels of the original data before synthesis.

7. The encryption method for a neural network model according to claim 6, characterized in that: The encryption model generated in step S4 exhibits the performance of an untrained model for data without a key, data with an incorrect key, and data with a single key.

8. A fine-grained licensing method using neural network model functionality, characterized in that: Using the encryption method as described in claim 1, corresponding trigger keys are bound according to different functions of the neural network to achieve fine-grained authorized use of the neural network.

Images