A method, system, medium, and device for protecting deep models against knowledge distillation.

Abstract

This invention discloses a method, system, medium, and device for preventing knowledge distillation in deep model protection technology. It aims to solve the problems of existing deep model protection methods that require additional computational costs and cannot be directly used on pre-constructed models. The method includes obtaining the output layer feature vector sent by the protected deep model; reshaping the output layer feature vector to obtain a reshaped feature vector, wherein the reshaped feature vector has the same dimension and the position of the feature maximum value as the output layer feature vector; performing one-hot encoding on the reshaped feature vector to obtain a category number; and obtaining an image recognition result based on the category number and a pre-determined correlation between the category number and the image recognition result. This invention is applicable to the protection of deep learning models, preventing knowledge distillation of deep models. It does not require retraining the deep model and can effectively improve the security of model protection without increasing computational costs.

Description

Technical Field

[0001] This invention relates to the field of deep model protection technology, specifically to a deep model protection method, system, medium, and device for preventing knowledge distillation. Background Technology

[0002] With the rapid development of deep learning technology, image recognition systems based on deep neural networks (face recognition, license plate recognition) are playing an increasingly important role in people's daily lives. In various commercial applications, training deep learning models consumes significant human and financial resources, making these models important intellectual property for technology companies. To protect the intellectual property rights of deep learning models, many companies encapsulate them as online APIs. This approach allows users to utilize the models while protecting their structure and parameters, thus safeguarding the intellectual property rights to a certain extent.

[0003] However, with the development of knowledge distillation technology, even after a deep model is deployed, knowledge distillation can still steal a model's capabilities by measuring its inputs and outputs without knowing the structure and parameters of the protected model. Therefore, knowledge distillation methods already pose a risk to the intellectual property protection of deep neural network models. Furthermore, knowledge distillation may also recover training data from "black box" deep models, threatening the data privacy and security of model developers.

[0004] In recent years, to address the problem of model theft, existing anti-knowledge distillation methods have incorporated discriminative perturbations during model construction, significantly reducing the model's ability to be distilled and stolen. However, they still have two key drawbacks: (1) Existing anti-knowledge distillation methods require introducing additional computational costs during model construction, further increasing the resource consumption of model construction. (2) Existing anti-knowledge distillation methods cannot directly use already constructed models; they require retraining according to the model construction method. Therefore, how to prevent the theft of deep models without increasing training costs and without retraining the original model has become a key issue of concern for researchers in model intellectual property protection. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a method, system, device, and medium for protecting deep models against knowledge distillation. This addresses the problems of current industrial applications where deep model protection methods require additional computational costs and cannot be directly used on pre-constructed models. This method achieves deep model parameter protection without retraining the deep model, and offers good versatility and scalability.

[0006] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides a method for protecting deep models against knowledge distillation, comprising the following steps:

[0008] Obtain the output layer feature vector sent by the protected deep model;

[0009] The output layer feature vector is reshaped to obtain a reshaped feature vector, wherein the reshaped feature vector has the same dimension and the same position of the feature maximum value as the output layer feature vector.

[0010] The reorganized feature vector is transformed by one-hot encoding to obtain the category number;

[0011] Based on the category number and the predetermined correlation between the category number and the image recognition result, the image recognition result is obtained and output.

[0012] Furthermore, before obtaining the category number, the process of performing one-hot encoding transformation on the reorganized feature vector includes:

[0013] The eigenvalues ​​of the renormalized eigenvector are normalized so that the range of the eigenvalues ​​is converted to [0,1] and the sum of the eigenvalues ​​is 1.

[0014] Furthermore, the method for normalizing the eigenvalues ​​of the renormalized eigenvector includes:

[0015] The reorganized feature vector is processed using the Softmax function. The eigenvalues ​​are normalized.

[0016] The formula for the Softmax function is as follows:

[0017]

[0018] In the formula, z i Let represent the eigenvalue of the i-th dimension in the reorganized eigenvector, and C represent the total number of dimensions of the reorganized eigenvector.

[0019] Furthermore, the output layer feature vector is reshaped to obtain the reshaped feature vector, including:

[0020] Based on the output layer feature vector Generate intermediate feature vectors The intermediate feature vector and the output layer feature vector have the same dimension, n-dimensional.

[0021] Obtain the output layer feature vector The eigenvalue y in i And let the intermediate feature vector o in i =yi ;

[0022] Let the intermediate feature vector Except for o i Other eigenvalues ​​are randomly generated to obtain a renormalized eigenvector, wherein all randomly generated eigenvalues ​​must be less than 0. i .

[0023] Furthermore, let the intermediate feature vector Except for o i Other feature values ​​besides those mentioned above are randomly generated, including:

[0024] Let the output layer feature vector Except for y i Other eigenvalues ​​of the intermediate eigenvector Except for o i Randomly map other feature values ​​besides those specified.

[0025] Secondly, the present invention provides a deep model protection system against knowledge distillation, comprising:

[0026] The acquisition and loading module is used to acquire the output layer feature vector sent by the protected deep model;

[0027] The feature reshaping module is used to reshape the output layer feature vector to obtain a reshaped feature vector, wherein the reshaped feature vector has the same dimension and the same position of the feature maximum value as the output layer feature vector.

[0028] The encoding conversion module is used to perform one-hot encoding conversion on the reorganized feature vector to obtain the category number;

[0029] The query output module is used to obtain and output the image recognition results based on the category number and the pre-determined association between the category number and the image recognition results.

[0030] Furthermore, it also includes a normalization module, which is used to normalize the eigenvalues ​​of the renormalized eigenvector, so that the range of eigenvalues ​​is converted to [0,1] and the sum of the eigenvalues ​​is 1.

[0031] Thirdly, the present invention provides a deep model protection device against knowledge distillation, characterized in that it includes a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method described in the first aspect.

[0032] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the steps of the method described in the first aspect.

[0033] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0034] 1. This invention utilizes a feature renormalization function to replace the output layer feature vector while maintaining the dimension and the position of the feature maximum value, thus preventing the deep model from being knowledge distilled. It does not require retraining the deep model and can effectively improve the security of model protection without increasing the additional computational cost.

[0035] 2. This invention has strong versatility and scalability, is plug-and-play, and can protect all deep models whose outputs are described by the maximum value of vectors. In addition, it is highly flexible, requiring only the feature vector of the last layer of the protected model to perform model protection, and has a wide range of applications.

[0036] 3. By normalizing the eigenvalues ​​of the reorganized feature vector, this invention can eliminate data anomalies in the reorganized feature vector, making it impossible for the theft model to detect that the acquired output layer feature vector has been replaced by the reorganized feature vector, thereby increasing the difficulty of theft and enhancing the protection effect of the deep model. Attached Figure Description

[0037] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0038] Figure 1 This is a flowchart of a deep model protection method against knowledge distillation provided in an embodiment of the present invention;

[0039] Figure 2 yes Figure 1 The diagram shows the principle block diagram of the deep model protection method against knowledge distillation. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0041] Example 1:

[0042] For model theft based on knowledge distillation, this behavior aims to learn the behavior of the protected model by measuring the input and output of the unknown model and then using knowledge distillation, thereby achieving model theft, when the model's structural parameters are unknown. For the protected model, using a sample whose input and output can be measured, a loss function can be constructed as shown below using Kullback-Leibler divergence, allowing the theft model to simulate the behavior of the protected model:

[0043]

[0044] In the formula, Indicates the protected model. Represents the theft model, θ T θ represents the parameters of the protected model. S This indicates the parameters of the stolen model, KL() represents the Kullback-Leibler divergence, and x i This represents the input data image.

[0045] As can be seen from the loss function above, if the input of the protected model is known, the stolen model can approximate the output of the protected model after learning from a large number of samples.

[0046] To protect the behavior of a model from being stolen, this invention provides a method for protecting deep models against knowledge distillation, comprising the following steps:

[0047] Step 1: Obtain the output layer feature vector sent by the protected depth model.

[0048] Specifically, the output layer feature vector is a feature encoding, which is assumed to be [0.1,0.2,0.1,0.5,0.04,0.001,0.02,0.005,0.02,0.009,0.005] in this embodiment; its dimension is 11, and the maximum value of the feature is 0.5 in the 4th position.

[0049] Step 2: Reshape the output layer feature vector to obtain a reshaped feature vector, wherein the reshaped feature vector has the same dimension and the same position of the feature maximum value as the output layer feature vector.

[0050] It should be noted that the remodeled feature vector is located in the same position as the maximum feature value of the input layer feature vector. This is to ensure that the results obtained by the user during the use of the deep model remain unchanged. However, the remodeled feature vector needs to change the magnitude of other feature values ​​in the input layer feature vector. This is to ensure that the user cannot obtain the feature distribution output by the protected model, so that it does not meet the conditions for knowledge distillation.

[0051] In this embodiment, the specific process of reshaping the output layer feature vector and obtaining the reshaped feature vector in step 2 includes:

[0052] Step 21, based on the output layer feature vector Generate intermediate feature vectors The intermediate feature vector and the output layer feature vector have the same dimension, n-dimensional.

[0053] Step 22, obtain the output layer feature vector. The eigenvalue y in i And let the intermediate feature vector o in i =y i .

[0054] Step 23, let the intermediate feature vector Except for o i Other eigenvalues ​​are randomly generated to obtain a renormalized eigenvector, wherein all randomly generated eigenvalues ​​must be less than 0. i .

[0055] Preferably, step 23 involves setting the intermediate feature vector... Except for o i Methods for randomly generating other feature values ​​include:

[0056] Let the output layer feature vector Except for y i Other eigenvalues ​​of the intermediate eigenvector Except for o i Randomly map other feature values ​​besides those specified.

[0057] It should be noted that the method for randomly generating feature values ​​is not limited to the random mapping mentioned above. It can also be a mapping with certain rules or a random number sequence generated by combining the srand() function with the rand() function. The specific method is not limited, as long as the maximum value of the feature remains unchanged.

[0058] Step 3: Normalize the eigenvalues ​​of the renormalized eigenvector so that the range of the eigenvalues ​​is converted to [0,1] and the sum of the eigenvalues ​​is 1.

[0059] Preferably, the reshaped feature vector is processed using the Softmax function. The eigenvalues ​​are normalized.

[0060] The formula for the Softmax function is as follows:

[0061]

[0062] In the formula, z i Let represent the eigenvalue of the i-th dimension in the reorganized eigenvector, and C represent the total number of dimensions of the reorganized eigenvector.

[0063] It can be understood that by normalizing the feature values ​​in step 3, the data anomalies of the reshaped feature vector can be eliminated, making the theft model unable to perceive that the obtained output layer feature vector has been replaced by the reshaped feature vector, thereby increasing the difficulty of theft and enhancing the protection effect of the deep model.

[0064] Specifically, in this embodiment, one result of the reorganized feature vector obtained through the above steps can be represented as: [0.2,0.1,0,0.6,0.001,0.04,0.005,0.02,0.02,0.009,0.005]; its dimension is 11, and the maximum feature value is 0.6 of the 4th position.

[0065] Step 4: Perform one-hot encoding transformation on the reorganized feature vector to obtain the category number.

[0066] It should be noted that one-hot encoding is also known as one-bit valid encoding. For example, [0,1,0,0,0,0,0,0,0,0] This encoding method uses an N-bit state register to encode N states. Each state has its own independent register bit, and at any given time, only one bit is valid.

[0067] Specifically, in this embodiment, the output layer feature vector is transformed by one-hot encoding, and the resulting category number is [0,0,0,1,0,0,0,0,0,0,0]; while the reorganized feature vector in this embodiment is transformed by one-hot encoding, and the resulting category number is also [0,0,0,1,0,0,0,0,0,0,0]; the two are the same, that is, the maximum value is 1 in the 4th position, and the other positions are set to 0, so it will not affect the correctness of the subsequent image recognition results.

[0068] Understandably, because reshaping disrupts the distribution of the original features, it can prevent the protected model from being stolen by knowledge distillation.

[0069] Step 5: Based on the category number and the predetermined association between the category number and the image recognition result, obtain the image recognition result and output it.

[0070] Specifically, in this embodiment, the association between the category number and the image recognition result is as follows: [cat, dog, rabbit, fish, sheep, cow, pig, tiger, dragon, snake].

[0071] Therefore, the image recognition result corresponding to category number [0,0,0,1,0,0,0,0,0,0,0] is: fish.

[0072] Example 2:

[0073] This embodiment provides a deep model protection system to prevent knowledge distillation, including:

[0074] The acquisition and loading module is used to acquire the output layer feature vector sent by the protected deep model;

[0075] The feature reshaping module is used to reshape the output layer feature vector to obtain the reshaped feature vector, where the reshaped feature vector has the same dimension and the same position of the feature maximum as the output layer feature vector.

[0076] The encoding conversion module is used to perform one-hot encoding conversion on the reorganized feature vector to obtain the category number;

[0077] The query output module is used to query and output the image recognition results based on the category number and the pre-determined association between the category number and the image recognition results.

[0078] In this embodiment, a normalization module is also included, which is used to normalize the eigenvalues ​​of the renormalized eigenvector so that the range of the eigenvalues ​​is converted to [0,1] and the sum of the eigenvalues ​​is 1.

[0079] Example 3:

[0080] This embodiment provides a computer-readable storage medium for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in Embodiment 1.

[0081] Example 4

[0082] This embodiment provides a computing device, including:

[0083] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described in Embodiment 1.

[0084] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for protecting deep models against knowledge distillation, characterized in that, Includes the following steps: Obtain the output layer feature vector sent by the protected deep model; The output layer feature vector is reshaped to obtain a reshaped feature vector, wherein the reshaped feature vector has the same dimension and the same position of the feature maximum value as the output layer feature vector. The reorganized feature vector is transformed by one-hot encoding to obtain the category number; Based on the category number and the pre-determined correlation between the category number and the image recognition result, the image recognition result is obtained and output; The output layer feature vector is reshaped to obtain the reshaped feature vector, which includes: Based on the output layer feature vector Generate intermediate feature vectors The intermediate feature vector and the output layer feature vector have the same dimension. dimension; Obtain the output layer feature vector eigenvalues ​​in And let the intermediate feature vector In ; Let the intermediate feature vector Except Other eigenvalues ​​are randomly generated to obtain a renormalized eigenvector, wherein all randomly generated eigenvalues ​​must be less than 1. .

2. The method for protecting deep models against knowledge distillation according to claim 1, characterized in that, Before performing one-hot encoding on the reorganized feature vector to obtain the category number, the following steps are also included: The eigenvalues ​​of the renormalized eigenvector are normalized to convert the range of eigenvalues ​​to a normalized range. And the sum of the eigenvalues ​​is 1.

3. The method for protecting deep models against knowledge distillation according to claim 2, characterized in that, The method for normalizing the eigenvalues ​​of the renormalized eigenvector includes: pass The function on the renormalized eigenvector The eigenvalues ​​are normalized. The The formula for the function is expressed as: ； In the formula, Represents the th eigenvector of the reorganized eigenvector dimensional eigenvalues This represents the total dimension of the remodeled eigenvectors.

4. The method for protecting deep models against knowledge distillation according to claim 1, characterized in that, Let the intermediate feature vector Except Other feature values ​​besides those mentioned above are randomly generated, including: Let the output layer feature vector Except Other eigenvalues ​​of the intermediate eigenvector Except Randomly map other feature values ​​besides those specified.

5. A deep model protection system against knowledge distillation, characterized in that, The system is used to perform the deep model protection method against knowledge distillation as described in any one of claims 1-4, the system comprising: The acquisition and loading module is used to acquire the output layer feature vector sent by the protected deep model; The feature reshaping module is used to reshape the output layer feature vector to obtain the reshaped feature vector, where the reshaped feature vector has the same dimension and the same position of the feature maximum as the output layer feature vector. The encoding conversion module is used to perform one-hot encoding conversion on the reorganized feature vector to obtain the category number; The query output module is used to obtain and output the image recognition results based on the category number and the pre-determined association between the category number and the image recognition results.

6. The deep model protection system against knowledge distillation according to claim 5, characterized in that, Also includes: The normalization module is used to normalize the eigenvalues ​​of the renormalized eigenvectors, converting the range of eigenvalues ​​to a normalized range. And the sum of the eigenvalues ​​is 1.

7. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any one of the methods according to claims 1 to 4.

8. A computing device, characterized in that, include: One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including instructions for performing any one of the methods according to claims 1 to 4.

Images