An edge model protection method based on an authorization verification mechanism and a system thereof

Abstract

The application discloses an edge model protection method based on an authorization verification mechanism and a system thereof. In the method, an authorization and authentication layer is designed for the input feature map of a selected convolution kernel to protect the model. The presence of the authentication layer in the network makes the network unable to normally operate with an unauthorized feature map, and the accuracy of the network is greatly degraded, which plays a role in protecting the model. Meanwhile, the authorized feature map can obtain the original feature map through the operation of the authentication layer, which ensures the normal use of the network. In addition, in order to successfully deploy the method, a system for implementing the edge model protection method is also disclosed. The application utilizes deep learning technology and an authorization and authentication mechanism, and can realize effective and light edge model protection, which greatly protects the structure and parameters of the edge model.

Description

Technical Field

[0001] This invention relates to the field of deep learning model protection, and in particular to an edge-end model protection method and system based on an authorization verification mechanism. Background Technology

[0002] Deep learning models are now widely used in various scenarios, such as road sign recognition for autonomous vehicles and medical image analysis. On the one hand, training deep neural network (DNN) models is very costly; owners often need to invest significant resources to train a model capable of solving real-world tasks. On the other hand, as the application scope of DNN models becomes increasingly widespread, deploying them directly on edge devices is becoming a trend. Recent research shows that the number of deep learning model embeds in the Google Play Store nearly doubled from February 2020 to April 2021.

[0003] However, these edge device models are vulnerable to theft and unauthorized access by attackers. These DNN models deployed on edge devices can be reverse engineered to obtain their structure and parameters. When these edge device models are unprotected, attackers can easily access and abuse them, causing significant financial losses to the model owners. Therefore, an effective edge device model protection technology is urgently needed to protect the interests of model owners.

[0004] Edge models consist of feature extractors and classifiers. Feature extractors are mainly composed of convolutional kernels, while classifiers are primarily fully connected layers. Current edge model protection techniques mainly rely on model encryption. These methods encrypt the model, allowing publishers to deploy it to hardware devices. While these methods protect the model during the storage phase, the security of the execution environment is often difficult to guarantee. Researchers have found that once these DNN models are decrypted and run, the model in its execution state can still be captured and reverse-engineered by attackers to obtain its high-level representation, posing a challenge to model encryption-based methods.

[0005] To protect the running model, some methods place it within a Trusted Execution Environment (TEE). A TEE is a secure computing environment—an isolated, protected computing space that ensures that code and data running within it cannot be accessed, tampered with, or leaked by unauthorized entities. However, running the model within a TEE incurs significant runtime overhead, which is unacceptable in practical applications.

[0006] In summary, the key to protecting the edge model lies in three aspects: (1) effectively protecting the structure and parameters of the model; (2) protecting it not only in the storage state but also during execution; and (3) minimizing the runtime overhead of the model. Summary of the Invention

[0007] This invention addresses the shortcomings of existing technologies by proposing a method and system for protecting edge-end models based on an authorization verification mechanism. This invention employs an authorization verification mechanism to isolate the right to use the model, thereby protecting the intellectual property rights of the model. Under authorized use, the protected model can achieve almost the same accuracy as the unprotected model; without authorization, the accuracy of the protected model will significantly degrade. The protection scheme proposed in this invention achieves effective protection of the intellectual property rights of edge-end models with minimal additional computational overhead.

[0008] This invention first provides an edge-end model protection method based on an authorization verification mechanism, comprising the following steps:

[0009] 1) In the edge-end model, determine the convolutional kernel to be protected and the authorization parameters; obfuscate the convolutional kernel to be protected; the edge-end model includes a feature extractor and a classifier, the feature extractor includes a convolutional kernel, and the classifier includes a fully connected layer; the authorization parameters include an authorization matrix and an authorization factor;

[0010] 2) Construct the authorization module and generate the verification layer based on the authorization parameters in step 1);

[0011] 3) Deploy the authorization module built in step 2) to the edge model, and embed the generated verification layer into the edge model to complete the construction of the edge model protection system.

[0012] As a preferred embodiment of the present invention, in step 1), the convolution kernel to be protected is determined based on the distance between each convolution kernel in the edge model and the input layer, as well as the number of parameters of each convolution kernel.

[0013] As a preferred embodiment of the present invention, the authorization matrix in step 1) is a matrix with the same dimension as the input feature map of the convolution kernel to be protected, wherein each element of the matrix is ​​generated by a random number; the random number is generated according to a random distribution; and the authorization factor is a floating-point number, which is directly generated by a random distribution.

[0014] As a preferred embodiment of the present invention, the obfuscation of the convolution kernel to be protected in step 1) specifically involves: introducing an obfuscation factor α into the convolution kernel to be protected [w]·[x1]+[b]=[y1], and expanding all the parameters of the original convolution kernel to be protected by a factor of α to obfuscate it, resulting in an obfuscated convolution kernel of α[w]·[x1]+α[b]=α[y1]; where w is an equivalent product matrix constructed based on the parameters of the convolution kernel to be protected, b is an equivalent bias matrix constructed based on the parameters of the convolution kernel to be protected, x1 is the input feature map, and y1 is the output of the convolution kernel to be protected after inputting the feature map.

[0015] In a preferred embodiment of the present invention, in step 2), the verification layer includes a multiplication part and an addition part; the multiplication element γ of the multiplication part is:

[0016]

[0017] The matrix element [d] for the addition part is:

[0018]

[0019] [w]·[k]+[b]=[y2]

[0020] Where β is the authorization factor in the authorization parameters; k is the authorization matrix in the authorization parameters; and y2 is the output of the convolution kernel operation to be protected on the input authorization matrix.

[0021] As a preferred embodiment of the present invention, in step 3), deploying the authorization module specifically involves: deploying the authorization module separately in a secure environment, and using the authorization module to authorize the input feature map of the obfuscated convolutional kernel;

[0022] As a preferred embodiment of the present invention, in step 3), the generated verification layer is embedded into the neural network and then the output of the obfuscated convolutional kernel is processed.

[0023] As a preferred embodiment of the present invention, the secure environment is a trusted execution environment.

[0024] The present invention also provides a system for implementing the above-described edge-end model protection method, comprising:

[0025] The kernel obfuscation module is used to determine the kernels to be protected and obfuscate them.

[0026] The authorization module is placed independently in a secure environment; it authorizes the input feature map by determining the authorization parameters of the convolution kernel to be protected, and then transmits the authorized feature map to the confused convolution kernel for convolution.

[0027] The verification layer generation module is used to construct a verification layer based on the authorization parameters, embed the verification layer after the obfuscated convolutional kernel, and perform calculations on the output of the obfuscated convolutional kernel through the verification layer.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] 1) This invention employs a mechanism that authorizes the use of edge models. This mechanism authorizes the use of models during inference, preventing unauthorized deep learning from being used, thus effectively protecting deep learning models from unauthorized use. Therefore, it overcomes the problem in existing technologies that cannot protect models during runtime, thereby achieving effective protection of deep learning models on edge devices.

[0030] 2) Because the authorization mechanism proposed in this invention can protect the model without affecting its accuracy, it overcomes the problem that adding defense to the model in the prior art will cause a loss of accuracy, thus greatly improving the practicality of the method and avoiding the need to compromise between model accuracy and robustness.

[0031] 3) Because the authorization process proposed in this invention only requires one matrix multiplication operation and one matrix addition operation, the additional overhead of the method in a trusted execution environment is very small compared to the original model overhead. Therefore, it overcomes the problem that most existing methods significantly increase model overhead, making the method more lightweight in practical use and avoiding trade-offs between system computational overhead and defense robustness. Attached Figure Description

[0032] Figure 1 This is a flowchart of the authorization verification method of the present invention;

[0033] Figure 2 This is a schematic diagram of the verification layer generation and embedding process of the present invention;

[0034] Figure 3 This is a schematic diagram of the operation process of the authorization verification system of the present invention. Detailed Implementation

[0035] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0036] like Figure 1 As shown, to address the issue of intellectual property protection for edge models, this invention proposes an edge model protection method and system based on an authorization verification mechanism. The specific steps are as follows:

[0037] 1) Dataset preparation and preprocessing

[0038] Prepare a training dataset according to the model task. In this example, we take the image classification task of the CIFAR-100 dataset as an example.

[0039] 1-1) The CIFAR-100 dataset was used as the initial sample set.

[0040] CIFAR-100 (Canadian Institute for Advanced Research 100) is a widely used image dataset for computer vision research. It contains 100 categories, with 600 color images (32x32 pixels) for each category, totaling 60,000 images. The images in CIFAR-100 are diverse, covering a wide range of everyday objects, animals, and natural scenes. This dataset can be used for tasks such as image classification, object recognition, and image segmentation. This example demonstrates an image classification task on this dataset.

[0041] 1-2) Preprocess the dataset

[0042] The dataset is loaded and normalized to map image pixel values ​​to between 0 and 1 to avoid numerical instability. Furthermore, data augmentation techniques such as random cropping, horizontal flipping, and color dithering are applied to enrich the dataset and reduce overfitting. The images are then resized to the size required for model input, and class labels are one-hot encoded for easier model processing. The dataset is divided into training and validation sets in an 80:20 ratio for model training, tuning, and evaluation.

[0043] 2) Training the original model

[0044] Determine the model architecture for solving the classification task, and use the dataset divided in steps 1-2) to train the model and test its accuracy. In this example, the ResNet50 model architecture is used as an example.

[0045] 2-1) Construct a model for solving the task

[0046] The classification model used in this embodiment is the ResNet50 model, which mainly includes a feature extractor and a feature classifier. The feature extractor is mainly composed of convolutional layers, and the feature classifier is composed of three fully connected layers. The activation function of the network is the ReLU function.

[0047] 2-2) Training the original model

[0048] M is trained using the training set constructed in steps 1-2), with a batch size of 64. A warm-up learning rate strategy is used during training, and the Adam optimizer is employed. The loss function is in the form of cross-entropy, as shown in the following formula:

[0049]

[0050] Where C represents the number of categories, p i For the true classification of the sample, q i This represents the predicted confidence probability of the sample.

[0051] The model's prediction accuracy is tested using a test set. Once the model's accuracy stabilizes, training ends; otherwise, training continues.

[0052] 3) Determination of the convolutional layer to be protected and the authorization parameters

[0053] 3-1) Selection of Convolutional Layers to be Protected

[0054] When selecting a convolutional kernel from the network as the kernel to be protected, it is important to choose a kernel that is relatively important to the network. Such a kernel has a greater impact on the model's prediction results, and choosing such a kernel will result in better robustness of protection. Generally speaking, convolutional kernels with more parameters and closer to the input layer are more important. In this example, we directly select the convolutional layer with the largest number of parameters, and denote the selected kernel as Conv.

[0055] 3-2) Determining Authorization Parameters

[0056] The parameters used for authorizing the model's usage are determined. These parameters mainly consist of two parts: the authorization matrix [k] and the authorization factor β. [k] is a matrix with the same dimension as the input feature map of Conv, where each element is generated by random numbers. These random numbers are generated according to a random distribution (such as a uniform or Gaussian distribution) to ensure that the matrix values ​​are random within a certain range. β is a floating-point number directly generated from a random distribution.

[0057] 4) Generation of verification layer

[0058] 4-1) Obfuscation of convolutional layer parameters to be protected

[0059] Convolution operations typically involve sliding a convolution kernel across the input feature map to perform a weighted sum of local regions, generating an output feature map. For a convolution operation, an equivalent multiplication matrix can be used to transform the original convolution operation into a matrix multiplication operation. Based on this principle, the essence of the convolution operation between the feature map and the convolution kernel is a matrix multiplication operation, represented by the following equation:

[0060] [w]·[x1]+[b]=[y1] (2)

[0061] Where w is the equivalent product matrix constructed based on the parameters of Conv, b is the equivalent bias matrix constructed based on the parameters of Conv, x1 is the input feature map, and y1 is the output of the convolution operation.

[0062] Now, we introduce a confusion factor α and multiply all the parameters of Conv by α, resulting in the following expression:

[0063] α[w]·[x1]+α[b]=α[y1] (3)

[0064] By obfuscating the parameters of the convolution kernel, we can hide them. The advantage of doing this is that even if an attacker may learn the details of the defense method in the future, they will not be able to directly obtain the parameters of the original network, which increases the robustness of the method. Let the obfuscated convolution kernel be FConv and use FConv to replace Conv in the original network.

[0065] 4-2) Construction of verification layer parameters

[0066] The validation layer mainly consists of two parts: a multiplication part and an addition part. When a feature map is input into the validation layer, it first undergoes one element-wise multiplication operation and one matrix addition operation. Let the multiplication element be denoted as γ, and the matrix element of the matrix addition operation be denoted as [d]. Values ​​are assigned to γ ​​and [d] respectively.

[0067]

[0068]

[0069] Of these, only y2 has not appeared before. The expression for y2 is given as follows:

[0070] [w]·[k]+[b]=[y2] (6)

[0071] Where k is the authorization matrix determined in step 3-2), and y2 is the output of the convolution operation.

[0072] 5) Deployment of the authorization-verification system

[0073] 5-1) Deployment of the authorization module

[0074] The authorization module is separated from the model and deployed separately in a secure environment, such as a TEE. This method uses the authorization module to authorize the input feature map of FConv. The authorization process for the FConv input feature map involves first performing element-wise multiplication with β, and then performing matrix addition with [k]. The expression for the authorized feature map input into the obfuscated FConv is as follows:

[0075] α[w]·(β[x1]+[k])+α[b]=α(β[y1]+[y2]-β[b]) (7)

[0076] 5-2) Embedding the verification layer

[0077] like Figure 2 As shown, the validation layer is embedded after FConv and performs calculations on the output of FConv. At this time, the output of the convolution kernel in equation (7) is not the same as the original output y1 in equation (2). In order to ensure that the network can still operate normally, we need to use the validation layer to compensate for the output of FConv. We embed the validation layer after FConv, and for a feature map authorized in step 5-1), it has the following expression:

[0078] γ(α(β[y1]+[y2]-β[b]))+[d]=[y1] (8)

[0079] Where γ is the multiplication operation of the verification layer constructed by equation (4), and [d] is the matrix addition operation of the verification layer constructed by equation (5).

[0080] Table 1 Comparison of accuracy between the original model and the model using the edge protection method of this invention.

[0081]

[0082] As shown in Table 1, calculations reveal that the original input feature map of Conv, after authorization, FConv calculations, and validation layer operations, reverts to the original output y1 in equation (2). This indicates that the entire authorization-validation process does not affect the accuracy of the original network. Meanwhile, as... Figure 3 As shown, because there is a verification layer in the network, unauthorized feature maps cannot obtain y1 after being processed by FConv and the verification layer. That is, unauthorized feature maps cannot make the network work properly, and the accuracy of the network will degrade significantly. This serves the purpose of protecting the model.

[0083] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for protecting edge-end models based on an authorization verification mechanism, characterized in that, Includes the following steps: 1) Determine the convolutional kernels to be protected based on the distance between the input layer and each layer's convolutional kernels in the edge model, as well as the number of parameters in each layer's convolutional kernels, and determine the authorization parameters in the edge model; obfuscate the convolutional kernels to be protected; The edge model includes a feature extractor and a classifier. The feature extractor includes a convolutional kernel, and the classifier includes a fully connected layer. The authorization parameters include the authorization matrix and the authorization factor; The authorization matrix is ​​a matrix with the same dimension as the input feature map of the convolutional kernel to be protected, wherein each element of the matrix is ​​generated by a random number; the random number is generated according to a random distribution; the authorization factor is a floating-point number, which is directly generated by a random distribution. Obfuscating the convolutional kernel to be protected specifically involves: obfuscating the convolutional kernel to be protected... Introducing a confusion factor And the parameters of the convolution kernels that were originally to be protected were all enlarged. This process is repeated to obfuscate the convolution kernel, resulting in a different kernel. ;in It is an equivalent product matrix constructed based on the parameters of the convolution kernel to be protected. It is an equivalent bias matrix constructed based on the parameters of the convolution kernel to be protected. It is the input feature map. It is the output of the convolution kernel operation to be protected after inputting the feature map; 2) Construct the authorization module and generate the verification layer based on the authorization parameters in step 1); The authorization module is placed independently in a secure environment. It authorizes the input feature map by determining the authorization parameters of the convolution kernel to be protected, and then transmits the authorized feature map to the confused convolution kernel for convolution. The verification layer includes a multiplication part and an addition part; Multiplication elements in the multiplication part for: ； Matrix elements of the addition part for: ； ； in, This refers to the authorization factor in the authorization parameters; For the authorization matrix in the authorization parameters, The input authorization matrix is ​​the output of the convolution kernel operation to be protected; 3) Deploy the authorization module built in step 2) to the edge model, and embed the generated verification layer into the edge model to complete the construction of the edge model protection system; protect the edge model through the constructed edge model protection system; the verification layer, after being embedded into the edge model, is used to perform calculations on the output of the obfuscated convolution kernel; The deployment of the authorization module specifically involves: deploying the authorization module separately in a secure environment, and using the authorization module to authorize the input feature maps of the obfuscated convolutional kernels; the authorized feature maps are then input into the obfuscated convolutional kernels as follows: ； The generated validation layer, after being embedded into the edge model, performs calculations on the output of the obfuscated convolutional kernel: 。 2. The edge-end model protection method according to claim 1, characterized in that, The secure environment is a trusted execution environment.

3. A system for implementing the edge-end model protection method of claim 1, characterized in that, include: The kernel obfuscation module is used to determine the kernels to be protected and obfuscate them. The authorization module is placed in a secure, independent environment. The input feature map is authorized using the authorization parameters of the convolution kernel to be protected, and the authorized feature map is then transferred to the confused convolution kernel for convolution. The verification layer generation module is used to construct a verification layer based on the authorization parameters, embed the verification layer after the obfuscated convolutional kernel, and perform calculations on the output of the obfuscated convolutional kernel through the verification layer.

Images