A copyright protection method for deep learning models based on Monte Carlo tree search

Abstract

This invention provides a method for copyright protection of deep learning models based on Monte Carlo tree search. The method first selects several intermediate layers in the deep learning model to be protected, and then uses Monte Carlo tree search to search for a small number of parameters in these intermediate layers that have a significant impact on model performance. By applying carefully crafted perturbations to these parameters, the model's performance is significantly reduced, thereby preventing malicious infringers from using the model. Authorized users can use a key to remove the perturbations added to the model parameters, thus restoring the model's original good performance and allowing normal use of the model.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to a copyright protection method for deep learning models based on Monte Carlo tree search. Background Technology

[0002] With the rapid development of deep learning technology, deep neural networks (DNNs) have been deployed in an increasing number of commercial scenarios due to their excellent performance. With the rise of Machine Learning as a Service (MLaaS), many commercial companies upload their trained high-performance models to the cloud and provide services to the outside world for profit. In this scenario, users can only obtain the model's prediction results and cannot access its internal parameters. Training a high-precision DNN model is a rather expensive and time-consuming task, and these trained DNN models should be considered intellectual property (IP) of the model trainer and protected. However, some malicious infringers may pirate trained models and illegally use these pirated models to profit, which greatly infringes on the intellectual property rights of the model owner.

[0003] Most existing deep learning model copyright protection methods are watermark-based authentication methods. In these methods, the model owner embeds a watermark into the DNN model. When the model is pirated, the owner can extract the watermark from the pirated model to authenticate their ownership. However, these DNN model watermark authentication methods are passive; the model owner can only passively authenticate their ownership after the model has been pirated, and cannot prevent infringers from illegally using the model. Therefore, it is necessary to address the problem that existing watermark methods can only passively authenticate ownership after model piracy and to achieve proactive authorization control for deep learning models. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a copyright protection method for deep learning models based on Monte Carlo tree search. This method uses Monte Carlo tree search to identify parameters that significantly impact model performance and modifies these parameters, causing a substantial decrease in model performance. This prevents malicious infringers from using the model, while authorized users with the key can decrypt the model and use it normally.

[0005] The objective of this invention is achieved through the following technical solution: a copyright protection method for deep learning models based on Monte Carlo tree search, the method comprising the following steps:

[0006] (1) Training Model F: The training process involves inputting training data into the deep learning model and continuously updating the model parameters using the gradient of the loss function until the loss function is less than a given threshold of 0.01; selecting one-tenth of the labeled data from the training set of the deep learning model, with the number of selected data points denoted as N; and forming an encrypted dataset D from these N data points. e =(X e ,Y e ); where X e = {x1, ..., x} N , and Y e ={y1,...,y N} represents the data set and label set of the encrypted dataset, respectively;

[0007] (2) Randomly select a certain number of intermediate layers from the deep learning model F. Each selected intermediate layer is called a cryptographic layer. All modified parameters are selected from the cryptographic layers. The set of all cryptographic layer numbers is denoted as . Construct the root node N0 of the Monte Carlo tree, which is an empty node. Set the iteration count i = 0, select the root node N0 as the current node, and set the computation cost c. Each node of the Monte Carlo tree contains: the position of the selection parameter, the amount of change of the selection parameter, and the deep learning model F on the encrypted dataset D. e =(X e ,Y e The loss function value on the node and the number of times the node was visited;

[0008] (3) Select one of the child nodes of the current node to explore, specifically:

[0009] (3.1) If the current node has no child nodes, then generate a child node for the current node, that is, randomly select a parameter w in the intermediate layer l and modify the parameter;

[0010] (3.2) If the current node has child nodes, then with a 50% probability, select the child node with the highest priority score from the child nodes of the current node according to the confidence interval upper bound algorithm, after weighing the node loss function value and the number of times the node is visited, and explore the selected child node; generate a child node for the current node with a 50% probability, and select the generated child node for exploration.

[0011] (4) Perform an expansion operation on the child node selected in step (3.2), that is, use the same method as in step (3.1) to generate a child node for the selected child node. This child node is called the expanded node.

[0012] (5) Perform simulation operations on the extended nodes generated in step (4). Within the maximum number of simulation rounds s, select one parameter from the intermediate layer l for modification in each simulation. After the simulation is completed, obtain the deep learning model F on the encrypted dataset D. e The loss function value on;

[0013] (6) Perform backpropagation operation, backpropagate the loss function value obtained in step (5) to the extended node and all its parent nodes, and increment the number of visits of all parent nodes of the extended node by 1;

[0014] (7) Continuously execute steps (3) to (6) until the computation cost c defined in step (2) is reached. After reaching the computation cost c, use the same confidence interval upper bound algorithm in step (3.2) to select the child node with the highest priority in the current node as the current node of the next iteration round, and set the iteration number i = i + 1.

[0015] (8) If the loss function value of the current node is greater than the given threshold T or the maximum number of iterations is reached, the Monte Carlo tree search process is terminated; otherwise, steps (3) to (7) are continued until the loss function value of the current node is greater than the given threshold T or the maximum number of iterations is reached. At this point, the performance of the model has deteriorated to the point that it cannot be used normally.

[0016] (9) Find the path from the root node N0 to the current node in the Monte Carlo tree, record the position of the parameter stored in each node on the path and the corresponding parameter change, and the recorded information forms a key; the model owner distributes the key to authorized users, and authorized users can use the key to find the position of the modified parameter and restore the modified parameter, so as to use the model normally; while for unauthorized users, the performance of the model will be reduced to the point that it cannot be used.

[0017] Furthermore, in step (1), the deep learning model can be any of the following: convolutional neural network, graph neural network, or recurrent neural network.

[0018] Furthermore, in step (2), the selected intermediate layers include fully connected layers and convolutional layers.

[0019] Furthermore, in step (3), the position of the selection parameter in the root node N0 and the amount of change of the selection parameter are both set to empty, the loss function value is set to 0, and the number of times the node is visited is set to 0.

[0020] Furthermore, in step (3), the loss function of the deep learning model can be either the cross-entropy loss function or the mean squared error loss function.

[0021] Furthermore, in step (3.1), the parameter is modified, and the mathematical expression for the modification amount η is as follows:

[0022] Where θ is a hyperparameter, sign represents the sign function, and L(F,D) e W represents the loss function value of the deep learning model F on the encrypted dataset. l This represents the set of parameters for the intermediate layer l; select the generated child nodes for exploration.

[0023] The beneficial effects of this invention are as follows:

[0024] This invention implements proactive authorization control, ensuring that only authorized users can properly use the deep learning model. For unauthorized users, the model's performance will drastically degrade. Compared to existing DNN watermarking methods, this method prevents unauthorized users from using the model, thus more effectively protecting the copyright of the DNN model. Furthermore, most watermarking methods retrain the model to embed the watermark, introducing significant computational overhead and completely altering the model's parameters. In contrast, this invention requires encrypting only a minimal number of parameters, making minimal changes to the model, and incurring only low computational overhead. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the algorithm flow of an embodiment of the present invention. Detailed Implementation

[0026] The following is in conjunction with the appendix Figure 1 The technical solution of the present invention will be further described in detail below.

[0027] This invention proposes a copyright protection method for deep learning models based on Monte Carlo tree search, which includes the following steps:

[0028] (1) Training Model F: The training process involves inputting training data into the deep learning model and continuously updating the model parameters using the gradient of the loss function until the loss function is less than a given threshold of 0.01; selecting one-tenth of the labeled data from the training set of the deep learning model, with the number of selected data points denoted as N; and forming an encrypted dataset D from these N data points. e =(X e ,Y e ); where X e = {x1, ..., x} N , and Y e ={y1,...,y N} represents the data set and label set of the encrypted dataset, respectively;

[0029] (2) Randomly select a certain number of intermediate layers from the deep learning model F. Each selected intermediate layer is called a cryptographic layer. All modified parameters are selected from the cryptographic layers. The set of all cryptographic layer numbers is denoted as . Construct the root node N0 of the Monte Carlo tree, which is an empty node. Set the iteration count i = 0, select the root node N0 as the current node, and set the computation cost c. Each node of the Monte Carlo tree contains: the position of the selection parameter, the amount of change of the selection parameter, and the deep learning model F on the encrypted dataset D. e =(X e ,Y e The loss function value on the node and the number of times the node was visited;

[0030] (3) Select one of the child nodes of the current node to explore, specifically:

[0031] (3.1) If the current node has no child nodes, then generate a child node for the current node, that is, randomly select a parameter w in the intermediate layer l and modify the parameter;

[0032] (3.2) If the current node has child nodes, then with a 50% probability, select the child node with the highest priority score from the child nodes of the current node according to the confidence interval upper bound algorithm, after weighing the node loss function value and the number of times the node is visited, and explore the selected child node; generate a child node for the current node with a 50% probability, and select the generated child node for exploration.

[0033] (4) Perform an expansion operation on the child node selected in step (3.2), that is, use the same method as in step (3.1) to generate a child node for the selected child node. This child node is called the expanded node.

[0034] (5) Perform simulation operations on the extended nodes generated in step (4). Within the maximum number of simulation rounds s, select one parameter from the intermediate layer l for modification in each simulation. After the simulation is completed, obtain the deep learning model F on the encrypted dataset D. e The loss function value on;

[0035] (6) Perform backpropagation operation, backpropagate the loss function value obtained in step (5) to the extended node and all its parent nodes, and increment the number of visits of all parent nodes of the extended node by 1;

[0036] (7) Continuously execute steps (3) to (6) until the computation cost c defined in step (2) is reached. After reaching the computation cost c, use the same confidence interval upper bound algorithm in step (3.2) to select the child node with the highest priority in the current node as the current node of the next iteration round, and set the iteration number i = i + 1.

[0037] (8) If the loss function value of the current node is greater than the given threshold T or the maximum number of iterations is reached, the Monte Carlo tree search process is terminated; otherwise, steps (3) to (7) are continued until the loss function value of the current node is greater than the given threshold T or the maximum number of iterations is reached. At this point, the performance of the model has deteriorated to the point that it cannot be used normally.

[0038] (9) Find the path from the root node N0 to the current node in the Monte Carlo tree, record the position of the parameter stored in each node on the path and the corresponding parameter change, and the recorded information forms a key; the model owner distributes the key to authorized users, and authorized users can use the key to find the position of the modified parameter and restore the modified parameter, so as to use the model normally; while for unauthorized users, the performance of the model will be reduced to the point that it cannot be used.

[0039] Based on the current node, select a node to explore, specifically:

[0040] (4.1) If the current node has no child nodes, then a child node is generated for the current node. That is, a parameter w is randomly selected in the intermediate layer l, and the parameter is modified. The change η can be expressed as: Where θ is a hyperparameter, sign represents the sign function, and L(F,D) e W represents the loss function value of the deep learning model F on the encrypted dataset. l This represents the set of parameters for the intermediate layer l; select the generated child nodes for exploration.

[0041] (4.2) If the current node has child nodes, then with a 50% probability, select the child node with the highest priority score from among the child nodes of this node according to the confidence interval upper bound algorithm, after weighing the node loss function value and the number of times the node has been visited, and explore the selected child node. Generate a child node for the current node with a 50% probability, and explore the selected generated child node.

[0042] The deep learning model can be any of the following: convolutional neural network, graph neural network, or recurrent neural network.

[0043] The selected intermediate layers include fully connected layers and convolutional layers.

[0044] The position and change amount of the selection parameters in the root node N0 are both set to empty, the loss function value is set to 0, and the number of times the node is visited is set to 0.

[0045] The loss function of the deep learning model can be either the cross-entropy loss function or the mean squared error loss function.

[0046] This invention encrypts the model by using Monte Carlo tree search to find and modify a small number of parameters that significantly impact model performance, resulting in a substantial performance decrease. Authorized users with the key can decrypt the model and use it normally. For unauthorized users, the encrypted model is rendered unusable, effectively protecting the copyright of the deep learning model. Compared to existing passive watermarking methods, this invention enables proactive authorization control, preventing unauthorized users from using the model and thus more effectively protecting its copyright. Since deep learning models typically contain a large number of parameters, directly encrypting all parameters would introduce significant time costs. This invention only encrypts a small number of parameters that have a significant impact on model performance, thus minimizing the time cost.

Claims

1. A copyright protection method for deep learning models based on Monte Carlo tree search, characterized in that, The method includes the following steps: (1) Training the model The training process involves inputting training data into a deep learning model and continuously updating the model's parameters using the gradient of the loss function until the loss function is less than a given threshold of 0.

01. One-tenth of the labeled data is selected from the training set of the deep learning model; the number of selected data points is denoted as . ; by this The encrypted dataset consists of several data points. ;in, and These are the data set and the label set of the encrypted dataset, respectively; (2) From deep learning models A certain number of intermediate layers are randomly selected. Each selected intermediate layer is called an encryption layer. All modified parameters are selected from the encryption layers. The set of all encryption layer numbers is denoted as . Construct the root node of the Monte Carlo tree. The root node For empty nodes, set the number of iterations. Select the root node As the current node, set the computation cost. Each node of the Monte Carlo tree contains: the location of the selection parameters, the amount of change in the selection parameters, and the deep learning model. In encrypted datasets The loss function value and the number of times the node was visited; (3) Select one of the child nodes of the current node to explore, specifically: (3.1) If the current node has no child nodes, then generate a child node for the current node, that is, in the intermediate layer. Randomly select a parameter And modify that parameter; (3.2) If the current node has child nodes, then with a 50% probability, select the child node with the highest priority score from the child nodes of the current node according to the confidence interval upper bound algorithm, after weighing the node loss function value and the number of times the node is visited, and explore the selected child node; generate a child node for the current node with a 50% probability, and select the generated child node for exploration. (4) Perform an expansion operation on the child node selected in step (3.2), that is, use the same method as in step (3.1) to generate a child node for the selected child node. This child node is called the expanded node. (5) Perform simulation operations on the extended nodes generated in step (4), in the maximum simulation round. Internally, an intermediate layer is selected in each simulation. One of the parameters is modified; after the simulation is completed, the deep learning model is obtained. In encrypted datasets The loss function value on; (6) Perform backpropagation operation, backpropagate the loss function value obtained in step (5) to the extended node and all its parent nodes, and increment the number of visits of all parent nodes of the extended node by 1; (7) Continuously execute steps (3) to (6) until the computational cost defined in step (2) is reached. To achieve computational cost Then, using the same confidence interval upper bound algorithm as in step (3.2), the highest priority child node among the current nodes is selected as the current node for the next iteration round, and the number of iterations is set. ; (8) If the loss function value of the current node is greater than the given threshold If the maximum number of iterations is reached, the Monte Carlo tree search process terminates; otherwise, continue executing steps (3) to (7) until the loss function value of the current node is greater than the given threshold. Or it may reach the maximum number of iterations, at which point the model's performance has deteriorated to the point that it can no longer be used normally; (9) Find the root node in the Monte Carlo tree The path to the current node records the location of the parameters stored in each node along the path, as well as the corresponding parameter changes. The recorded information forms a key. The model owner distributes the key to authorized users. Authorized users can use the key to find the location of the modified parameters and restore the modified parameters, thus using the model normally. However, for unauthorized users, the model's performance will degrade to the point that it cannot be used.

2. The copyright protection method for a deep learning model based on Monte Carlo tree search according to claim 1, characterized in that, In step (1), the deep learning model can be any of the following: convolutional neural network, graph neural network, or recurrent neural network.

3. The copyright protection method for a deep learning model based on Monte Carlo tree search according to claim 1, characterized in that, In step (2), the selected intermediate layers include fully connected layers and convolutional layers.

4. The copyright protection method for a deep learning model based on Monte Carlo tree search according to claim 1, characterized in that, In step (3), the root node The position and change amount of the selection parameters are both set to empty, the loss function value is set to 0, and the number of times the node is visited is set to 0.

5. The copyright protection method for a deep learning model based on Monte Carlo tree search according to claim 1, characterized in that, In step (3), the loss function of the deep learning model can be either the cross-entropy loss function or the mean squared error loss function.

6. The copyright protection method for a deep learning model based on Monte Carlo tree search according to claim 1, characterized in that, In step (3.1), the parameter is modified, and the amount of modification is... The mathematical expression is ; in, For hyperparameters, Represents a symbolic function. Representing deep learning models The loss function value on the encrypted dataset. Indicates the intermediate layer The parameter set; select the generated child nodes to explore.

Images