Data processing method and device for model partition watermarking

Abstract

The invention discloses a data processing method and device for model partition watermarks, and the method comprises the steps: obtaining an initial watermark set which is composed of a preset number of data extracted from a test set, determining the input data of each watermark of the initial watermark set according to a preset matrix rule, carrying out the sequential processing of the output data of each watermark of the initial watermark set, and obtaining a watermarking result; the method comprises the following steps: according to input data of watermarks and output data of the watermarks subjected to sequential processing, obtaining a processed watermark set according to the input data of the watermarks and the output data of the watermarks subjected to sequential processing, dividing the processed watermark set into a plurality of target watermark sets according to a partition watermark technology, and embedding the target watermark sets into the model to be embedded with the watermarks, and obtaining the model embedded with the watermark.

Description

Technical Field

[0001] The present application relates to the technical field of watermark data processing, and more specifically, to a data processing method and device for a model partition watermark. Background Art

[0002] Artificial intelligence technology, led by deep neural networks (DNNs), has developed rapidly, and its wide application in many fields such as medicine, finance, and autonomous driving has also demonstrated great potential. As a result, the security issues of deep neural network models have also received increasing attention. The training of deep neural network models is inseparable from the support of a large amount of precious data, which may involve sensitive areas. At the same time, developing and training these models requires a lot of time and energy, but the models themselves are extremely easy to copy and misappropriate. How to ensure the security of this data and how to protect the rights and interests of data owners have become issues that must be considered.

[0003] In the prior art, the concept of protecting the intellectual property rights of artificial intelligence models, including deep neural networks, is used by using model watermarks. The basic idea of ​​model watermarks is to apply digital watermarking technology to the field of artificial intelligence. Taking deep learning models as an example, it embeds a unique watermark or pattern into the weights or structure of DNNs, thereby authenticating and proving the ownership and authenticity of DNNs.

[0004] However, since watermarking in existing techniques modifies the weights or structures to slightly deviate from the originally trained model parameters, any subtle effect may cause the overall accuracy of the DNN to deteriorate.

[0005] Therefore, how to avoid the impact of embedded watermarks on the overall accuracy of DNN is an urgent problem to be solved in this application. Summary of the invention

[0006] In view of this, the present application discloses a data processing method and device for model partition watermark, aiming to avoid the impact of embedded watermark on the overall accuracy of DNN and to improve the security of data in the model.

[0007] In order to achieve the above purpose, the disclosed technical solution is as follows:

[0008] In a first aspect, the present application discloses a data processing method for a model partition watermark, the method comprising:

[0009] Obtaining an initial watermark set; wherein the initial watermark set is composed of a preset amount of data extracted from a test set;

[0010] Determine each watermark input data of the initial watermark set according to a preset matrix rule;

[0011] The output data of each watermark of the initial watermark set is processed in a deferred manner, and a processed watermark set is obtained according to the input data of each watermark and the output data of each watermark after the deferred processing;

[0012] According to the partition watermark technology, the processed watermark set is divided into a plurality of target watermark sets; wherein the target watermark set is a watermark set to be embedded in the model;

[0013] Embed multiple target watermark sets into the model to be watermarked to obtain a watermarked model.

[0014] Preferably, the step of determining each watermark input data of the initial watermark set according to a preset matrix rule includes:

[0015] Get the model owner key and input data in the initial watermark set;

[0016] Generate a key matrix matching the model input through the model owner key;

[0017] Determine the number of columns of the key matrix; wherein the number of columns of the key matrix is ​​determined by the number of rows of each input data in the initial watermark set;

[0018] Select a target key matrix with the same number of rows and columns and each target input data in the initial watermark set;

[0019] The target key matrix and each target input data are sequentially subjected to matrix multiplication calculation to obtain each watermark input data of the initial watermark set.

[0020] Preferably, the step of performing a deferred processing on the output data of each watermark of the initial watermark set and obtaining a processed watermark set according to the input data of each watermark and the output data of each watermark after the deferred processing comprises:

[0021] For the output data of each watermark of the initial watermark set, the output data is postponed by one position;

[0022] A processed watermark set is constructed according to the respective watermark input data and the subsequent output data of the respective watermarks.

[0023] Preferably, embedding the plurality of target watermark sets into the model to be watermarked to obtain the watermarked model includes:

[0024] For multiple target watermark sets, determine the preset layers and non-preset layers of the model to be embedded with watermarks; the preset layers are the layers to be embedded; the non-preset layers are all layers other than the preset layers;

[0025] Freeze the non-preset layers, and randomly embed the target watermark set into the preset layers until all the target watermark sets are embedded into the preset layers, thereby obtaining a model with embedded watermarks.

[0026] Preferably, it also includes:

[0027] In the process of confirming the copyright of the watermarked model, the watermark information is extracted from the watermarked model to confirm the copyright of the model.

[0028] The second aspect of the present application discloses a data processing device for a model partition watermark, the device comprising:

[0029] An acquisition unit, configured to acquire an initial watermark set; wherein the initial watermark set is composed of a preset amount of data extracted from a test set;

[0030] A determination unit, used to determine each watermark input data of the initial watermark set according to a preset matrix rule;

[0031] A processing unit, configured to perform a deferred processing on the output data of each watermark of the initial watermark set, and obtain a processed watermark set according to the input data of each watermark and the output data of each watermark after the deferred processing;

[0032] A division unit, used for dividing the processed watermark set into a plurality of target watermark sets according to a partition watermark technology; wherein the target watermark set is a watermark set to be embedded in a model;

[0033] The embedding unit is used to embed multiple target watermark sets into the model to be embedded with watermarks to obtain the model embedded with watermarks.

[0034] Preferably, the determining unit includes:

[0035] A first acquisition module, used to acquire the model owner key and input data in the initial watermark set;

[0036] A generating module, configured to generate a key matrix matching a model input through the model owner key;

[0037] A first determination module is used to determine the number of columns of the key matrix; wherein the number of columns of the key matrix is ​​determined by the number of rows of each input data in the initial watermark set;

[0038] A selection module, used for selecting a target key matrix with equal number of rows and columns and each target input data in the initial watermark set;

[0039] The calculation module is used to perform matrix multiplication calculation on the target key matrix and each target input data in sequence to obtain each watermark input data of the initial watermark set.

[0040] Preferably, the processing unit comprises:

[0041] A second acquisition module is used for delaying the output data of each watermark in the initial watermark set by one position;

[0042] The composition module is used to form a processed watermark set according to the watermark input data and the output data of the watermarks after extension.

[0043] Preferably, the embedding unit comprises:

[0044] The second determination module is used to determine the preset layers and non-preset layers of the model to be embedded with watermarks for multiple target watermark sets; the preset layers are the layers to be embedded; the non-preset layers are all layers other than the preset layers;

[0045] The freezing embedding module is used to freeze the non-preset layers and randomly embed the target watermark set into the preset layers until all the target watermark sets are embedded into the preset layers to obtain a model with embedded watermarks.

[0046] Preferably, it also includes

[0047] The extraction unit is used to extract watermark information from the model embedded with the watermark during the process of confirming the copyright of the model embedded with the watermark, so as to confirm the copyright ownership of the model.

[0048] It can be known from the above technical scheme that the present application discloses a data processing method and device for model partition watermark, which obtains an initial watermark set, wherein the initial watermark set is composed of a preset number of data extracted from a test set, and according to a preset matrix rule, the input data of each watermark of the initial watermark set is determined, the output data of each watermark of the initial watermark set is deferred, and according to each watermark input data and the output data of each watermark after deferred processing, a processed watermark set is obtained, and according to the partition watermark technology, the processed watermark set is divided into multiple target watermark sets, wherein the target watermark set is the watermark set to be embedded in the model, and multiple target watermark sets are embedded into the model to be embedded with watermarks to obtain a model with embedded watermarks.

[0049] Through the above scheme, according to the partition watermarking technology, the processed watermark set is divided into multiple target watermark sets. The partition watermarking technology will use several watermark sets to generate watermarks, and each watermark set can freely choose which layer or layers of the DNN to add watermarks. Since the model always tends to capture common features in the shallow layer, and the watermark belongs to the input-output pair of outliers, the watermark should have less impact on the shallow neurons. The partition watermarking technology reduces the negative impact of the watermark on the accuracy of model fitting. The partition watermarking technology also improves the robustness of the watermark in the face of attacks. When the attacker prunes or fine-tunes the model, if the attacker does not know where the model owner adds the watermark, the partition watermarking technology can better maintain the integrity of the watermark in the model, thereby avoiding the impact of the embedded watermark on the overall accuracy of the DNN. In addition, under the same decrease in the prediction accuracy of the model, the watermark survival rate of the partition watermark is higher, which means that the attacker needs to pay more effort and cost to successfully remove the watermark, so that the model itself is not easy to be copied and stolen, and the security of the data in the model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0051] Figure 1 A schematic diagram of a flow chart of a data processing method for a model partition watermark disclosed in an embodiment of the present application;

[0052] Figure 2 A flowchart of another data processing method for model partition watermark disclosed in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of the structure of a data processing device for a model partition watermark disclosed in an embodiment of the present application;

[0054] Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0057] As can be seen from the background technology, in the prior art, the concept of protecting the intellectual property rights of artificial intelligence models, including deep neural networks, is used by using model watermarks. The basic idea of ​​model watermarking is to apply digital watermarking technology to the field of artificial intelligence. Taking the deep learning model as an example, it embeds a unique watermark or pattern into the weights or structure of the DNN, thereby authenticating and proving the ownership and authenticity of the DNN. However, since the watermarks in the prior art modify the weights or structures, thereby slightly deviating from the original trained model parameters, any slight impact may cause the overall accuracy of the DNN to decrease. Therefore, how to avoid the impact of the embedded watermark on the overall accuracy of the DNN is a problem that needs to be urgently solved in this application.

[0058] In order to solve the above problems, the present application discloses a data processing method and device for model partition watermarking. According to the partition watermarking technology, the processed watermark set is divided into multiple target watermark sets. The partition watermarking technology will use several watermark sets to generate watermarks, and each watermark set can freely choose which layer or layers of the DNN to add watermarks. Since the model is always more inclined to capture common features at the shallow level, and the watermark belongs to the input-output pair of outliers, the watermark should have less impact on the shallow neurons. The negative impact of the watermark on the accuracy of model fitting is reduced by the partition watermarking technology. The partition watermarking technology also improves the robustness of the watermark in the face of attacks. When the attacker prunes or fine-tunes the model, if the attacker does not know the location where the model owner adds the watermark, the partition watermarking technology can better maintain the integrity of the watermark in the model, thereby avoiding the impact of the embedded watermark on the performance of the DNN. In addition, under the same decrease in the prediction accuracy of the model, the watermark survival rate of the partition watermark is higher, which means that the attacker needs to pay more effort and cost to successfully remove the watermark, so that the model itself is not easy to be copied and stolen, and the security of the data in the model is improved. The specific implementation method is described in detail through the following embodiments.

[0059] refer to Figure 1 As shown, a data processing method for a model partition watermark disclosed in an embodiment of the present application is shown, and the data processing method for a model partition watermark mainly includes the following steps:

[0060] S101: Acquire an initial watermark set; wherein the initial watermark set is composed of a preset amount of data extracted from a test set.

[0061] It should be noted that the initial watermark set is a watermark set that has not been processed by the preset matrix rule.

[0062] The preset number can be 3, 5, etc. The preset number is determined according to actual conditions and is not specifically limited in this application.

[0063] The dataset is usually divided into three parts: training set, validation set and test set. The test set is part of the dataset.

[0064] Among them, the test set is composed of many pieces of data, and some data are extracted from them to form a watermark set.

[0065] S102: Determine each watermark input data (W_x) of the initial watermark set according to a preset matrix rule.

[0066] It should be noted that the preset matrix rule is that if the number of columns of the key matrix (KEY_m) and the number of rows of each input data in the initial watermark set are equal, then the target key matrix with the same number of rows and columns and each target input data in the initial watermark set will be selected for multiplication. For example, when the number of columns of matrix A is equal to the number of rows of matrix B, A and B can be multiplied.

[0067] A rule that uses the model owner key (KEY) to generate a matrix of 0 to 1 data that matches the model input.

[0068] The model owner key is a string known only by the model owner.

[0069] Specifically, the process of determining each watermark input data of the initial watermark set according to the preset matrix rule is shown in A1-A5.

[0070] A1: Get the model owner key and input data in the initial watermark set.

[0071] The model owner is obtained based on identity information, and the model owner stores information such as the model owner key in the identity information.

[0072] A2: Generate a key matrix that matches the model input using the model owner key.

[0073] Among them, a matrix consisting of data from 0 to 1 that matches the model input is generated using the model owner key, which is called a key matrix. A matrix consisting of data from 0 to 1 is a matrix consisting of data greater than or equal to 0 and less than or equal to 1.

[0074] It should be noted that the size of the key matrix is ​​controllable, so it must comply with the preset matrix rules.

[0075] To help you understand how to generate a key matrix that matches the model input using the model owner key, here is an example:

[0076] For example, the model owner key is a string. For example, the model owner key is 'key';

[0077] The first step is to confirm the shape of KEY_m. The shape of KEY_m is different for different datasets. For example, for the cifar10 dataset (Cifar10 dataset is a classification dataset composed of color images), the shape of KEY_m is (32, 32); when processing the classic dataset in the field of deep learning (MNIST dataset), its shape is (28, 28);

[0078] The second step is to pad and truncate the model owner key; truncation means: if the key length is 10, only nine data are needed, then only the first nine data of the key are taken; padding means: if the key length is 10, 15 data are needed, and some characters are needed to fill it. This method uses the key itself to fill, that is, repeat the key itself until there is enough data. (Once the shape is determined, you can know how much data is needed);

[0079] The third step is to convert the model owner key into ASCII decimal form and normalize it (i.e. scale it to between 0 and 1).

[0080] The fourth step is to reshape the normalized data into the required shape, such as (3, 3), which is the key matrix that matches the model input.

[0081] A3: Determine the number of columns of the key matrix; wherein the number of columns of the key matrix is ​​determined by the number of rows of each input data in the initial watermark set.

[0082] It should be noted that the number of columns of KEY_m is determined by the number of rows of each input data in the watermark set. For example, if the size of the cifar10 dataset is (3, 32, 32), KEY_m should be a square matrix with the same size as the third element of the dataset, that is, (32, 32). Similarly, if the size of a dataset is (3, 8, 16), the size of KEY_m should be (16, 16), (KEY_m is a square matrix).

[0083] A4: Select a target key matrix with equal number of rows and columns and each target input data in the initial watermark set.

[0084] A5: Perform matrix multiplication calculations on the target key matrix and each target input data in sequence to obtain each watermark input data of the initial watermark set.

[0085] Each watermark input data of the initial watermark set is represented by W_x. Specifically, W_x is a set.

[0086] S103: Performing a deferred processing on the output data of each watermark in the initial watermark set, and obtaining a processed watermark set according to each watermark input data and the output data of each watermark after the deferred processing.

[0087] In S103, for the output data of each watermark in the initial watermark set, the output data of each watermark is postponed by one position, and the processed watermark set is formed by each watermark input data and the output data of each watermark after being postponed by one position. The processed watermark set is represented by D={(W_xi,W_yi)}. Among them, W_xi is the i-th data in W_x; W_yi is the output data of each watermark after the postponement processing.

[0088] In order to facilitate understanding of the process of processing the output data of each watermark in sequence to obtain the processed watermark set, an example is given here to illustrate:

[0089] Take the mnist dataset (handwritten dataset) as an example. The dataset contains pictures of handwritten digits and the corresponding numbers 0-9, that is, {(x,y)}, where x is a handwritten digit picture and y is any number from 0 to 9.

[0090] The watermark set is extracted from the test set, so the watermark output is any number from 0 to 9. For example, if W_yi is 2, the next digit is 3; if W_yi is 9, the next digit is 0.

[0091] Among them, even if W_yi itself cannot be added, postponing one bit will turn W_yi into the next label.

[0092] S104: Divide the processed watermark set into multiple target watermark sets according to the partition watermark technology; wherein the target watermark set is the watermark set to be embedded in the model.

[0093] In order to protect the security of the model itself, the model owner usually adds a watermark to the model. However, conventional watermarks will reduce the prediction accuracy of the model and cannot resist the fine-tuning and pruning by attackers. Therefore, a partitioned watermark technology for deep learning models is proposed.

[0094] The benefit of partitioned watermarking is that it reduces the negative impact of watermarks on model fitting accuracy, because the model always tends to capture common features at the shallow level, and the watermark belongs to the input-output pair of outliers, so the watermark should have less impact on shallow neurons, but usually the watermark is overfitted to the entire model, which will inevitably have a greater impact on shallow neurons; in addition, partitioned watermarking technology also improves the robustness of watermarks in the face of attacks. When attackers prune or fine-tune the model, if they do not know where the model owner added the watermark, the partitioned watermark can better maintain its integrity. With the same decrease in the model's prediction accuracy, the partitioned watermark has a higher watermark survival rate. This means that attackers need to make greater efforts and costs to successfully remove the watermark.

[0095] S105: Embed multiple target watermark sets into the model to be watermarked to obtain a watermarked model.

[0096] In S105, for multiple target watermark sets, the preset layers and non-preset layers of the model to be embedded with watermarks are determined, the non-preset layers are frozen, and the target watermark sets (Di) are randomly embedded in the preset layers until all the target watermark sets are embedded in the preset layers, thereby obtaining a model with embedded watermarks.

[0097] Among them, the preset layers are the layers to be embedded, for example, the preset layers are the layer_j and layer_k layers of the model; the non-preset layers are all layers other than the preset layers.

[0098] The model owner can choose to embed any target watermark set Di into one or several layers of the model by freezing the model weights. For example, if the model owner chooses to embed Di into the layer_j and layer_k layers of the model, the model owner needs to freeze all layers except the layer_j and layer_k layers, and then overfit Di into the model to be embedded with the watermark.

[0099] The watermarks to be embedded are randomly divided into multiple watermark sets, and the model owner decides which layer or layers of the model to embed different watermark sets into.

[0100] In the process of confirming the copyright of the watermarked model, the watermark information is extracted from the watermarked model to confirm the copyright of the model.

[0101] Herein, a watermark extraction operation is performed on the tested model to obtain corresponding watermark information.

[0102] The watermark exists in the form of an input-output pair. The "watermark input" is input into the initial model to obtain the model output. This is the watermark extraction operation. The model output is different from the watermark output.

[0103] The role of model watermark is the same as that of image watermark. When the model owner wants to claim ownership of the model, the model owner can determine the copyright based on the watermark {(x, y)} (note that there are multiple pairs of (x, y)).

[0104] There are many ways to generate model watermarks, but they can be summarized into an overall framework. The model watermark implantation process is:

[0105] (1) The model owner performs information conversion based on the identity information and the input model, converting the identity information into an implantable information form.

[0106] (2) Implant the generated implantable information into the model.

[0107] The process of watermark extraction is as follows:

[0108] (1) Perform watermark extraction on the tested model to obtain corresponding information.

[0109] (2) Compare the embedded information corresponding to the copyright owner to be verified with the extracted information to determine whether it is the model with the watermark embedded.

[0110] If the model owner suspects that a model m is stolen from his own model, the model owner can input the watermark input x into the model m, obtain the model output y_x, and then compare y_x with y. If the same ratio of y_x to y (same number of watermarks / total number of watermarks) reaches a certain threshold (the threshold is set according to the actual situation and is not specifically limited in this application), the model owner's own copyright can be claimed.

[0111] The process of embedding watermarks is different from the process of extracting watermarks. The process of adding watermarks is the process in which the model owner adds watermarks to his own model and declares ownership (refer to adding watermarks to images); the process of extracting watermarks is the process in which the model owner suspects that a suspected model has stolen his model, so he extracts his watermark on the suspected model to see if the extraction is successful. If successful, it means that the suspected model has stolen his model.

[0112] Regarding the watermark implantation process, specifically, in the DNN watermark, the "implantable information form" is "KEY_m"; and "implanting the generated implantable information into the model" is the process of calculating the watermark set D and embedding it into the model.

[0113] Regarding the process of watermark extraction operation (that is, the copyright confirmation process). First, the model owner suspects that a suspected model has stolen his model, and then the model owner extracts the watermark information on the suspected model. If the watermark information can be successfully extracted (that is, the extracted watermark reaches a certain threshold, it can be determined that the suspected model is indeed stolen from his own model. The specific operation is: the watermark is {(x,y)}, x is input into the suspected model, and y1 is obtained. It is necessary to compare y with y1. The results are "same" and "not the same". The watermark extraction operation does not use a key.

[0114] In order to facilitate the understanding of the data processing method of model partition watermark, combined with Figure 2 To explain:

[0115] Figure 2 In the process, a part (preset amount) of data is extracted from the test set as a watermark set; the watermark set is an initial watermark set; the initial watermark set is a watermark set that has not been processed by the preset matrix rule;

[0116] Get the model owner key and generate a key matrix that matches the model input based on the model owner key;

[0117] Perform matrix multiplication on the key matrix and the input data in the watermark set in sequence to obtain the watermark output; specifically, determine the number of columns of the key matrix (KEY_m) and the number of rows of each input data in the watermark set, select a target key matrix with the same number of rows and columns and each target input data in the watermark set, perform matrix multiplication on the target key matrix and each target input data in sequence to obtain each watermark input data in the watermark set;

[0118] For the output data of each watermark in the initial watermark set, the output data is postponed by one bit, and a processed watermark set is formed according to the input data of each watermark and the output data of each watermark after postponement;

[0119] The processed watermark set is divided into a plurality of different target watermark sets according to the partition watermark technology; wherein the target watermark set is the watermark set to be embedded in the model;

[0120] A plurality of different target watermark set partitions are embedded into the model to be watermarked to obtain a watermarked model.

[0121] This application proposes a partitioned watermarking technology. This watermarking technology will use several watermark sets to generate watermarks, and each watermark set can freely choose which layer or layers of the DNN to add the watermark, while the traditional watermarking technology usually adds the watermark to the entire model indiscriminately. The benefit of the partitioned watermarking technology is to reduce the negative impact of the watermark on the model fitting accuracy and the impact of the watermark on the model prediction accuracy, because the model always tends to capture common features in the shallow layer, and the watermark belongs to the input-output pair of the outlier, so the watermark should have less impact on the shallow neurons, but the watermark is usually overfitted to the entire model, which will inevitably have a greater impact on the shallow neurons; in addition, the partitioned watermarking technology also improves the robustness of the watermark in the face of attackers' fine-tuning and pruning. When the attacker prunes or fine-tunes the model, if they do not know where the model owner adds the watermark, the partitioned watermark can better maintain its integrity. Under the same decrease in the prediction accuracy of the model, the watermark survival rate of the partitioned watermark is higher. This means that the attacker needs to make greater efforts and costs to successfully remove the watermark.

[0122] In the embodiment of the present application, according to the partition watermark technology, the processed watermark set is divided into multiple target watermark sets. The partition watermark technology will use several watermark sets to generate watermarks, and each watermark set can freely choose which layer or layers of the DNN to add watermarks. Since the model is always more inclined to capture common features at the shallow level, and the watermark belongs to the input-output pair of outliers, the watermark should have less impact on the shallow neurons. The partition watermark technology reduces the negative impact of the watermark on the accuracy of model fitting. The partition watermark technology also improves the robustness of the watermark in the face of attacks. When the attacker prunes or fine-tunes the model, if the attacker does not know the location where the model owner adds the watermark, the partition watermark technology can better maintain the integrity of the watermark in the model, thereby avoiding the impact of the embedded watermark on the performance of the DNN. In addition, under the same decrease in the prediction accuracy of the model, the watermark survival rate of the partition watermark is higher, which means that the attacker needs to pay more effort and cost to successfully remove the watermark, so that the model itself is not easy to be copied and stolen, and the security of the data in the model is improved.

[0123] Based on the above embodiments Figure 1 A data processing method for a model partition watermark is disclosed, and an embodiment of the present application also discloses a data processing device for a model partition watermark, such as Figure 3 As shown, the data processing device of the model partition watermark includes:

[0124] The acquisition unit 301 is used to acquire an initial watermark set; wherein the initial watermark set is composed of a preset amount of data extracted from the test set;

[0125] A determination unit 302, configured to determine each watermark input data of the initial watermark set according to a preset matrix rule;

[0126] The processing unit 303 is used to perform a deferred processing on the output data of each watermark of the initial watermark set, and obtain a processed watermark set according to each watermark input data and the output data of each watermark after the deferred processing;

[0127] A division unit 304 is used to divide the processed watermark set into a plurality of target watermark sets according to the partition watermark technology; wherein the target watermark set is the watermark set to be embedded in the model;

[0128] The embedding unit 305 is used to embed multiple target watermark sets into the model to be embedded with watermarks to obtain a model with embedded watermarks.

[0129] Further, the determining unit 302 includes:

[0130] A first acquisition module, used to acquire the model owner key and input data in the initial watermark set;

[0131] A generation module, for generating a key matrix matching a model input through a model owner key;

[0132] A first determination module is used to determine the number of columns of the key matrix, wherein the number of columns of the key matrix is ​​determined by the number of rows of each input data in the initial watermark set;

[0133] A selection module, used for selecting a target key matrix with equal number of rows and columns and each target input data in the initial watermark set;

[0134] The calculation module is used to perform matrix multiplication calculation on the target key matrix and each target input data in sequence to obtain each watermark input data of the initial watermark set.

[0135] Furthermore, the processing unit 303 includes:

[0136] A second acquisition module is used to postpone the output data of each watermark in the initial watermark set by one position;

[0137] The composition module is used to form a processed watermark set according to each watermark input data and the output data of each watermark after extension.

[0138] Furthermore, the embedding unit 305 includes:

[0139] The second determination module is used to determine the preset layers and non-preset layers of the model to be embedded with watermarks for multiple target watermark sets; the preset layers are the layers to be embedded; the non-preset layers are all layers other than the preset layers;

[0140] The freezing embedding module is used to freeze the non-preset layers and randomly embed the target watermark set into the preset layers until all the target watermark sets are embedded into the preset layers to obtain a model with embedded watermarks.

[0141] Furthermore, the data processing device of the model partition watermark also includes:

[0142] The extraction unit is used to extract watermark information from the model embedded with the watermark during the copyright confirmation process of the model embedded with the watermark, so as to confirm the copyright ownership of the model.

[0143] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the data processing method of the model partition watermark as described above.

[0144] The present application also provides an electronic device, whose structural diagram is as follows: Figure 4 As shown, it specifically includes a memory 401 and one or more instructions 402, wherein the one or more instructions 402 are stored in the memory 401 and are configured to be executed by one or more processors 403 to execute the one or more instructions 402 to perform the data processing method of the above-mentioned model partition watermark.

[0145] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0146] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0147] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0148] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0149] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0150] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method for model partition watermark, characterized in that: The method comprises: Obtaining an initial watermark set; wherein the initial watermark set is composed of a preset amount of data extracted from a test set; Determine each watermark input data of the initial watermark set according to a preset matrix rule; The output data of each watermark of the initial watermark set is processed in a deferred manner, and a processed watermark set is obtained according to the input data of each watermark and the output data of each watermark after the deferred processing; According to the partition watermark technology, the processed watermark set is divided into a plurality of target watermark sets; wherein the target watermark set is a watermark set to be embedded in the model; Embed multiple target watermark sets into the model to be watermarked to obtain a watermarked model.

2. The method according to claim 1, characterized in that The step of determining each watermark input data of the initial watermark set according to a preset matrix rule comprises: Get the model owner key and input data in the initial watermark set; Generate a key matrix matching the model input through the model owner key; Determine the number of columns of the key matrix; wherein the number of columns of the key matrix is ​​determined by the number of rows of each input data in the initial watermark set; Select a target key matrix with the same number of rows and columns and each target input data in the initial watermark set; The target key matrix and each target input data are sequentially subjected to matrix multiplication calculation to obtain each watermark input data of the initial watermark set.

3. The method according to claim 1, characterized in that The output data of each watermark of the initial watermark set is processed in a deferred manner, and a processed watermark set is obtained according to the input data of each watermark and the output data of each watermark after the deferred processing, including: For the output data of each watermark of the initial watermark set, the output data is postponed by one position; A processed watermark set is constructed according to the respective watermark input data and the subsequent output data of the respective watermarks.

4. The method according to claim 1, characterized in that The method of embedding a plurality of target watermark sets into a model to be embedded with watermarks to obtain a model embedded with watermarks comprises: For multiple target watermark sets, determine the preset layers and non-preset layers of the model to be embedded with watermarks; the preset layers are the layers to be embedded; the non-preset layers are all layers other than the preset layers; Freeze the non-preset layers, and randomly embed the target watermark set into the preset layers until all the target watermark sets are embedded into the preset layers, thereby obtaining a model with embedded watermarks.

5. The method according to claim 1, characterized in that Also includes: In the process of confirming the copyright of the watermarked model, the watermark information is extracted from the watermarked model to confirm the copyright of the model.

6. A data processing device for model partition watermark, characterized in that: The device comprises: An acquisition unit, configured to acquire an initial watermark set; wherein the initial watermark set is composed of a preset amount of data extracted from a test set; A determination unit, used to determine each watermark input data of the initial watermark set according to a preset matrix rule; A processing unit, configured to perform a deferred processing on the output data of each watermark of the initial watermark set, and obtain a processed watermark set according to the input data of each watermark and the output data of each watermark after the deferred processing; A division unit, used for dividing the processed watermark set into a plurality of target watermark sets according to a partition watermark technology; wherein the target watermark set is a watermark set to be embedded in a model; The embedding unit is used to embed multiple target watermark sets into the model to be embedded with watermarks to obtain the model embedded with watermarks.

7. The device according to claim 6, characterized in that The determining unit comprises: A first acquisition module, used to acquire the model owner key and input data in the initial watermark set; A generating module, configured to generate a key matrix matching a model input through the model owner key; A first determination module is used to determine the number of columns of the key matrix; wherein the number of columns of the key matrix is ​​determined by the number of rows of each input data in the initial watermark set; A selection module, used for selecting a target key matrix with equal number of rows and columns and each target input data in the initial watermark set; The calculation module is used to perform matrix multiplication calculation on the target key matrix and each target input data in sequence to obtain each watermark input data of the initial watermark set.

8. The device according to claim 6, characterized in that The processing unit comprises: A second acquisition module is used for delaying the output data of each watermark in the initial watermark set by one position; The composition module is used to form a processed watermark set according to the watermark input data and the output data of the watermarks after extension.

9. The device according to claim 6, characterized in that The embedding unit comprises: The second determination module is used to determine the preset layers and non-preset layers of the model to be embedded with watermarks for multiple target watermark sets; the preset layers are the layers to be embedded; the non-preset layers are all layers other than the preset layers; The freezing embedding module is used to freeze the non-preset layers and randomly embed the target watermark set into the preset layers until all the target watermark sets are embedded into the preset layers to obtain a model with embedded watermarks.

10. The device according to claim 6, characterized in that Also includes The extraction unit is used to extract watermark information from the model embedded with the watermark during the process of confirming the copyright of the model embedded with the watermark, so as to confirm the copyright ownership of the model.

Images