A data processing method and device for model partition watermarking

Abstract

The present application discloses a data processing method and device for model partition watermark, which obtains an initial watermark set, wherein the initial watermark set is composed of a preset number of data extracted from a test set, and determines the input data of each watermark of the initial watermark set according to a preset matrix rule, and performs deferred processing on the output data of each watermark of the initial watermark set, and obtains a processed watermark set according to each watermark input data and the output data of each watermark after deferred processing, and divides the processed watermark set into multiple target watermark sets according to the partition watermark technology, wherein the target watermark set is the watermark set of a model to be embedded, and multiple target watermark sets are embedded into the model to be embedded with watermarks to obtain a model with embedded watermarks.

Description

Technical Field

[0001] The present application relates to the technical field of watermark data processing, and more specifically, to a data processing method and device for model partition watermarking. Background Art

[0002] Artificial intelligence technologies led by deep neural networks (DNNs) have developed rapidly, and their extensive applications in multiple fields such as healthcare, finance, and autonomous driving have also demonstrated great potential. Subsequently, the security issues of deep neural network models have increasingly attracted attention. The training of deep neural network models relies on a large amount of precious data support, and this data may involve sensitive fields. At the same time, developing and training these models requires a large amount of time and effort, but the models themselves are extremely easy to copy and pirate. How to ensure the security of this data and how to protect the rights and interests of data owners have become issues that must be considered.

[0003] In the prior art, there is a concept of using model watermarks to protect the intellectual property rights of artificial intelligence models including deep neural networks. The basic idea of model watermarks is to apply digital watermarking technology in the field of artificial intelligence. Taking a deep learning model as an example, it embeds unique watermarks or patterns into the weights or structures of the DNN to authenticate and prove the ownership and authenticity of the DNN.

[0004] However, since the watermarks in the prior art modify the weights or structures, slightly deviating from the original trained model parameters, any subtle influence may lead to a decrease in the overall accuracy of the DNN.

[0005] Therefore, how to avoid the influence of embedded watermarks on the overall accuracy of the DNN is an urgent problem to be solved in the present application. Summary of the Invention

[0006] In view of this, the present application discloses a data processing method and device for model partition watermarking, aiming to avoid the influence of embedded watermarks on the overall accuracy of the DNN and improve the security of the data in the model.

[0007] To achieve the above object, the disclosed technical solutions are as follows:

[0008] The first aspect of the present application discloses a data processing method for model partition watermarking, and the method includes:

[0009] Obtain an initial watermark set; wherein, the initial watermark set is composed of extracting a preset number of data from a test set;

[0010] Determine each watermark input data of the initial watermark set according to a preset matrix rule;

[0011] Perform a sequential processing on the output data of each watermark in the initial watermark set, and obtain a processed watermark set according to the input data of each watermark and the output data of each watermark after sequential processing;

[0012] According to the partition watermark technology, divide the processed watermark set into multiple target watermark sets; wherein, the target watermark set is the watermark set to be embedded in the model;

[0013] Embed multiple target watermark sets into the model to be embedded with watermarks, and obtain a model with embedded watermarks.

[0014] Preferably, the determining of the input data of each watermark in the initial watermark set according to the preset matrix rule includes:

[0015] Obtain the model owner key and the input data in the initial watermark set;

[0016] Generate a key matrix that matches the model input through the model owner key;

[0017] Determine the number of columns of the key matrix; wherein, the number of columns of the key matrix is determined by the number of rows of the input data in the initial watermark set;

[0018] Select a target key matrix with the same number of rows and columns and each target input data in the initial watermark set;

[0019] Perform matrix multiplication calculations on the target key matrix and each target input data in sequence to obtain the input data of each watermark in the initial watermark set.

[0020] Preferably, the performing a sequential processing on the output data of each watermark in the initial watermark set, and obtaining a processed watermark set according to the input data of each watermark and the output data of each watermark after sequential processing includes:

[0021] For the output data of each watermark in the initial watermark set, shift the output data by one position;

[0022] According to the input data of each watermark and the output data of each watermark after shifting, form a processed watermark set.

[0023] Preferably, the embedding multiple target watermark sets into the model to be embedded with watermarks and obtaining a model with embedded watermarks includes:

[0024] For multiple target watermark sets, determine the preset layer and non-preset layer of the model to be embedded with watermarks; the preset layer is the layer to be embedded; the non-preset layer is all layers other than the preset layer;

[0025] Freeze the non-preset layers, and randomly embed the target watermark sets into the preset layers until all the target watermark sets are embedded into the preset layers, obtaining a model with embedded watermarks.

[0026] Preferably, it further includes:

[0027] During the process of copyright confirmation of the model with embedded watermarks, extract the watermark information from the model with embedded watermarks to confirm the copyright ownership of the model.

[0028] The second aspect of this application discloses a data processing device for model partition watermarks. The device includes:

[0029] An acquisition unit for acquiring an initial watermark set; wherein, the initial watermark set is composed of preset amounts of data extracted from a test set;

[0030] A determination unit for determining each watermark input data of the initial watermark set according to a preset matrix rule;

[0031] A processing unit for performing a sequential processing on the output data of each watermark in the initial watermark set, and obtaining a processed watermark set according to each watermark input data and the sequentially processed output data of each watermark;

[0032] A partitioning unit for partitioning the processed watermark set into multiple target watermark sets according to the partition watermark technology; wherein, the target watermark set is the watermark set to be embedded in the model;

[0033] An embedding unit for embedding multiple target watermark sets into the model to be embedded with watermarks, obtaining a model with embedded watermarks.

[0034] Preferably, the determination unit includes:

[0035] A first acquisition module for acquiring the model owner key and the input data in the initial watermark set;

[0036] A generation module for generating a key matrix matching the model input through the model owner key;

[0037] A first determination module for determining the number of columns of the key matrix; wherein, the number of columns of the key matrix is determined by the number of rows of each input data in the initial watermark set;

[0038] A selection module for selecting a target key matrix with the same number of rows and columns and each target input data in the initial watermark set;

[0039] A calculation module for sequentially performing matrix multiplication calculations on the target key matrix and each target input data to obtain each watermark input data of the initial watermark set.

[0040] Preferably, the processing unit includes:

[0041] A second obtaining module, configured to shift the output data of each watermark in the initial watermark set by one position.

[0042] A forming module, configured to form a processed watermark set according to the input data of each watermark and the output data of each watermark after shifting.

[0043] Preferably, the embedding unit includes:

[0044] A second determining module, configured to determine a preset layer and a non-preset layer of a model for embedding a watermark for multiple target watermark sets; the preset layer is the layer to be embedded; the non-preset layer is all layers other than the preset layer.

[0045] A freezing embedding module, configured to freeze the non-preset layer and randomly embed the target watermark sets into the preset layer until all the target watermark sets are embedded into the preset layer, obtaining a model with the watermark embedded.

[0046] Preferably, it further includes

[0047] An extraction unit, configured to extract watermark information from the model with the watermark embedded during the process of copyright confirmation of the model with the watermark embedded, so as to confirm the copyright ownership of the model.

[0048] As can be seen from the above technical solutions, the present application discloses a data processing method and apparatus for model partition watermarking. An initial watermark set is obtained, where the initial watermark set is composed of preset amounts of data extracted from a test set. According to a preset matrix rule, the input data of each watermark in the initial watermark set is determined, the output data of each watermark in the initial watermark set is shifted, and according to the input data of each watermark and the output data of each watermark after shifting, a processed watermark set is obtained. According to the partition watermarking technology, the processed watermark set is divided into multiple target watermark sets, where the target watermark set is the watermark set to be embedded in the model. The multiple target watermark sets are embedded into the model for embedding the watermark, obtaining a model with the watermark embedded.

[0049] Through the above solution, according to the partition watermarking technology, the processed watermark set is divided into multiple target watermark sets. This partition watermarking technology generates watermarks using several watermark sets, and each watermark set can freely choose which layer or layers of the DNN to add watermarks. Since the model always tends to capture general features more in the shallow layers, and watermarks belong to the input-output pairs of outliers, the influence of watermarks on shallow neurons should be minimized. By using the partition watermarking technology, the negative impact of watermarks on the model fitting accuracy is reduced, and the robustness of watermarks against attacks is also improved. When an attacker prunes or fine-tunes the model, if the attacker does not know the location where the model owner adds watermarks, the partition watermarking technology can better maintain the integrity of the watermarks in the model, thus avoiding the impact of embedded watermarks on the overall accuracy of the DNN. In addition, when the prediction accuracy of the model drops by the same amount, the survival rate of watermarks in the partition watermarking is higher, which means that the attacker needs to make greater efforts and pay a higher cost to successfully remove the watermarks, making the model itself not easily copied and misappropriated, and improving the security of the data in the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0051] Figure 1 It is a schematic flowchart of a data processing method for model partition watermarking disclosed in an embodiment of the present application;

[0052] Figure 2 It is a schematic flowchart of another data processing method for model partition watermarking disclosed in an embodiment of the present application;

[0053] Figure 3 It is a schematic structural diagram of a data processing device for model partition watermarking disclosed in an embodiment of the present application;

[0054] Figure 4 It is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of them. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0056] In this application, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0057] As can be seen from the background art, in the prior art, there is a concept of using model watermarking to protect the intellectual property rights of artificial intelligence models including deep neural networks. The basic idea of model watermarking is to apply digital watermarking technology in the field of artificial intelligence. Taking a deep learning model as an example, it embeds unique watermarks or patterns into the weights or structure of the DNN to authenticate and prove the ownership and authenticity of the DNN. However, since the watermark in the prior art modifies the weights or structure, thus slightly deviating from the original trained model parameters, any slight influence may cause a decrease in the overall accuracy of the DNN. Therefore, how to avoid the influence of embedding the watermark on the overall accuracy of the DNN is an urgent problem to be solved in this application.

[0058] To solve the above problems, this application discloses a data processing method and device for model partition watermarking. According to the partition watermarking technology, the processed watermark set is divided into multiple target watermark sets. This partition watermarking technology generates watermarks using several watermark sets, and each watermark set can freely choose which layer or layers of the DNN to add watermarks. Since the model always tends to capture general features more in the shallow layer, and the watermark belongs to the input-output pairs of outliers, the influence of the watermark on the shallow neurons should be minimized. The partition watermarking technology reduces the negative impact of the watermark on the model fitting accuracy and also improves the robustness of the watermark against attacks. When an attacker prunes or fine-tunes the model, if the attacker does not know the location where the model owner adds the watermark, the partition watermarking technology can better maintain the integrity of the watermark in the model, thus avoiding the influence of embedding the watermark on the performance of the DNN. In addition, under the same decrease in the prediction accuracy of the model, the watermark survival rate of the partition watermark is higher, which means that the attacker needs to make greater efforts and pay a higher cost to successfully remove the watermark, thereby making the model itself not easily copied and misappropriated, and improving the security of the data in the model. The specific implementation manner will be specifically described in the following embodiments.

[0059] Reference Figure 1 As shown, a data processing method for model partition watermarking disclosed in an embodiment of this application mainly includes the following steps:

[0060] S101: Obtain an initial watermark set; where the initial watermark set is composed of a preset number of data extracted from the test set.

[0061] It should be noted that the initial watermark set is a watermark set that has not been processed by the preset matrix rule.

[0062] The preset number can be 3, 5, etc. The determination of the preset number is set according to the actual situation, and this application does not make specific limitations.

[0063] Generally, the data set is divided into three parts: a training set, a validation set, and a test set. The test set is a part of the data set.

[0064] Among them, the test set is composed of many pieces of data, and some data are extracted from it to form a watermark set.

[0065] S102: Determine each watermark input data (W_x) of the initial watermark set according to the preset matrix rule.

[0066] It should be noted that the preset matrix rule is that the number of columns of the key matrix (KEY_m) is equal to the number of rows of each input data in the initial watermark set, then the rule of multiplying the target key matrix with equal number of rows and columns and each target input data in the initial watermark set is selected. For example, when the number of columns of matrix A is equal to the number of rows of matrix B, A and B can be multiplied.

[0067] The rule of generating a matrix composed of data from 0 to 1 that matches the model input by using the model owner's key (KEY).

[0068] The model owner's key is a string known only to the model owner.

[0069] Specifically, the process of determining each watermark input data of the initial watermark set according to the preset matrix rule is shown in A1 - A5.

[0070] A1: Obtain the model owner's key and the input data in the initial watermark set.

[0071] Among them, obtain the model owner according to the identity information, and the model owner stores information such as the model owner's key in the identity information.

[0072] A2: Generate a key matrix that matches the model input through the model owner's key.

[0073] Among them, using the model owner's key to generate a matrix composed of data from 0 to 1 that matches the model input is called the key matrix. The matrix composed of data from 0 to 1 is a matrix composed of data greater than or equal to 0 and less than or equal to 1.

[0074] It should be noted that the size of the key matrix is controllable and thus must conform to the preset matrix rules.

[0075] To facilitate the understanding of generating a key matrix that matches the model input from the model owner's key, an example is given here for illustration:

[0076] For example, the model owner's key is a string of characters. For example, the model owner's key is 'key';

[0077] First step, confirm the shape of KEY_m. For different datasets, the shape of KEY_m is different. For example, for the cifar10 dataset (the Cifar10 dataset is a classification dataset consisting of color images), the shape of KEY_m is (32, 32); when dealing with the classic dataset (mnist dataset) in the field of deep learning, its shape is (28, 28);

[0078] Second step, the model owner's key needs to be padded and truncated; truncation means: if the key length is 10 and only nine data are required, only the first nine data of the key are taken; padding means: if the key length is 10 and 15 data are required, some characters need to be used for padding. In this method, the key itself is used for padding, that is, the key itself is repeated until there are enough data. (After determining the shape, it is possible to know how much data is needed);

[0079] Third step, convert the model owner's key into the decimal form of ascii code and normalize it (that is, scale it to between 0 and 1);

[0080] Fourth step, reshape the normalized data into the required shape, such as (3, 3), which is the key matrix that matches the model input.

[0081] A3: Determine the number of columns of the key matrix; among them, the number of columns of the key matrix is determined by the number of rows of each input data in the initial watermark set.

[0082] It should be noted that the number of columns of KEY_m is determined by the number of rows of each input data in the watermark set. For example, for the cifar10 dataset with a size of (3, 32, 32), KEY_m should be a square matrix with the same size as the third element of the dataset, that is, (32, 32). Similarly, if a dataset has a size of (3, 8, 16), the size of KEY_m should be (16, 16) (KEY_m is a square matrix).

[0083] A4: Select a target key matrix with the same number of rows and columns and each target input data in the initial watermark set.

[0084] A5: Perform matrix multiplication calculations on the target key matrix and each target input data in turn to obtain each watermark input data of the initial watermark set.

[0085] Each watermark input data of the initial watermark set is represented by W_x. Specifically, W_x is a set.

[0086] S103: Performing a deferred processing on the output data of each watermark in the initial watermark set, and obtaining a processed watermark set according to each watermark input data and the output data of each watermark after the deferred processing.

[0087] In S103, for the output data of each watermark in the initial watermark set, the output data of each watermark is postponed by one position, and the processed watermark set is formed by each watermark input data and the output data of each watermark after being postponed by one position. The processed watermark set is represented by D={(W_xi,W_yi)}. Among them, W_xi is the i-th data in W_x; W_yi is the output data of each watermark after the postponement processing.

[0088] In order to facilitate understanding of the process of processing the output data of each watermark in sequence to obtain the processed watermark set, an example is given here to illustrate:

[0089] Take the mnist dataset (handwritten dataset) as an example. The dataset contains pictures of handwritten digits and the corresponding numbers 0-9, that is, {(x,y)}, where x is a handwritten digit picture and y is any number from 0 to 9.

[0090] The watermark set is extracted from the test set, so the watermark output is any number from 0 to 9. For example, if W_yi is 2, the next digit is 3; if W_yi is 9, the next digit is 0.

[0091] Among them, even if W_yi itself cannot be added, postponing one bit will turn W_yi into the next label.

[0092] S104: Divide the processed watermark set into multiple target watermark sets according to the partition watermark technology; wherein the target watermark set is the watermark set to be embedded in the model.

[0093] In order to protect the security of the model itself, the model owner usually adds a watermark to the model. However, conventional watermarks will reduce the prediction accuracy of the model and cannot resist the fine-tuning and pruning by attackers. Therefore, a partitioned watermark technology for deep learning models is proposed.

[0094] The advantage of the partitioned watermarking technique is to reduce the negative impact of the watermark on the model fitting accuracy. Since the model always tends to capture general features more in the shallow layers, and the watermark belongs to the input-output pairs of outliers, the watermark should have less impact on the shallow neurons. However, usually the watermark is overfitted into the whole model, which inevitably has a greater impact on the shallow neurons. In addition, the partitioned watermarking technique also improves the robustness of the watermark against attacks. When the attacker prunes or fine-tunes the model, if they don't know the location where the model owner adds the watermark, the partitioned watermark can better maintain its integrity. Under the same decrease in the prediction accuracy of the model, the survival rate of the watermark of the partitioned watermark is higher. This means that the attacker needs to put in more effort and cost to successfully remove the watermark.

[0095] S105: Embed multiple target watermark sets into the model to be watermarked to obtain the watermarked model.

[0096] In S105, for multiple target watermark sets, determine the preset layers and non-preset layers of the model to be watermarked, freeze the non-preset layers, and randomly embed the target watermark set (Di) into the preset layers until all the target watermark sets are embedded into the preset layers to obtain the watermarked model.

[0097] Among them, the preset layer is the layer to be embedded, for example, the preset layer is the layer_j and layer_k of the model; the non-preset layer is all the layers other than the preset layer.

[0098] The model owner can, through the operation of freezing the weights of the model, choose to embed any target watermark set Di into a certain layer or several layers of the model. For example, if the model owner chooses to embed Di into the layer_j and layer_k of the model, the model owner needs to freeze all the layers except the layer_j and layer_k, and then overfit Di into the model to be watermarked.

[0099] Randomly divide the watermark to be embedded into multiple watermark sets, and let the model owner decide which layer or layers of the model to embed different watermark sets into.

[0100] In the process of copyright confirmation of the watermarked model, extract the watermark information from the watermarked model to confirm the copyright ownership of the model.

[0101] Among them, perform the watermark extraction operation on the tested model to obtain the corresponding watermark information.

[0102] The watermark exists in the form of input-output pairs. Input the "watermark input" into the initial model to obtain the model output, which is the watermark extraction operation. The model output is different from the watermark output.

[0103] The function of model watermark is the same as that of image watermark. When the model owner wants to claim ownership of the model, the model owner can determine the copyright according to the watermark {(x, y)} (note that there are multiple pairs of (x, y)).

[0104] There are many methods for generating model watermarks, but generally there is an overall framework. The process of implanting model watermarks is as follows:

[0105] (1) The model owner performs information conversion based on the identity information and the input model, and converts the identity information into an implantable information form.

[0106] (2) Implant the generated implantable information into the model.

[0107] The process of watermark extraction is as follows:

[0108] (1) Perform watermark extraction operations on the tested model to obtain corresponding information.

[0109] (2) Compare the implantable information corresponding to the copyright owner to be verified with the extracted information to determine whether it is the model implanted with this watermark.

[0110] If the model owner suspects that a certain model m is a stolen model of his own, the model owner can input the watermark x into the model m to obtain the model output y_x, and then compare y_x with y. If the same ratio (the number of the same watermarks / the total number of watermarks) of y_x and y reaches a certain threshold (this threshold is set according to the actual situation and is not specifically limited in this application), the model owner can claim his own copyright.

[0111] The process of implanting watermarks is different from the process of watermark extraction. The process of adding watermarks is the process in which the model owner adds watermarks to his own model to claim ownership (refer to adding image watermarks to images); the watermark extraction process is that the model owner suspects that a certain suspected model has stolen his own model, so he extracts his own watermark on the suspected model to see if it can be successfully extracted. If successful, it means that the suspected model has stolen his own model.

[0112] For the process of implanting watermarks. Specifically, in the DNN watermark, the "implantable information form" is "KEY_m"; the process of "implanting the generated implantable information into the model" is the process of calculating the watermark set D and embedding it into the model.

[0113] For the process of watermark extraction operation (which is the copyright confirmation process). First, the model owner suspects that a certain suspected model has stolen their model. Then, the model owner extracts watermark information from the suspected model. If the watermark information can be successfully extracted (that is, when the extracted watermark reaches a certain threshold), it can be determined that the suspected model has indeed stolen their model. The specific operation is as follows: The watermark is {(x, y)}. Input x into the suspected model to obtain y1. It is necessary to compare y with y1. The results are "the same" and "not the same". The watermark extraction operation does not use a key.

[0114] To facilitate understanding of the process of the data processing method for model partition watermarks, in combination with Figure 2 it is described as follows:

[0115] Figure 2 In, extract part (a preset quantity) of the data from the test set as the watermark set; this watermark set is the initial watermark set; the initial watermark set is the watermark set that has not been processed by the preset matrix rule;

[0116] Obtain the model owner's key and generate a key matrix that matches the model input according to the model owner's key;

[0117] Perform matrix multiplication on the key matrix and the input data in the watermark set in sequence to obtain the watermark output; specifically, determine the number of columns of the key matrix (KEY_m) and the number of rows of each input data in the watermark set, select the target key matrix with equal number of rows and columns and each target input data in the watermark set, and perform matrix multiplication calculations on the target key matrix and each target input data in sequence to obtain each watermark input data of the watermark set;

[0118] For the output data of each watermark in the initial watermark set, shift the output data by one position. According to each watermark input data and the output data of each watermark after shifting, form the processed watermark set;

[0119] According to the partition watermark technology, divide the processed watermark set into multiple different target watermark sets; among them, the target watermark set is the watermark set to be embedded in the model;

[0120] Partition and embed multiple different target watermark sets into the model to be embedded with watermarks to obtain the model with embedded watermarks.

[0121] This application proposes a partitioned watermarking technique. This watermarking technique generates watermarks using several watermark sets, and each watermark set can freely choose which layer or layers of the DNN to add watermarks to, while traditional watermarking techniques usually add watermarks to the entire model without discrimination. The advantage of the partitioned watermarking technique is to reduce the negative impact of watermarks on the model fitting accuracy and the impact of watermarks on the model prediction accuracy. Since the model always tends to capture general features in the shallow layers, and watermarks belong to outlier input-output pairs, watermarks should have less impact on shallow neurons. However, usually watermarks are overfitted to the entire model, which inevitably has a greater impact on shallow neurons. In addition, the partitioned watermarking technique also improves the robustness of watermarks against attacker fine-tuning and pruning. When attackers prune or fine-tune the model, if they do not know the location where the model owner adds watermarks, the partitioned watermark can better maintain its integrity. When the prediction accuracy of the model drops by the same amount, the watermark survival rate of the partitioned watermark is higher. This means that attackers need to put in more effort and cost to successfully remove the watermark.

[0122] In an embodiment of this application, according to the partitioned watermarking technique, the processed watermark set is divided into multiple target watermark sets. This partitioned watermarking technique generates watermarks using several watermark sets, and each watermark set can freely choose which layer or layers of the DNN to add watermarks to. Since the model always tends to capture general features in the shallow layers, and watermarks belong to outlier input-output pairs, watermarks should have less impact on shallow neurons. By using the partitioned watermarking technique, the negative impact of watermarks on the model fitting accuracy is reduced, and the partitioned watermarking technique also improves the robustness of watermarks against attacks. When attackers prune or fine-tune the model, if the attackers do not know the location where the model owner adds watermarks, the partitioned watermarking technique can better maintain the integrity of the watermarks in the model, thus avoiding the impact of embedded watermarks on the performance of the DNN. In addition, when the prediction accuracy of the model drops by the same amount, the watermark survival rate of the partitioned watermark is higher, which means that attackers need to put in more effort and cost to successfully remove the watermark, thereby making the model itself not easily copied and misappropriated and improving the security of the data in the model.

[0123] Based on the above embodiments Figure 1 A data processing method for model partitioned watermarks disclosed, and an embodiment of this application also correspondingly discloses a data processing device for model partitioned watermarks, as Figure 3 shown. The data processing device for model partitioned watermarks includes:

[0124] An acquisition unit 301, configured to acquire an initial watermark set; wherein, the initial watermark set is composed of preset amounts of data extracted from a test set;

[0125] A determination unit 302, configured to determine each watermark input data of the initial watermark set according to a preset matrix rule;

[0126] A processing unit 303, configured to perform a carry - over process on the output data of each watermark in the initial watermark set, and obtain a processed watermark set according to each watermark input data and the output data of each watermark after the carry - over process;

[0127] A partitioning unit 304, configured to partition the processed watermark set into multiple target watermark sets according to the partition watermark technology; wherein, the target watermark set is the watermark set to be embedded in the model;

[0128] An embedding unit 305, configured to embed multiple target watermark sets into the model to be embedded with watermarks, and obtain a model with embedded watermarks.

[0129] Further, the determination unit 302 includes:

[0130] A first acquisition module, configured to acquire the model owner key and the input data in the initial watermark set;

[0131] A generation module, configured to generate a key matrix matching the model input through the model owner key;

[0132] A first determination module, configured to determine the number of columns of the key matrix, wherein the number of columns of the key matrix is determined by the number of rows of each input data in the initial watermark set;

[0133] A selection module, configured to select a target key matrix with the same number of rows and columns and each target input data in the initial watermark set;

[0134] A calculation module, configured to perform matrix multiplication calculations on the target key matrix and each target input data in sequence, and obtain each watermark input data of the initial watermark set.

[0135] Further, the processing unit 303 includes:

[0136] A second acquisition module, configured to carry over the output data of each watermark in the initial watermark set by one position;

[0137] A composition module, configured to compose a processed watermark set according to each watermark input data and the output data of each watermark after the carry - over process.

[0138] Further, the embedding unit 305 includes:

[0139] A second determination module, configured to determine the preset layer and the non - preset layer of the model to be embedded with watermarks for multiple target watermark sets; the preset layer is the layer to be embedded; the non - preset layer is all layers other than the preset layer;

[0140] A freezing embedding module, which is used to freeze non-preset layers and randomly embed the target watermark sets into the preset layers until all the target watermark sets are embedded into the preset layers, obtaining a model with embedded watermarks.

[0141] Furthermore, the data processing device for model partition watermarking further includes:

[0142] An extraction unit, which is used to extract watermark information from the model with embedded watermarks during the process of copyright confirmation of the model with embedded watermarks, so as to confirm the copyright attribution of the model.

[0143] The embodiment of the present application also provides a storage medium, which includes stored instructions. When the instructions run, they control the device where the storage medium is located to execute the data processing method for model partition watermarking as described above.

[0144] The embodiment of the present application also provides an electronic device, and its structural schematic diagram is as Figure 4 shown, specifically including a memory 401 and one or more instructions 402. One or more instructions 402 are stored in the memory 401 and are configured to be executed by one or more processors 403 to execute the above-mentioned data processing method for model partition watermarking.

[0145] For the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0146] It should be noted that the embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0147] The steps in the methods of the embodiments of the present application can be adjusted, combined, and deleted according to actual needs.

[0148] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0149] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0150] The foregoing are only the preferred embodiments of the present application, and it should be noted that for those of ordinary skill in the art, several improvements and refinements can be made without departing from the principle of the present application, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A data processing method for model partition watermark, characterized in that: The method comprises: Obtaining an initial watermark set; wherein the initial watermark set is composed of a preset amount of data extracted from a test set; Determine each watermark input data of the initial watermark set according to a preset matrix rule; The output data of each watermark of the initial watermark set is processed in a deferred manner, and a processed watermark set is obtained according to the input data of each watermark and the output data of each watermark after the deferred processing; According to the partition watermark technology, the processed watermark set is divided into a plurality of target watermark sets; wherein the target watermark set is a watermark set to be embedded in the model; For multiple target watermark sets, determine the preset layers and non-preset layers of the model to be embedded with watermarks; the preset layers are the layers to be embedded; the non-preset layers are all layers other than the preset layers; Freeze the non-preset layers, and randomly embed the target watermark set into the preset layers until all the target watermark sets are embedded into the preset layers, thereby obtaining a model with embedded watermarks.

2. The method according to claim 1, characterized in that The step of determining each watermark input data of the initial watermark set according to a preset matrix rule comprises: Get the model owner key and input data in the initial watermark set; Generate a key matrix matching the model input through the model owner key; Determine the number of columns of the key matrix; wherein the number of columns of the key matrix is ​​determined by the number of rows of each input data in the initial watermark set; Select a target key matrix with the same number of rows and columns and each target input data in the initial watermark set; The target key matrix and each target input data are sequentially subjected to matrix multiplication calculation to obtain each watermark input data of the initial watermark set.

3. The method according to claim 1, characterized in that The output data of each watermark of the initial watermark set is processed in a deferred manner, and a processed watermark set is obtained according to the input data of each watermark and the output data of each watermark after the deferred processing, including: For the output data of each watermark of the initial watermark set, the output data is postponed by one position; A processed watermark set is constructed according to the respective watermark input data and the subsequent output data of the respective watermarks.

4. The method according to claim 1, characterized in that: Also includes: In the process of confirming the copyright of the watermarked model, the watermark information is extracted from the watermarked model to confirm the copyright of the model.

5. A data processing device for model partition watermark, characterized in that: The device comprises: An acquisition unit, configured to acquire an initial watermark set; wherein the initial watermark set is composed of a preset amount of data extracted from a test set; A determination unit, used to determine each watermark input data of the initial watermark set according to a preset matrix rule; A processing unit, configured to perform a deferred processing on the output data of each watermark of the initial watermark set, and obtain a processed watermark set according to the input data of each watermark and the output data of each watermark after the deferred processing; A division unit, used for dividing the processed watermark set into a plurality of target watermark sets according to a partition watermark technology; wherein the target watermark set is a watermark set to be embedded in a model; An embedding unit, used for embedding a plurality of target watermark sets into a model to be embedded with watermarks, so as to obtain a model embedded with watermarks; The embedding unit comprises: The second determination module is used to determine the preset layers and non-preset layers of the model to be embedded with watermarks for multiple target watermark sets; the preset layers are the layers to be embedded; the non-preset layers are all layers other than the preset layers; The freezing embedding module is used to freeze the non-preset layers and randomly embed the target watermark set into the preset layers until all the target watermark sets are embedded into the preset layers to obtain a model with embedded watermarks.

6. The device according to claim 5, characterized in that The determining unit comprises: A first acquisition module, used to acquire the model owner key and input data in the initial watermark set; A generating module, configured to generate a key matrix matching a model input through the model owner key; A first determination module is used to determine the number of columns of the key matrix; wherein the number of columns of the key matrix is ​​determined by the number of rows of each input data in the initial watermark set; A selection module, used for selecting a target key matrix with equal number of rows and columns and each target input data in the initial watermark set; The calculation module is used to perform matrix multiplication calculation on the target key matrix and each target input data in sequence to obtain each watermark input data of the initial watermark set.

7. The device according to claim 5, characterized in that The processing unit comprises: A second acquisition module is used for delaying the output data of each watermark in the initial watermark set by one position; The composition module is used to form a processed watermark set according to the watermark input data and the output data of the watermarks after extension.

8. The device according to claim 5, characterized in that Also includes: The extraction unit is used to extract watermark information from the model embedded with the watermark during the process of confirming the copyright of the model embedded with the watermark, so as to confirm the copyright ownership of the model.

Images