Data processing methods, devices and methods for identifying infringement of neural network models

Abstract

The present application discloses a data processing method, including: obtaining target watermark information; selecting a target convolution kernel from a first neural network model, obtaining a first operation result of an element in a target convolution kernel matrix; mapping the target watermark information to the target convolution kernel according to a mapping relationship between the watermark information and the first operation result of an element in the convolution kernel matrix, and obtaining a target neural network model. This method solves the problem of large storage overhead caused by the storage matrix in the watermark embedding algorithm of the neural network model under the prior art.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and specifically to two data processing methods, two data processing devices, two electronic devices and two storage devices; the present application also relates to a method for identifying infringement of a neural network model. Background Art

[0002] Neural network models can be applied to many fields such as face recognition, speech recognition, natural language processing (NLP), etc. It takes a lot of effort to train a complex and high-performance neural network model. Generating a neural network model generally requires multiple stages, including: network structure design stage, preliminary data set preparation stage, network performance optimization stage, and training the neural network through a large amount of time and computing resources. After going through multiple stages, a complete set of weight parameters of the neural network model is finally generated, so that the neural network model can solve some problems well.

[0003] However, if a third party obtains the weight parameter file, they can easily rebuild a network with the same performance. Therefore, for the owner of the neural network model, copyright protection of the model is more valuable. Embedding watermark information for neural network models is one of the most effective ways to protect intellectual property rights.

[0004] Under the existing technology, the watermark embedding algorithm of the neural network model adopts a random matrix embedding method. This method has the following problems: since the size of the matrix to be saved is the product of the number of binary bits of the embedded watermark information and the number of weight parameters of the embedding layer of the neural network model, if multiple networks need to embed watermarks, or multiple layers of the network need to embed watermarks, the storage matrix requires a large storage space. Summary of the invention

[0005] The present application provides a data processing method, device, electronic device and storage device to solve the problem of large storage overhead caused by the storage matrix in the watermark embedding algorithm of the neural network model in the prior art.

[0006] The present application provides a data processing method, comprising:

[0007] Obtain target watermark information;

[0008] Selecting a target convolution kernel from the first neural network model, and obtaining a first operation result of an element in a target convolution kernel matrix;

[0009] According to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model.

[0010] Optionally, mapping the target watermark information to the target convolution kernel according to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix to obtain the target neural network model includes:

[0011] The elements in the target convolution kernel matrix are adjusted until the first operation result between the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

[0012] Optionally, obtaining a first operation result of an element in a target convolution kernel matrix includes:

[0013] Obtaining a first operation result of a weight parameter in a target convolution kernel matrix;

[0014] The adjusting the elements in the target convolution kernel matrix until the first operation result between the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship to obtain the target neural network model includes:

[0015] The weight parameters in the target convolution kernel matrix are adjusted until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

[0016] Optionally, the adjusting the weight parameters in the target convolution kernel matrix until a first operation result between the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship to obtain a target neural network model includes:

[0017] According to the sub-input set and the sub-label set, the weight parameters in the convolution kernel matrix are adjusted until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, and the target neural network model is obtained, wherein the sub-input set is a parameter set for calculating the first operation result of the weight parameters in the convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the weight parameters in the convolution kernel matrix.

[0018] Optionally, adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until a first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship includes:

[0019] The weight parameters of the convolution kernel are adjusted according to the sub-input set and the sub-label set and the watermark loss function embedded in the first neural network model until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship and the watermark loss function converges.

[0020] Optionally, also include:

[0021] Obtaining reference weight information of a convolution kernel matrix of a reference neural network model; the reference neural network model and the target neural network model have the same structure; the convolution kernel of the reference neural network model corresponds to the target convolution kernel selected from the first neural network model;

[0022] Obtaining target weight information of the convolution kernel matrix of the target neural network model in which part of the watermark information has been embedded;

[0023] The reference weight information and the target weight information are used as training sets to train a detection neural network model, and according to the distinguishability data of the reference weight information and the target weight information output by the detection neural network model, the training method of the target neural network model is adjusted to obtain a target neural network model that meets the watermark information hiding conditions.

[0024] Optionally, adjusting the model training method of the target neural network according to the distinguishability information between the reference weight information and the target weight information output by the detection neural network model includes:

[0025] Obtaining a regularization term for hiding watermark information according to the distinguishability information;

[0026] Adding the regularization term for hiding the watermark information based on the model loss function corresponding to the target neural network model when the conventional training set is used to train the target neural network model and the regularization term for embedding the watermark information;

[0027] Based on the model loss function, the regularization term for embedding watermark information, and the regularization term for hiding watermark information, the weight information of the target neural network model is updated.

[0028] Optionally, the input subset is constructed by the following method:

[0029] Sub-input set: X = {x t |t∈(0,1,2...,n-1)}, where n is the number of target watermark information bits. For each x t , constructed according to the following formula:

[0030]

[0031] Where i∈(0,1,2,...,kerm), j∈(0,1,2,...,kerm), k∈(0,1,2,...,channel), kerm,kern are the width and height of the convolution kernel respectively, channel is the number of channels of the selected convolution kernel, x t is a three-dimensional matrix.

[0032] Optionally, the sub-tag set is constructed by the following method:

[0033] Sub-tag set: L = {l t |t∈(0,1,2...,n-1)}, where n is the number of target watermark information bits,

[0034] For each l t , constructed according to the following formula:

[0035]

[0036] Here, α is an adjustable hyperparameter, b is the sequence of target watermark information, and b[t] is the value of the tth bit in the target watermark information.

[0037] Optionally, the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix is ​​a mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix;

[0038] The mapping relationship between the watermark information and the sum of elements in the convolution kernel matrix includes:

[0039] When the value of the bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0;

[0040] When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0;

[0041] or,

[0042] When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0;

[0043] When the value of a bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0.

[0044] The present application also provides a data processing method, comprising:

[0045] Obtaining a target neural network model;

[0046] Obtaining a sub-input set; the target neural network model is obtained by: adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the weight parameters in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the weight parameters in the target convolution kernel matrix;

[0047] The sub-input set is input into the target neural network model to obtain output target watermark information.

[0048] The present application also provides a data processing device, comprising:

[0049] A target watermark information obtaining unit, used to obtain target watermark information;

[0050] A convolution kernel selection unit, used to select a target convolution kernel from the first neural network model and obtain a first operation result of an element in a target convolution kernel matrix;

[0051] The target watermark information mapping unit is used to map the target watermark information to the target convolution kernel according to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, so as to obtain the target neural network model.

[0052] The present application also provides an electronic device, comprising:

[0053] processor;

[0054] The memory is used to store a program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed:

[0055] Obtain target watermark information;

[0056] Selecting a target convolution kernel from the first neural network model, and obtaining a first operation result of an element in a target convolution kernel matrix;

[0057] According to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model.

[0058] The present application also provides a storage device storing a program of a data processing method, wherein the program is executed by a processor to perform the following steps:

[0059] Obtain target watermark information;

[0060] Selecting a target convolution kernel from the first neural network model, and obtaining a first operation result of an element in a target convolution kernel matrix;

[0061] According to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model.

[0062] The present application also provides a data processing device, comprising:

[0063] A target neural network model obtaining unit, used for obtaining a target neural network model;

[0064] A sub-input set obtaining unit, used for obtaining a sub-input set; the target neural network model is obtained by: adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the weight parameters in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the weight parameters in the target convolution kernel matrix;

[0065] The target watermark information obtaining unit is used to input the sub-input set into the target neural network model to obtain output target watermark information.

[0066] The present application also provides an electronic device, comprising:

[0067] processor;

[0068] The memory is used to store a program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed:

[0069] Obtaining a target neural network model;

[0070] Obtaining a sub-input set; the target neural network model is obtained by: adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the weight parameters in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the weight parameters in the target convolution kernel matrix;

[0071] The sub-input set is input into the target neural network model to obtain output target watermark information.

[0072] The present application also provides a storage device storing a program of a data processing method, wherein the program is executed by a processor to perform the following steps:

[0073] Obtaining a target neural network model;

[0074] Obtaining a sub-input set; the target neural network model is obtained by: adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the weight parameters in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the weight parameters in the target convolution kernel matrix;

[0075] The sub-input set is input into the target neural network model to obtain output target watermark information.

[0076] This application provides a method for identifying infringement of a neural network model, including:

[0077] Obtaining a first neural network model to be protected;

[0078] According to the data processing method as described above, a target neural network model with target watermark information added is obtained;

[0079] Using the specified data information to trigger the target neural network model, and obtaining the watermark information output by the target neural network model;

[0080] The output watermark information is compared with the target watermark information to identify whether the first neural network model is infringed.

[0081] Compared with the prior art, this application has the following advantages:

[0082] The present application provides a data processing method, which selects a target convolution kernel from a first neural network model, obtains a first operation result of an element in a target convolution kernel matrix; maps the target watermark information to the target convolution kernel according to a mapping relationship between the watermark information and the first operation result of the element in the convolution kernel matrix, and obtains a target neural network model. The data processing method provided by the present application does not need to save the matrix, which reduces storage overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1A This is a schematic diagram of a scenario provided in the first embodiment of the present application.

[0084] Figure 1 It is a flowchart of a data processing method provided in the first embodiment of the present application.

[0085] Figure 2 It is a schematic diagram of a network architecture designed according to the method of the first embodiment of the present application.

[0086] Figure 3 It is a flow chart of a data processing method provided in the second embodiment of the present application.

[0087] Figure 4 This is a schematic diagram of extracting target watermark information provided by this application.

[0088] Figure 5 It is a schematic diagram of a data processing device provided in the third embodiment of the present application.

[0089] Figure 6 It is a schematic diagram of an electronic device provided by the fourth embodiment and the seventh embodiment of the present application.

[0090] Figure 7 It is a schematic diagram of a data processing device provided in the sixth embodiment of the present application.

[0091] Figure 8 This is a flowchart of a method for identifying infringement of a neural network model provided in the ninth embodiment of the present application. DETAILED DESCRIPTION

[0092] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0093] In order to present the present application more clearly, the application scenario of the data processing method provided in the first embodiment of the present application is briefly introduced.

[0094] The data processing method provided in the first embodiment of the present application can be applied to the scenario of interaction between the client and the server, such as Figure 1A When the target watermark information needs to be embedded into the first neural network model, the client usually first establishes a connection with the server. After the connection, the client sends the first neural network model and the target watermark information to the server. After the server receives the first neural network model and the target watermark information, it first selects the target convolution kernel from the first neural network model to obtain the first operation result of the elements in the target convolution kernel matrix; then, according to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model. After that, the server provides the target neural network model to the client, and the client receives the target neural network model.

[0095] The first embodiment of the present application provides a data processing method. Figure 1 Make an introduction.

[0096] like Figure 1 As shown, in step S101, target watermark information is obtained.

[0097] The target watermark information refers to the watermark information to be mapped to the first neural network model, and the target watermark information may be a bit sequence of a predetermined number of bits.

[0098] like Figure 1 As shown, in step S102, a target convolution kernel is selected from the first neural network model to obtain the sum of the elements in the target convolution kernel matrix.

[0099] The neural network model learns and masters the data processing capabilities for input signals through multi-layer feature extraction and by adjusting and optimizing the neural network parameters. The input signal can be a digital signal sample of multimedia information such as voice, image, video, etc. The digital signal sample and network structure determine the type of data processing capability of the neural network, and the parameters of the neural network determine the quality of the data processing capability. Neural networks include various forms of network structures, such as convolutional neural networks (CNN), recurrent neural networks (RNN), and generative adversarial networks (GAN).

[0100] The first neural network model may refer to a neural network model that has not yet mapped the target watermark information into weight parameters.

[0101] The target convolution kernel refers to the convolution kernel to be mapped with the target watermark information. The target convolution kernel can be single-channel or multi-channel.

[0102] The sum of the elements in the target convolution kernel matrix may refer to the sum of the weight parameters in the target convolution kernel matrix. For example, the target convolution kernel matrix is:

[0103] 1 2 3 4 5 6 7 8 9

[0104] The sum of the elements in the target convolution kernel matrix is ​​1+2+3+4+5+6+7+8+9=45.

[0105] like Figure 1 As shown, in step S103, according to the mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model.

[0106] It should be noted that the mapping relationship adopted in the first embodiment of the present application is the mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix. In a specific implementation, the sum of the elements in the convolution kernel matrix may also be other first operation results of the elements in the convolution kernel matrix.

[0107] By mapping the target watermark information to the weight parameters of the first neural network model, the target neural network model is obtained, which can effectively protect the weight parameters in the target neural network model. When an unauthorized third party uses the weight parameters in the target neural network model, the target watermark information can be obtained based on the mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix to determine whether the third party has infringed.

[0108] The mapping relationship between the watermark information and the sum of elements in the convolution kernel matrix includes:

[0109] When the value of the bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0;

[0110] When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0;

[0111] or,

[0112] When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0;

[0113] When the value of a bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0.

[0114] In the first embodiment of the present application, the positive or negative value of the sum of the elements of the convolution kernel matrix is ​​used to represent the watermark information 0 or 1. If the sum of the elements of the convolution kernel matrix is ​​positive, it represents the watermark information 1, otherwise it represents the watermark information 0. Alternatively, if the sum of the elements of the convolution kernel matrix is ​​negative, it represents the watermark information 0, otherwise it represents the watermark information 1.

[0115] Of course, in a specific implementation, in addition to the above mapping relationship, other mapping relationships can also be established between the watermark information and the elements in the convolution kernel matrix.

[0116] For example, if the target watermark information is 1, 1, 1, 0, the sum of the weight parameters of the first three convolution kernel matrices corresponding to the bits of the target watermark information is greater than 0, and the sum of the weight parameters of the fourth convolution kernel matrix is ​​less than 0.

[0117] According to the mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix, mapping the target watermark information to the target convolution kernel to obtain the target neural network model includes:

[0118] The elements in the target convolution kernel matrix are adjusted until the sum of the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

[0119] Get the sum of the elements in the target convolution kernel matrix, including:

[0120] Get the sum of the weight parameters in the target convolution kernel matrix;

[0121] The adjusting the elements in the target convolution kernel matrix until the sum of the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship to obtain the target neural network model includes:

[0122] The weight parameters in the target convolution kernel matrix are adjusted until the sum of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

[0123] The step of adjusting the weight parameters in the target convolution kernel matrix until the sum of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship to obtain a target neural network model includes:

[0124] According to the sub-input set and the sub-label set, the weight parameters in the convolution kernel matrix are adjusted until the sum of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, and the target neural network model is obtained, wherein the sub-input set is a parameter set for calculating the sum of the weight parameters in the convolution kernel matrix, and the sub-label set is a set of target values ​​of the sum of the weight parameters in the convolution kernel matrix.

[0125] The sub-input set can be constructed by the following method:

[0126] Sub-input set: X = {x t |t∈(0,1,2...,n-1)}, where n is the number of target watermark information bits. For each x t , constructed according to the following formula:

[0127]

[0128] Where i∈(0,1,2,...,kerm), j∈(0,1,2,...,kerm), k∈(0,1,2,...,channel), kerm,kern are the width and height of the convolution kernel respectively, channel is the number of channels of the selected convolution kernel, x t is a three-dimensional matrix.

[0129] The sub-tag set is constructed by the following method:

[0130] Sub-tag set: L = {l t |t∈(0,1,2...,n-1)}, where n is the number of target watermark information bits,

[0131] For each l t , constructed according to the following formula:

[0132]

[0133] Here, α is an adjustable hyperparameter, b is the sequence of target watermark information, and b[t] is the value of the tth bit in the target watermark information.

[0134] It should be noted that the sub-input constructed above is essentially a matrix of all 1s, which is used to calculate the sum of the elements of the convolution kernel matrix. The sub-input can be convolved with the target convolution kernel matrix, and the result is a matrix, the value of the element in the middle of this matrix is ​​the sum of the elements of the target convolution kernel matrix.

[0135] For example:

[0136] Sub input is

[0137] 1 1 1 1 1 1 1 1 1

[0138] The convolution kernel matrix is:

[0139] 1 2 3 4 5 6 7 8 9

[0140] The sub-input is convolved with the target convolution kernel matrix to obtain:

[0141] 12 21 16 27 45 33 24 39 28

[0142] The value of the middle element of the obtained matrix, 45, is the sum of the elements of the target convolution kernel matrix.

[0143] Sub-inputs can also be matrices that are not all 1. They can be used to adjust the weights of the elements of the convolution kernel matrix. For example, the constructed sub-inputs can make the weight of the edge value of the convolution kernel 0.1, the weight of the four fields 0.5, and the center weight 1. Such assignment forms are essentially the same.

[0144] The following example illustrates the role of sub-labels. If the sub-input is

[0145] 1 1 1 1 1 1 1 1 1

[0146] The convolution kernel matrix is:

[0147] 1 2 3 4 5 6 7 8 9

[0148] If the mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix is: when the bit value of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0. Assuming that you want to embed 0, the sub-label is constructed as -0.2, but the sum of the convolution kernel is calculated to be 45, which means that the sum of the convolution kernel needs to be modified until its value becomes close to the value of the sub-label -0.2.

[0149] The step of adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until the sum of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship includes:

[0150] The weight parameters of the convolution kernel are adjusted according to the sub-input set and the sub-label set and the watermark loss function embedded in the first neural network model until the sum of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship and the watermark loss function converges.

[0151] After the target watermark information is mapped to the weight parameters of the target neural network, the statistical characteristics of the weight parameters of the target neural network will change. Therefore, it is easy to be detected by professionals and vulnerable to attack. Therefore, the weight parameters of the target neural network need to be hidden.

[0152] The first embodiment of the present application may further include:

[0153] Obtaining reference weight information of a convolution kernel matrix of a reference neural network model; the reference neural network model and the target neural network model have the same structure; the convolution kernel of the reference neural network model corresponds to the target convolution kernel selected from the first neural network model;

[0154] Obtaining target weight information of the convolution kernel matrix of the target neural network model in which part of the watermark information has been embedded;

[0155] The reference weight information and the target weight information are used as training sets to train a detection neural network model, and according to the distinguishability data of the reference weight information and the target weight information output by the detection neural network model, the training method of the target neural network model is adjusted to obtain a target neural network model that meets the watermark information hiding conditions.

[0156] The training process of the detection neural network model and the training process of the target neural network model when mapping the watermark information to the weight parameters of the convolution kernel are adversarial training processes, that is, while training the target neural network model (the process of mapping the watermark information to the weight parameters of the convolution kernel), the detection neural network model is trained. The purpose of training the detection neural network model is to enable it to distinguish the target weight information from the reference weight information without the watermark information embedded. When the target neural network model maps the watermark information to the weight parameters of the convolution kernel, the weight information is further updated according to the distinguishability data fed back by the detection neural network model, so that its target weight information is indistinguishable from the reference weight information without the watermark information embedded. In the above adversarial training process, the detection neural network model and the target neural network model are cross-trained, and the training set of one neural network model will change with the change of the training set of the other neural network model until the loss functions of the two neural network models converge.

[0157] The adjusting the model training method of the target neural network according to the distinguishability information of the reference weight information and the target weight information output by the detection neural network model includes:

[0158] Obtaining a regularization term for hiding watermark information according to the distinguishability information;

[0159] Adding the regularization term for hiding the watermark information based on the model loss function corresponding to the target neural network model when the conventional training set is used to train the target neural network model and the regularization term for mapping the watermark information;

[0160] Based on the model loss function, the regularization term for mapping the watermark information, and the regularization term for hiding the watermark information, the weight information of the target neural network model is updated.

[0161] In order to more clearly illustrate the method of the first embodiment of the present application, Figure 2 For introduction. Figure 2As shown, it is a schematic diagram of the network architecture designed according to the method of the first embodiment of the present application. The neural network model is a dual-task network with two tasks: the first is its original task (if you want to embed the target watermark information in a face recognition network, then the most basic condition is that the performance of the neural network after embedding the target watermark information is almost unaffected when performing face detection), and the main input and main output correspond to this task. The second task is the watermark sequence (if the embedded watermark sequence is 1 0 1 0, then the sum of the elements of the specific unknown convolution kernel matrix should be positive and negative), and the sub-input and sub-output correspond to this task, and the detection neural network model is an adversarial network. By loading the watermark loss function onto the original loss function, a dual-task network can be constructed. The second task calculates the loss of the sub-output (subOutput) and the sub-label (for example, subOutput is 1 1 1 1, but the sub-label is sublabel = 0.2, -0.2, 0.2, -0.2, then the loss of the two is mse (subOutput, sublabel), where mse is the mean square error. Through a series of operations such as automatic derivation, the loss is eventually made close to 0, that is, when the value of subOutput is close to sublabel, the network converges, and the final sum of the convolution kernel is positive and negative, which is consistent with the target watermark sequence, and the embedding is completed.

[0162] The second embodiment of the present application provides a data processing method. Figure 3 Make an introduction.

[0163] like Figure 3 As shown, in step S301, a target neural network model is obtained.

[0164] The target neural network model has the same structure as the watermark information when it is embedded. For example, a network model with the same network architecture as that designed by the method of the first embodiment of the present application can be built.

[0165] like Figure 3 As shown, in step S302, a sub-input set is obtained; the target neural network model is obtained in the following manner: according to the sub-input set and the sub-label set, the weight parameters in the convolution kernel matrix are adjusted until the sum of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship; wherein the sub-input set is a parameter set for calculating the sum of the weight parameters in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the sum of the weight parameters in the target convolution kernel matrix.

[0166] The sub-input set is constructed in the same manner as the first embodiment of the present application and will not be described in detail here.

[0167] like Figure 3As shown, in step S303, the subset input set is input into the target neural network model to obtain output target watermark information.

[0168] The second embodiment of the present application provides a method for obtaining target watermark information based on a target neural network model. In specific implementation, another method can also be used to obtain the target watermark information: obtain the target convolution kernel; calculate the sum of the weight parameters of the target convolution kernel matrix, and obtain the target watermark information based on the mapping relationship.

[0169] For example, Figure 4 As shown in the figure, there are three target convolution kernels, each of which is an eight-channel kernel of 3x3x8. The first target watermark information corresponds to the first channel of the convolution kernel. When extracting the first target watermark information, the sum of the weight parameters of the matrix of the first channel of each convolution kernel is calculated respectively, sum1, sum2, sum3, and then all the cumulative sums are calculated to be 15. Since 15 is greater than 0, the first watermark value is 1 (mapping relationship: when the sum of the weight parameters of the target convolution kernel matrix is ​​greater than 0, the corresponding watermark information is 1).

[0170] Corresponding to the data processing method provided in the first embodiment of the present application, the third embodiment of the present application provides a data processing device.

[0171] like Figure 5 As shown, the device comprises:

[0172] The target watermark information obtaining unit 501 is used to obtain the target watermark information;

[0173] A convolution kernel selection unit 502 is used to select a target convolution kernel from the first neural network model and obtain a first operation result of an element in a target convolution kernel matrix;

[0174] The target watermark information mapping unit 503 is used to map the target watermark information to the target convolution kernel according to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, so as to obtain the target neural network model.

[0175] Optionally, the target watermark information mapping unit is specifically used to:

[0176] The elements in the target convolution kernel matrix are adjusted until the first operation result between the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

[0177] Optionally, the convolution kernel selection unit is specifically used for:

[0178] Obtaining a first operation result of a weight parameter in a target convolution kernel matrix;

[0179] The target watermark information mapping unit is specifically used to:

[0180] The weight parameters in the target convolution kernel matrix are adjusted until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

[0181] Optionally, the target watermark information mapping unit is specifically used to:

[0182] According to the sub-input set and the sub-label set, the weight parameters in the convolution kernel matrix are adjusted until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, and the target neural network model is obtained, wherein the sub-input set is a parameter set for calculating the first operation result of the weight parameters in the convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the weight parameters in the convolution kernel matrix.

[0183] Optionally, the target watermark information mapping unit is specifically used to:

[0184] The weight parameters of the convolution kernel are adjusted according to the sub-input set and the sub-label set and the watermark loss function embedded in the first neural network model until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship and the watermark loss function converges.

[0185] Optionally, the device further comprises:

[0186] A reference weight information obtaining unit, used to obtain reference weight information of a convolution kernel matrix of a reference neural network model; the reference neural network model has the same structure as the target neural network model; the convolution kernel of the reference neural network model corresponds to the target convolution kernel selected from the first neural network model;

[0187] A target weight information obtaining unit, used to obtain target weight information of a convolution kernel matrix of a target neural network model in which a portion of watermark information has been embedded;

[0188] A model training unit is used to train a detection neural network model using the reference weight information and the target weight information as training sets, and adjust the training method of the target neural network model according to the distinguishability data of the reference weight information and the target weight information output by the detection neural network model to obtain a target neural network model that meets the watermark information hiding conditions.

[0189] Optionally, the model training unit is specifically used for:

[0190] Obtaining a regularization term for hiding watermark information according to the distinguishability information;

[0191] Adding the regularization term for hiding the watermark information based on the model loss function corresponding to the target neural network model when the conventional training set is used to train the target neural network model and the regularization term for mapping the watermark information;

[0192] Based on the model loss function, the regularization term for mapping the watermark information, and the regularization term for hiding the watermark information, the weight information of the target neural network model is updated.

[0193] Optionally, the input subset is constructed by the following method:

[0194] Sub-input set: X = {x t |t∈(0,1,2...,n-1)}, where n is the number of target watermark information bits. For each x t , constructed according to the following formula:

[0195]

[0196] Where i∈(0,1,2,...,kerm), j∈(0,1,2,...,kerm), k∈(0,1,2,...,channel), kerm,kern are the width and height of the convolution kernel respectively, channel is the number of channels of the selected convolution kernel, x t is a three-dimensional matrix.

[0197] Optionally, the sub-tag set is constructed by the following method:

[0198] Sub-tag set: L = {l t |t∈(0,1,2...,n-1)}, where n is the number of target watermark information bits,

[0199] For each l t , constructed according to the following formula:

[0200]

[0201] Here, α is an adjustable hyperparameter, b is the sequence of target watermark information, and b[t] is the value of the tth bit in the target watermark information.

[0202] Optionally, the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix is ​​a mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix; the mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix includes:

[0203] When the value of the bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0;

[0204] When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0;

[0205] or,

[0206] When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0;

[0207] When the value of a bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0.

[0208] It should be noted that for the detailed description of the device provided in the third embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0209] Corresponding to the data processing method provided in the first embodiment of the present application, the fourth embodiment of the present application provides an electronic device.

[0210] like Figure 6 As shown, the electronic device comprises:

[0211] Processor 601;

[0212] The memory 602 is used to store the program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed:

[0213] Obtain target watermark information;

[0214] Selecting a target convolution kernel from the first neural network model, and obtaining a first operation result of an element in a target convolution kernel matrix;

[0215] According to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model.

[0216] Optionally, mapping the target watermark information to the target convolution kernel according to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix to obtain the target neural network model includes:

[0217] The elements in the target convolution kernel matrix are adjusted until the first operation result between the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

[0218] Optionally, obtaining a first operation result of an element in a target convolution kernel matrix includes:

[0219] Obtaining a first operation result of a weight parameter in a target convolution kernel matrix;

[0220] The adjusting the elements in the target convolution kernel matrix until the first operation result between the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship to obtain the target neural network model includes:

[0221] The weight parameters in the target convolution kernel matrix are adjusted until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

[0222] Optionally, the adjusting the weight parameters in the target convolution kernel matrix until a first operation result between the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship to obtain a target neural network model includes:

[0223] According to the sub-input set and the sub-label set, the weight parameters in the convolution kernel matrix are adjusted until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, and the target neural network model is obtained, wherein the sub-input set is a parameter set for calculating the first operation result of the weight parameters in the convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the weight parameters in the convolution kernel matrix.

[0224] Optionally, adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until a first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship includes:

[0225] The weight parameters of the convolution kernel are adjusted according to the sub-input set and the sub-label set and the watermark loss function embedded in the first neural network model until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship and the watermark loss function converges.

[0226] Optionally, the electronic device further performs the following steps:

[0227] Obtaining reference weight information of a convolution kernel matrix of a reference neural network model; the reference neural network model and the target neural network model have the same structure; the convolution kernel of the reference neural network model corresponds to the target convolution kernel selected from the first neural network model;

[0228] Obtaining target weight information of the convolution kernel matrix of the target neural network model in which part of the watermark information has been embedded;

[0229] The reference weight information and the target weight information are used as training sets to train a detection neural network model, and according to the distinguishability data of the reference weight information and the target weight information output by the detection neural network model, the training method of the target neural network model is adjusted to obtain a target neural network model that meets the watermark information hiding conditions.

[0230] Optionally, adjusting the model training method of the target neural network according to the distinguishability information between the reference weight information and the target weight information output by the detection neural network model includes:

[0231] Obtaining a regularization term for hiding watermark information according to the distinguishability information;

[0232] Adding the regularization term for hiding the watermark information based on the model loss function corresponding to the target neural network model when the conventional training set is used to train the target neural network model and the regularization term for mapping the watermark information;

[0233] Based on the model loss function, the regularization term for mapping the watermark information, and the regularization term for hiding the watermark information, the weight information of the target neural network model is updated.

[0234] Optionally, the input subset is constructed by the following method:

[0235] Sub-input set: X = {x t |t∈(0,1,2...,n-1)}, where n is the number of target watermark information bits. For each x t , constructed according to the following formula:

[0236]

[0237] Where i∈(0,1,2,...,kerm), j∈(0,1,2,...,kerm), k∈(0,1,2,...,channel), kerm,kern are the width and height of the convolution kernel respectively, channel is the number of channels of the selected convolution kernel, x t is a three-dimensional matrix.

[0238] Optionally, the sub-tag set is constructed by the following method:

[0239] Sub-tag set: L = {l t |t∈(0,1,2...,n-1)}, where n is the number of target watermark information bits,

[0240] For each l t , constructed according to the following formula:

[0241]

[0242] Here, α is an adjustable hyperparameter, b is the sequence of target watermark information, and b[t] is the value of the tth bit in the target watermark information.

[0243] Optionally, the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix is ​​a mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix; the mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix includes:

[0244] When the value of the bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0;

[0245] When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0;

[0246] or,

[0247] When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0;

[0248] When the value of a bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0.

[0249] It should be noted that for the detailed description of the electronic device provided in the fourth embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0250] Corresponding to the data processing method provided in the first embodiment of the present application, the fifth embodiment of the present application provides a storage device storing a program of the data processing method, the program is executed by a processor to perform the following steps:

[0251] Obtain target watermark information;

[0252] Selecting a target convolution kernel from the first neural network model, and obtaining a first operation result of an element in a target convolution kernel matrix;

[0253] According to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model.

[0254] It should be noted that for the detailed description of the storage device provided in the fifth embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0255] Corresponding to the data processing method provided in the second embodiment of the present application, the sixth embodiment of the present application provides a data processing device.

[0256] like Figure 7 As shown, the device comprises:

[0257] A target neural network model obtaining unit 701 is used to obtain a target neural network model;

[0258] The sub-input set obtaining unit 702 is used to obtain a sub-input set; the target neural network model is obtained by: adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until the first operation result of the target watermark information and the element in the target convolution kernel matrix satisfies the mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the element in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the element in the target convolution kernel matrix;

[0259] The target watermark information obtaining unit 703 is used to input the sub-input set into the target neural network model to obtain output target watermark information.

[0260] It should be noted that for the detailed description of the device provided in the sixth embodiment of the present application, reference can be made to the relevant description of the second embodiment of the present application, which will not be repeated here.

[0261] Corresponding to the data processing method provided in the second embodiment of the present application, the seventh embodiment of the present application provides an electronic device.

[0262] like Figure 6 As shown, the electronic device comprises:

[0263] Processor 601;

[0264] The memory 602 is used to store the program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed:

[0265] Obtaining a target neural network model;

[0266] Obtaining a sub-input set; the target neural network model is obtained by: adjusting the elements in the convolution kernel matrix according to the sub-input set and the sub-label set until the first operation result of the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the elements in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the elements in the target convolution kernel matrix;

[0267] The sub-input set is input into the target neural network model to obtain output target watermark information.

[0268] It should be noted that for the detailed description of the electronic device provided in the seventh embodiment of the present application, reference can be made to the relevant description of the second embodiment of the present application, which will not be repeated here.

[0269] Corresponding to the data processing method provided in the second embodiment of the present application, the eighth embodiment of the present application provides a storage device storing a program of the data processing method, the program is executed by a processor to perform the following steps:

[0270] Obtaining a target neural network model;

[0271] Obtaining a sub-input set; the target neural network model is obtained by: adjusting the elements in the convolution kernel matrix according to the sub-input set and the sub-label set until the first operation result of the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the elements in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the elements in the target convolution kernel matrix;

[0272] The sub-input set is input into the target neural network model to obtain output target watermark information.

[0273] It should be noted that for the detailed description of the storage device provided in the eighth embodiment of the present application, reference can be made to the relevant description of the second embodiment of the present application, which will not be repeated here.

[0274] Corresponding to the data processing method provided in the first embodiment of the present application, the ninth embodiment of the present application provides a method for identifying infringement of a neural network model, please refer to Figure 8 , which is a flow chart of the neural network model infringement identification method provided by this application. Figure 8 Make an introduction.

[0275] like Figure 8 As shown, in step S801, a first neural network model to be protected is obtained.

[0276] Since deep neural network models require powerful computing power, neural network expertise, and training data to obtain a highly accurate model, it is easy to plagiarize deep neural network models because they are difficult to build. In reality, more and more neural networks have become targets of plagiarism. Therefore, it is increasingly important to protect the intellectual property rights of neural network models.

[0277] like Figure 8As shown, in step S802, according to the data processing method provided in the second embodiment of the present application, a target neural network model with target watermark information added is obtained.

[0278] In the second embodiment, how to obtain the target neural network model with the target watermark information added has been described in detail, which will not be repeated here. Please refer to the second embodiment of the present application.

[0279] like Figure 8 As shown, in step S803, the target neural network model is triggered by using the specified data information to obtain the watermark information output by the target neural network model.

[0280] The target neural network model can be triggered using model data associated with the target tag to obtain watermark information output by the target neural network model.

[0281] like Figure 8 As shown, in step S804, the output watermark information is compared with the target watermark information to identify whether the first neural network model is infringed.

[0282] If the output watermark information is consistent with the target watermark information, it is considered that the first neural network model to be protected has been infringed.

[0283] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0284] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0285] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0286] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0287] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

Claims

1. A data processing method, characterized in that: include: Obtain target watermark information; Selecting a target convolution kernel from the first neural network model, and obtaining a first operation result of an element in a target convolution kernel matrix; According to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model; including: The elements in the target convolution kernel matrix are adjusted until the first operation result between the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

2. The method according to claim 1, characterized in that Obtaining the first operation result of the elements in the target convolution kernel matrix includes: Obtaining a first operation result of a weight parameter in a target convolution kernel matrix; The adjusting the elements in the target convolution kernel matrix until the first operation result between the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship to obtain the target neural network model includes: The weight parameters in the target convolution kernel matrix are adjusted until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, thereby obtaining a target neural network model.

3. The method according to claim 2, characterized in that The step of adjusting the weight parameters in the target convolution kernel matrix until the first operation result between the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship to obtain the target neural network model includes: According to the sub-input set and the sub-label set, the weight parameters in the convolution kernel matrix are adjusted until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship, and the target neural network model is obtained, wherein the sub-input set is a parameter set for calculating the first operation result of the weight parameters in the convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the weight parameters in the convolution kernel matrix.

4. The method according to claim 3, characterized in that The step of adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until the first operation result between the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship includes: The weight parameters of the convolution kernel are adjusted according to the sub-input set and the sub-label set and the watermark loss function embedded in the first neural network model until the first operation result of the target watermark information and the weight parameters in the target convolution kernel matrix satisfies the mapping relationship and the watermark loss function converges.

5. The method according to claim 4, characterized in that Also includes: Obtaining reference weight information of a convolution kernel matrix of a reference neural network model; the reference neural network model and the target neural network model have the same structure; The convolution kernel of the reference neural network model corresponds to the target convolution kernel selected from the first neural network model; Obtaining target weight information of the convolution kernel matrix of the target neural network model in which part of the watermark information has been embedded; The reference weight information and the target weight information are used as training sets to train a detection neural network model, and according to the distinguishability data of the reference weight information and the target weight information output by the detection neural network model, the training method of the target neural network model is adjusted to obtain a target neural network model that meets the watermark information hiding conditions.

6. The method according to claim 5, characterized in that The adjusting the model training method of the target neural network according to the distinguishability information of the reference weight information and the target weight information output by the detection neural network model includes: Obtaining a regularization term for hiding watermark information according to the distinguishability information; Adding the regularization term for hiding the watermark information based on the model loss function corresponding to the target neural network model when the conventional training set is used to train the target neural network model and the regularization term for mapping the watermark information; Based on the model loss function, the regularization term for mapping the watermark information, and the regularization term for hiding the watermark information, the weight information of the target neural network model is updated.

7. The method according to claim 1, characterized in that The mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix is ​​the mapping relationship between the watermark information and the sum of the elements in the convolution kernel matrix; The mapping relationship between the watermark information and the sum of elements in the convolution kernel matrix includes: When the value of the bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0; When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0; or, When the value of the bit of the watermark information is 0, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is greater than 0; When the value of a bit of the watermark information is 1, the sum of the elements of the convolution kernel matrix corresponding to the bit of the watermark information is less than 0.

8. A data processing method, characterized in that: include: Obtaining a target neural network model; Obtaining a sub-input set; the target neural network model is obtained by: adjusting the elements in the convolution kernel matrix according to the sub-input set and the sub-label set until the target watermark information and the first operation result of the elements in the target convolution kernel matrix satisfy a mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the elements in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the elements in the target convolution kernel matrix; The sub-input set is input into the target neural network model to obtain output target watermark information.

9. A data processing device, characterized in that: include: A target watermark information obtaining unit, used to obtain target watermark information; A convolution kernel selection unit, used to select a target convolution kernel from the first neural network model and obtain a first operation result of an element in a target convolution kernel matrix; The target watermark information mapping unit is used to map the target watermark information to the target convolution kernel according to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, so as to obtain the target neural network model; including: adjusting the elements in the target convolution kernel matrix until the first operation result between the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship, so as to obtain the target neural network model.

10. An electronic device, characterized in that: include: processor; The memory is used to store a program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed: Obtain target watermark information; Selecting a target convolution kernel from the first neural network model, and obtaining a first operation result of an element in a target convolution kernel matrix; According to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model; including: adjusting the elements in the target convolution kernel matrix until the first operation result of the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship to obtain the target neural network model.

11. A storage device, characterized in that: A program storing a data processing method is executed by a processor to perform the following steps: Obtain target watermark information; Selecting a target convolution kernel from the first neural network model, and obtaining a first operation result of an element in a target convolution kernel matrix; According to the mapping relationship between the watermark information and the first operation result of the elements in the convolution kernel matrix, the target watermark information is mapped to the target convolution kernel to obtain the target neural network model; including: adjusting the elements in the target convolution kernel matrix until the first operation result of the target watermark information and the elements in the target convolution kernel matrix satisfies the mapping relationship to obtain the target neural network model.

12. A data processing device, characterized in that: include: A target neural network model obtaining unit, used for obtaining a target neural network model; A sub-input set obtaining unit, used for obtaining a sub-input set; the target neural network model is obtained by: adjusting the weight parameters in the convolution kernel matrix according to the sub-input set and the sub-label set until the target watermark information and the first operation result of the element in the target convolution kernel matrix satisfy a mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the element in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the element in the target convolution kernel matrix; The target watermark information obtaining unit is used to input the sub-input set into the target neural network model to obtain output target watermark information.

13. An electronic device, characterized in that: include: processor; The memory is used to store a program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed: Obtaining a target neural network model; Obtaining a sub-input set; the target neural network model is obtained by: adjusting the elements in the convolution kernel matrix according to the sub-input set and the sub-label set until the target watermark information and the first operation result of the elements in the target convolution kernel matrix satisfy a mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the elements in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the elements in the target convolution kernel matrix; The sub-input set is input into the target neural network model to obtain output target watermark information.

14. A storage device, characterized in that: A program storing a data processing method is executed by a processor to perform the following steps: Obtaining a target neural network model; Obtaining a sub-input set; the target neural network model is obtained by: adjusting the elements in the convolution kernel matrix according to the sub-input set and the sub-label set until the target watermark information and the first operation result of the elements in the target convolution kernel matrix satisfy a mapping relationship; wherein the sub-input set is a parameter set for calculating the first operation result of the elements in the target convolution kernel matrix, and the sub-label set is a set of target values ​​of the first operation result of the elements in the target convolution kernel matrix; The sub-input set is input into the target neural network model to obtain output target watermark information.

15. A method for identifying infringement of a neural network model, characterized in that: include: Obtaining a first neural network model to be protected; According to the method according to any one of claims 1 to 8, a target neural network model with target watermark information added is obtained; Using the specified data information to trigger the target neural network model, and obtaining the watermark information output by the target neural network model; The output watermark information is compared with the target watermark information to identify whether the first neural network model is infringed.

Images