Watermark embedding and extracting method and electronic equipment

Abstract

The invention provides a watermark embedding and extracting method and electronic equipment, and the method comprises the steps: receiving an obtaining request sent by at least one user; randomly generating at least one piece of watermark information for at least one target layer of the target model according to the parameter weight distribution characteristics of the target model; for each target layer, embedding the watermark information corresponding to the target layer into the target layer according to the watermark embedding strength and the watermark embedding position of the target layer to obtain a first target model; and sending the first target model to a user corresponding to the acquisition request.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and electronic device for watermark embedding and watermark extraction. Background Art

[0002] At present, with the rapid development of artificial intelligence and deep learning, machine learning models or deep learning models are widely used in fields such as natural language processing, image recognition, and data generation. When users use related applications in these fields, they need to download related models from the server. During the model sharing or downloading process, the server will send the model to the user. Before sending the model, the server needs to embed a watermark in the model, and then send the model with the embedded watermark to the user to prevent the parameter weights of the model from being obtained or tampered with by attackers during the model sharing or downloading process. Summary of the invention

[0003] In view of this, the present disclosure provides a watermark embedding method and device.

[0004] One aspect of the present disclosure provides a watermark embedding method, including: receiving an acquisition request sent by at least one user; the acquisition request is used to acquire a target model; randomly generating at least one watermark information for at least one target layer of the target model according to the parameter weight distribution characteristics of the target model; for each target layer, embedding the watermark information corresponding to the target layer into the target layer according to the watermark embedding strength and watermark embedding position of the target layer to obtain a first target model; and sending the first target model to the user corresponding to the acquisition request.

[0005] According to an embodiment of the present disclosure, the acquisition request includes a user identifier, and the method further includes: acquiring a user key corresponding to the user identifier based on the user identifier; generating at least one watermark detection key for at least one target layer of the first target model respectively based on the user key and at least one watermark information; and sending the at least one watermark detection key to the user corresponding to the acquisition request.

[0006] According to an embodiment of the present disclosure, watermark information corresponding to the target layer is embedded into the target layer according to the watermark embedding strength and the watermark embedding position of the target layer to obtain a first target model, including: adjusting the watermark information corresponding to the target layer according to the watermark embedding strength of the target layer to obtain the first watermark information; and adjusting the parameter weight at the watermark embedding position of the target layer using the first watermark information to obtain the first target model.

[0007] According to an embodiment of the present disclosure, the target layer has M parameter weights, where m is an integer greater than or equal to 1 and less than or equal to M; the parameter weights at the watermark embedding position of the target layer are adjusted using the first watermark information, including: in response to the contribution of the mth parameter weight to the output result of the target model being less than a first threshold, adjusting the mth parameter weight using the first watermark information.

[0008] According to an embodiment of the present disclosure, the method also includes: in response to the layer contribution of the target layer being less than a second threshold, setting the watermark embedding strength of the target layer to a first strength; the layer contribution of the target layer is determined based on the contribution of each parameter weight within the target layer to the output result of the target model; in response to the layer contribution of the target layer being greater than or equal to the second threshold, setting the watermark embedding strength of the target layer to a second strength; the first strength is greater than the second strength.

[0009] According to an embodiment of the present disclosure, at least one watermark detection key is sent to a user corresponding to an acquisition request, including: encrypting at least one watermark detection key to obtain at least one encrypted watermark detection key; and sending at least one encrypted watermark detection key to a user corresponding to the acquisition request.

[0010] Another aspect of the present disclosure also provides a watermark extraction method, including: receiving a first target model; for each target layer of the first target model, extracting the watermark information of the target layer using a watermark detection key corresponding to the target layer according to the watermark embedding strength and watermark embedding position of the target layer.

[0011] According to an embodiment of the present disclosure, watermark information of a target layer is extracted using a watermark detection key corresponding to the target layer, including: receiving at least one watermark detection key for at least one target layer of a first target model respectively; extracting the watermark information of the target layer according to a user key corresponding to a user identifier and a watermark detection key corresponding to the target layer, when the sum of the correlations between a parameter weight of at least one target layer and a respective watermark detection key of at least one target layer is greater than a preset correlation.

[0012] According to an embodiment of the present disclosure, the method also includes: determining that the parameter weights of the first target model have been tampered with when the sum of the correlations between the parameter weights of at least one target layer and the respective watermark detection keys of at least one target layer is less than or equal to a preset correlation.

[0013] The third aspect of the present disclosure also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the following method: receiving an acquisition request sent by at least one user; the acquisition request is used to obtain a target model; randomly generating at least one watermark information for at least one target layer of the target model according to the parameter weight distribution characteristics of the target model; for each target layer, embedding the watermark information corresponding to the target layer into the target layer according to the watermark embedding strength and watermark embedding position of the target layer to obtain a first target model; and sending the first target model to the user corresponding to the acquisition request.

[0014] The fourth aspect of the present disclosure also provides an electronic device, comprising: one or more processors; a trusted execution environment for storing at least one watermark detection key; a storage device for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors execute the following method: receiving a first target model; for each target layer of the first target model, extracting the watermark information of the target layer according to the watermark embedding strength and watermark embedding position of the target layer using the watermark detection key corresponding to the target layer.

[0015] According to a watermark embedding method provided by an embodiment of the present disclosure, an acquisition request sent by at least one user is received, and at least one watermark information for at least one target layer of the target model is randomly generated according to the parameter weight distribution characteristics of the target model. The watermark information of each target layer is unique, which improves the concealment of the watermark information and makes it more difficult to extract and crack the watermark information. The watermark information is randomly generated to ensure that the watermark information of each target layer is different and unpredictable. The watermark information is generated according to the parameter weight distribution characteristics of the target model, so that the embedded watermark information can be close to the parameter weight distribution characteristics, so that the embedded watermark information will not have a significant impact on the performance of the target model. The watermark embedding process is controlled by the watermark embedding strength and the watermark embedding position, so that the original parameter weight of the target model can be embedded without significantly changing the original parameter weight of the target model, ensuring that the target model can be used normally after the watermark information is embedded. The target model embedded with the watermark information is sent to the user to prevent the parameter weight of the target model from being tampered with or copied during the distribution and use of the target model.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0018] Figure 1 A schematic diagram showing an application scenario of a watermark embedding method, apparatus, device, medium and program product according to an embodiment of the present disclosure;

[0019] Figure 2 A flow chart of a watermark embedding method according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 3 A schematic diagram of the structure of a target model of a watermark embedding method according to an embodiment of the present disclosure is shown;

[0021] Figure 4 A schematic diagram schematically shows a watermark embedding method according to an embodiment of the present disclosure;

[0022] Figure 5 A schematic diagram schematically shows a watermark detection key of a watermark embedding method according to an embodiment of the present disclosure;

[0023] Figure 6 A flow chart of a watermark extraction method according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 7 A flowchart schematically shows a watermark extraction method according to an embodiment of the present disclosure; and

[0025] Figure 8 A block diagram schematically shows an electronic device suitable for implementing a watermark extraction method according to an embodiment of the present disclosure;

[0026] Fig. 9 A block diagram of an electronic device suitable for implementing a watermark embedding method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Likewise, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0029] At present, with the rapid development of artificial intelligence and deep learning, machine learning models or deep learning models are widely used in fields such as natural language processing, image recognition, and data generation. When users use related applications in these fields, they need to download related models from the server. During the model sharing or downloading process, the server will send the model to the user. Before sending the model, the server needs to embed a watermark in the model, and then send the model with the embedded watermark to the user to prevent the parameter weights of the model from being obtained or tampered with by attackers during the model sharing or downloading process.

[0030] The current model watermark embedding method has the following problems:

[0031] (1) After embedding the watermark in the model parameter weight, the model parameter weight has changed, and the model after embedding the watermark needs to be retrained. The training cost is high and the process is cumbersome.

[0032] (2) The same watermark is embedded in the parameter weights of each layer of the model, which makes it easier to crack the watermark but less secure.

[0033] (3) Every user who obtains the model uses the same watermark detection key to extract the watermark. If the model is tampered with, it is difficult to trace the source of the tampering.

[0034] Figure 1 The following schematically shows an application scenario diagram of a watermark embedding method according to an embodiment of the present disclosure.

[0035] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0036] The user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication systems and client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0037] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0038] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0039] It should be noted that the watermark embedding method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the message processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The watermark embedding method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the message processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0041] The following will be based on Figure 1 The scene described by Figure 2~Figure 5 A watermark embedding method of the disclosed embodiment is described in detail.

[0042] Figure 2 The flowchart of a watermark embedding method according to an embodiment of the present disclosure is schematically shown. Figure 2As shown, a watermark embedding method according to an embodiment of the present disclosure includes steps S201 to S204.

[0043] S201: Receive an acquisition request sent by at least one user.

[0044] Among them, the acquisition request is used to obtain the target model. The target model is a model that has been trained and can perform specific tasks, such as a model for realizing intelligent question and answer, image recognition, speech recognition, data generation and other tasks. The target model has a multi-layer structure, and each layer includes multiple parameter weights. For example, the target model can be a convolutional neural network, a recurrent neural network, etc. After training, the target model can process the input data and generate corresponding output. The server can receive acquisition requests sent by one or more users. Each acquisition request sent by a user can be used to obtain one or more target models.

[0045] For example, see Figure 3 . Figure 3 FIG. 1 is a schematic diagram of a target model of a watermark embedding method according to an embodiment of the present disclosure. Figure 3 As shown in the figure, the target model is a neural network model, including an input layer, three hidden layers (L1, L2, L3) and an output layer. Each layer includes multiple neurons. Each circle represents a neuron. The connecting lines between layers represent the direction of data transmission. Each neuron receives the output of multiple neurons from the previous layer as input, and then adjusts the input through the parameter weight corresponding to each neuron, and then uses the transfer function to calculate an output. The weights marked in each circle (W11, W12, W13, W14... W33, W34) represent the parameter weights corresponding to each neuron. The parameter weights corresponding to each neuron include the weights on the connecting lines of all output directions corresponding to each neuron. , , is the input of the target model, , , is the output of the target model.

[0046] The output of the i-th neuron in the hidden layer L1 is:

[0047]

[0048] This is the transfer function of the neurons in the hidden layer L1.

[0049]

[0050] is a constant representing the bias of the neurons in the L1 layer. Similarly, the output of each neuron in the hidden layer L2 and the hidden layer L3 can be calculated. The server can receive a request from the user terminal device. For example, when a user downloads an application that depends on the target model on a mobile phone, a request is sent to the server to obtain the target model required by the application from the server. Alternatively, when the function of the target model of the server is integrated into a system, a request can be sent to the server to download the target model from the server.

[0051] S202: Randomly generate at least one watermark information for at least one target layer of the target model according to the parameter weight distribution characteristics of the target model.

[0052] The parameter weight distribution characteristics of the target model include the mean, variance or sparsity of the parameter weights of each target layer of the target model. The target layer of the target model is the layer in the target model where the watermark information needs to be embedded. The target layer can be pre-set, or a layer that has a significant impact on the output of the target model can be used as the target layer, such as the convolution layer, fully connected layer or batch normalization layer of the convolutional neural network model.

[0053] The watermark information corresponding to each target layer is randomly generated, wherein the watermark information can be a specific value, a sequence of values, or specific identification information. The watermark information is used to identify the owner or creator of the target model, and can also be used to detect whether the target model has been tampered with or used by an unauthorized user. Optionally, for each target layer, a random number generator can be used to generate a random number as the watermark information, or a random number matrix of the same shape as the parameter weight matrix can be randomly generated according to the parameter weight matrix of the target layer as the watermark information corresponding to the target layer.

[0054] A possible implementation of S202 is: when all parameter weights of the target model obey In the case of a Gaussian distribution, for the i-th target layer of the target model, a random number generator is used to generate a random number As the watermark information of the i-th target layer, . 0 is the mean value of all parameter weights, is the variance of all parameter weights. The higher the weight concentration of the target model, The larger the value, the greater the watermark value. The sum of the random numbers of each target layer can also be made zero to reduce the impact on the output results of the target model. For each target layer of the target model, a unique random number is generated as watermark information.

[0055] Another possible implementation of S202 is: when the target model is a neural network model, select several convolution layers in the target model as target layers. For each target layer, randomly generate a random number matrix whose shape matches the convolution kernel of the target layer so that the random number matrix can be embedded in the parameter weights of the convolution kernel of the target layer. Suppose we have a CNN model for image recognition, and we select three convolution layers as target layers to embed watermarks. For each layer, we generate a watermark information vector whose elements are random numbers, but the sum is zero to reduce the impact on the layer output.

[0056] S203: For each target layer, embed the watermark information corresponding to the target layer into the target layer according to the watermark embedding strength and the watermark embedding position of the target layer, so as to obtain a first target model.

[0057] The watermark embedding strength is used to characterize the degree of modification of the original parameter weights of the target model by the watermark information. If the watermark embedding strength is too large, the original parameter weights of the target model will change greatly, and the performance of the target model will be reduced. If the watermark embedding strength is too small, the watermark information will be easily ignored or lost during the use of the target model. The optimal value of the watermark embedding strength can be determined by experiments, or by using an optimization algorithm to determine the optimal value of the watermark embedding strength.

[0058] The watermark embedding position refers to the specific embedding position of the watermark information in the target layer of the target model. For example, the watermark embedding position can be a specific row or column in the parameter weight matrix of the target layer, or a specific parameter weight. Choosing the correct embedding position can improve the robustness of the watermark information, so that the embedded watermark information can still be detected after the target model is compressed or fine-tuned. Similarly, the appropriate watermark embedding position will also ensure that the embedding of the watermark information will not have a significant impact on the performance of the target model.

[0059] According to the watermark embedding strength and watermark embedding position of the target layer, the watermark information corresponding to the target layer is embedded into the target layer to obtain the first target model. For example, if the watermark information corresponding to the target layer is a random number, the size of the random number can be adjusted according to the watermark embedding strength of the target layer, and the adjusted random number is added to the parameter weight at the watermark embedding position of the target layer. Or when the watermark information corresponding to the target layer is identification information, the watermark information is embedded into the watermark embedding position of the target layer through a specific encoding method. Among them, the first target model is the target model after the watermark information is embedded.

[0060] S204: Send the first target model to the user corresponding to the acquisition request.

[0061] The first target model is the target model after the watermark is embedded. The user corresponding to the acquisition request is the user who sends the acquisition request. The sending process can use an encryption protocol to protect the first target model from being intercepted or tampered. Before sending to the user, the target model can be compressed, encrypted or otherwise processed to ensure the security of the first target model during the sending process.

[0062] For example, when a user downloads an application on a mobile phone, a request is sent to the server. The server embeds the watermark information into the target model, obtains the first target model, and sends the first target model to the user's mobile phone. The user starts receiving the first target model by clicking the "Download" button in the application. The server can set breakpoint resuming to ensure that the user can receive the first target model completely even when the network is unstable.

[0063] According to a watermark embedding method provided by an embodiment of the present disclosure, an acquisition request sent by at least one user is received, and at least one watermark information for at least one target layer of the target model is randomly generated according to the parameter weight distribution characteristics of the target model. The watermark information of each target layer is unique, which improves the concealment of the watermark information and makes it more difficult to extract and crack the watermark information. The watermark information is randomly generated to ensure that the watermark information of each target layer is different and unpredictable. The watermark information is generated according to the parameter weight distribution characteristics of the target model, so that the embedded watermark information can be close to the parameter weight distribution characteristics, so that the embedded watermark information will not have a significant impact on the performance of the target model. The watermark embedding process is controlled by the watermark embedding strength and the watermark embedding position, so that the original parameter weight of the target model can be embedded without significantly changing the original parameter weight of the target model, ensuring that the target model can be used normally after the watermark information is embedded. The target model embedded with the watermark information is sent to the user to prevent the parameter weight of the target model from being tampered with or copied during the distribution and use of the target model.

[0064] based on Figure 2 According to a watermark embedding method according to an embodiment of the present disclosure, in some embodiments, embedding watermark information corresponding to the target layer into the target layer according to the watermark embedding strength and watermark embedding position of the target layer to obtain a first target model may include the following steps:

[0065] The watermark information corresponding to the target layer is adjusted according to the watermark embedding strength of the target layer to obtain the first watermark information.

[0066] Each target layer of the target model has a corresponding watermark embedding strength. The watermark information can be scaled or transformed according to the watermark embedding strength of the target layer. The adjusted watermark information is the first watermark information.

[0067] The first watermark information is used to adjust the parameter weight at the watermark embedding position of the target layer to obtain a first target model.

[0068] After determining the watermark embedding position of the target layer, the first watermark information is used to adjust the parameter weight corresponding to the watermark embedding position, that is, the watermark information is embedded into certain parameter weights of the target layer. The watermark embedding position is certain parameter weights of the target layer. The adjustment process can use the watermark information to directly adjust the size of the parameter weight.

[0069] For example, see Figure 4 . Figure 4 FIG. 1 is a schematic diagram of a watermark embedding method according to an embodiment of the present disclosure. Figure 4 As shown in Figure 1, the target layers of the target model are L1, L2, and L3. The gray circles in the figure represent the watermark embedding position of each target layer. The embedding strength corresponding to the target layer L1 is , the watermark information corresponding to L1 is a random number , the watermark embedding position of the target layer L1 is the parameter weight , , The parameter weight of the target layer L1 after embedding the watermark information is: , , , .in,

[0070]

[0071]

[0072]

[0073] The embedding strength corresponding to the target layer L2 is , the watermark information corresponding to L2 is a random number , the watermark embedding position of the target layer L2 is the parameter weight The parameter weight of the target layer L2 after embedding the watermark information is: .in,

[0074]

[0075] The embedding strength corresponding to the target layer L3 is , the watermark information corresponding to L1 is a random number , the watermark embedding position of the target layer L3 is the parameter weight , , The parameter weight of the target layer L3 after embedding the watermark information is: , , .in,

[0076]

[0077]

[0078]

[0079] The input of the target model after embedding the watermark information is , , , the output is , , .

[0080] Based on the above embodiment, the watermark embedding position of the target layer can be determined according to the contribution of each parameter weight of the target layer to the output result of the target model. The target layer has M parameter weights, where m is an integer greater than or equal to 1 and less than or equal to M. Using the first watermark information to adjust the parameter weight at the watermark embedding position of the target layer includes: in response to the contribution of the mth parameter weight to the output result of the target model being less than a first threshold, using the first watermark information to adjust the mth parameter weight.

[0081] The contribution of the mth parameter weight in the target layer is used to characterize the influence of the mth parameter weight on the final output result of the target model, and also indicates the importance of the mth parameter weight to the target model. The greater the contribution of the parameter weight, the greater the influence of the parameter weight on the output result of the target model. The first threshold is the contribution threshold for determining the watermark embedding position. When the contribution of the parameter weight is less than the first threshold, it means that the parameter weight has a small influence on the output result of the target model. At this time, the first watermark information can be used to adjust the parameter weight to avoid the embedded watermark information affecting the output result of the target model. The first threshold can be a single value or a threshold matrix corresponding to the parameter weight matrix. Calculate the contribution of each parameter weight in the parameter weight matrix, and then determine whether it is a watermark embedding position based on the contribution threshold of the corresponding position in the threshold matrix.

[0082] The contribution of the parameter weight can be determined according to the absolute value of the gradient value of the parameter weight. The absolute value of the gradient of the parameter weight can be determined according to a plurality of training data used to train the target model and a loss function used to train the target model.

[0083] For example, the training data used to train the target model may include sample data and sample labels corresponding to the sample data. The sample labels corresponding to the sample data are the output results obtained after the sample data is input into the target model and the target model is processed. Specifically, the target model can be trained using the gradient descent method based on multiple training data, and the gradient values ​​corresponding to the multiple parameter weights of the target layer of the target model during the training process are monitored. The gradient values ​​corresponding to the multiple parameter weights can be obtained by taking the partial derivative of the parameter weights of the loss function. Take the gradient values ​​corresponding to the multiple parameter weights in the last round of training, and then take the absolute values ​​of the gradient values ​​corresponding to the multiple parameter weights to obtain the contribution degrees corresponding to the multiple parameter weights.

[0084] In some embodiments, the first threshold can be determined based on the ratio of the number of parameter weights in the target layer whose contribution is less than the first threshold to the number of all parameter weights in the target layer. For example, the contribution corresponding to 80% of the parameter weights in the target layer is less than the first threshold. The parameter weights can be sorted from small to large according to the contribution, and the top 10% of the parameter weights can be selected to embed the watermark information, which can improve the concealment and security of the watermark information without affecting the overall performance of the target model. In this case, the maximum value of the contribution corresponding to the top 10% of the parameter weights can be determined as the first threshold.

[0085] Based on the above embodiment, the watermark embedding strength of the target layer can be determined according to the layer contribution of the target layer, which specifically includes the following steps:

[0086] In response to the layer contribution of the target layer being less than a second threshold, the watermark embedding strength of the target layer is set to a first strength.

[0087] In response to the layer contribution of the target layer being greater than or equal to a second threshold, the watermark embedding strength of the target layer is set to a second strength.

[0088] The layer contribution of the target layer is determined based on the contribution of each parameter weight in the target layer to the output result of the target model. Specifically, the layer contribution of the target layer can be determined according to the norm of the gradient matrix composed of the gradient values ​​of each parameter weight in the target layer.

[0089] The second threshold is a contribution threshold for determining the watermark embedding strength. When the layer contribution of the target layer is less than the second threshold, it means that the target layer has a small influence on the output result of the target model, and the watermark embedding strength of the target layer is set to be larger. When the layer contribution of the target layer is greater than or equal to the second threshold, it means that the target layer has a large influence on the output result of the target model, and the watermark embedding strength of the target layer is set to be smaller, that is, the first strength is greater than the second strength, so as to avoid the embedded watermark information affecting the performance of the target model. The second threshold can be preset manually or determined by experiment.

[0090] For example, when the watermark embedding strength is lower than 0.001, the embedded watermark information is not obvious and does not protect the target model. When the watermark embedding strength exceeds 0.01, the performance of the target model after the watermark is embedded will decrease. Therefore, the second strength can be set to 0.001 and the first strength can be set to 0.01 to ensure that the embedded watermark information can protect the target model without affecting the original performance of the target model.

[0091] The watermark embedding position of the target layer is determined according to the contribution of each parameter weight of the target layer to the output result of the target model, and the embedding strength of the target layer is determined according to the layer contribution of the target layer. This can ensure the concealment of the embedded watermark information and reduce the impact of the watermark information on the performance of the target model.

[0092] Multiple users can send acquisition requests to obtain the same target model. After obtaining the target model, if each user uses the same watermark detection key to extract the watermark, once the watermark detection key is obtained by an unauthorized user, the unauthorized user can also use the obtained watermark detection key to extract the watermark information of the target model, causing the target model to be at risk of being copied and tampered with by unauthorized users.

[0093] To this end, based on Figure 2 A watermark embedding method according to an embodiment of the present disclosure may further include the following steps:

[0094] According to the user ID, obtain the user key corresponding to the user ID.

[0095] The acquisition request sent by the user includes a user identifier. The user identifier is identification information used to identify the user who sent the acquisition request. The user identifier is a unique identification code of the user and may include a user name, email address, mobile phone number, etc. The user identifier may be bound to a user terminal device or a user identity.

[0096] The server can maintain a user database, which is used to store the correspondence between user IDs and user keys. When the server receives a request from a user, it searches the user database for the corresponding user key based on the user ID in the request. The server can also randomly generate a user key based on the received request. For example, when all parameter weights of the target model are subject to In the case of Gaussian distribution, a random number r is generated as the user key. The user key can be a random number, a numerical sequence, or specific identification information.

[0097] At least one watermark detection key is generated for at least one target layer of the first target model according to the user key and at least one watermark information.

[0098] The watermark detection key is used to detect whether there is watermark information in the first target model, and is also used to extract the watermark information in the first target model. The watermark detection key corresponding to each target layer is generated using the user key and the watermark information of each target layer of the target model without embedded watermark information. Optionally, the watermark embedding strength and watermark embedding position corresponding to each target layer can be added to the watermark detection key corresponding to each target layer.

[0099] The generation of the watermark detection key depends on the user key and the watermark information. Therefore, even if multiple users obtain the same first target model, they need to use their respective corresponding watermark detection keys to extract the watermark information of the first target model.

[0100] For example, see Figure 5 . Figure 5 FIG. 1 is a schematic diagram of a method for generating a watermark detection key according to a watermark embedding method according to an embodiment of the present disclosure. Figure 5 As shown, the target layers of the target model are L1, L2, and L3. Received acquisition requests from N users. The user IDs of the N users are: user ID 1, user ID 2, ... user ID N. Randomly generate user keys corresponding to the user IDs. The user key corresponding to user ID 1 is The user key corresponding to user ID 2 is The user key corresponding to user ID N is .

[0101] The watermark information corresponding to the target layer L1 is a random number , the watermark detection key corresponding to the target layer L1 is The watermark information corresponding to the target layer L2 is a random number , the watermark detection key corresponding to the target layer L2 is The watermark information corresponding to the target layer L3 is a random number , the watermark detection key corresponding to the target layer L3 is .

[0102] For user ID 1, the watermark detection keys corresponding to the target layers L1, L2, and L3 can be generated according to the following formula:

[0103]

[0104]

[0105]

[0106] For user ID 2, the watermark detection keys corresponding to the target layers L1, L2, and L3 can be generated according to the following formula:

[0107]

[0108]

[0109]

[0110] For the user identifier N, the watermark detection key corresponding to the target layers L1, L2, and L3 can be generated according to the following formula:

[0111]

[0112]

[0113]

[0114] At least one watermark detection key is sent to a user corresponding to the acquisition request.

[0115] The watermark detection key is returned to the user corresponding to the acquisition request, so that each user can use the unique watermark detection key to perform watermark extraction and watermark existence detection on the target layer of the first target model.

[0116] According to the user identifier, a user key corresponding to the user identifier is obtained. This step is to authorize the user who sends the acquisition request. The user with the user key has the right to use the target model. According to the user key and at least one watermark information, at least one watermark detection key is generated for at least one target layer of the first target model, ensuring that only authorized users can use the corresponding watermark detection key to perform watermark detection and watermark extraction on the first target model, ensuring the security of the target model during use, and preventing unauthorized users from extracting or tampering with the watermark information of the first target model.

[0117] Based on the above embodiment, sending at least one watermark detection key to the user corresponding to the acquisition request may include: encrypting at least one watermark detection key to obtain at least one encrypted watermark detection key; and sending at least one encrypted watermark detection key to the user corresponding to the acquisition request.

[0118] Before sending the watermark detection key, the watermark detection key is encrypted to ensure that the watermark detection key is not eavesdropped or intercepted during the sending process. For example, before the watermark detection key is sent, the watermark detection key can be encrypted using the public key corresponding to the user ID. After the user receives the watermark detection key, the watermark detection key can be decrypted using the private key.

[0119] based on Figure 2 A watermark embedding method according to an embodiment of the present disclosure is shown, and a watermark extraction method according to an embodiment of the present disclosure includes: receiving a first target model; for each target layer of the first target model, extracting the watermark information of the target layer according to the watermark embedding strength and watermark embedding position of the target layer using a watermark detection key corresponding to the target layer.

[0120] Among them, the first target model is a target model embedded with watermark information. The watermark embedding strength and watermark embedding position of the target layer can be obtained from the configuration file of the first target model or from the watermark detection key. The watermark detection key can be obtained from the server and stored in the trusted execution environment of the user's terminal. When using the watermark detection key, the watermark detection key corresponding to the first target model can be queried in the trusted execution environment. The user's terminal can call the API interface of the watermark extraction algorithm provided by the server or the watermark extraction algorithm integrated in the user's terminal itself, and use the watermark detection key corresponding to the target layer to extract the watermark information of the target layer, and restore the first target model to a target model without embedded watermark information.

[0121] For example, see Figure 6 . Figure 6 FIG. 1 is a schematic diagram of the execution process of a watermark extraction method according to an embodiment of the present disclosure. Figure 6 As shown, the execution process of a watermark extraction method includes S601 to S605.

[0122] S601: The terminal sends an acquisition request, and the acquisition request user acquires the target model;

[0123] S602: After receiving the acquisition request, the server randomly generates at least one watermark information for at least one target layer of the target model according to the parameter weight distribution characteristics of the target model;

[0124] S603: The server performs a watermark information embedding process. Specifically, for each target layer, according to the watermark embedding strength and watermark embedding position of the target layer, the watermark information corresponding to the target layer is embedded into the target layer to obtain a first target model;

[0125] S604: The server returns the first target model to the terminal;

[0126] S605: The terminal receives the first target model, and for each target layer of the first target model, extracts the watermark information of the target layer according to the watermark embedding strength and the watermark embedding position of the target layer and using the watermark detection key corresponding to the target layer.

[0127] On the basis of the above embodiment, extracting the watermark information of the target layer using the watermark detection key corresponding to the target layer may include: receiving at least one watermark detection key for at least one target layer of the first target model; when the sum of the correlations between the parameter weight of at least one target layer and the respective watermark detection keys of at least one target layer is greater than a preset correlation, extracting the watermark information of the target layer according to the user key corresponding to the user identifier and the watermark detection key corresponding to the target layer.

[0128] The server generates a watermark detection key corresponding to each target layer according to the watermark information and user key of each target layer of the target model, and sends the watermark detection key to the user who initiates the acquisition request. The user's terminal can obtain the watermark detection key from the server.

[0129] For each target layer of the first target model, the watermark detection key corresponding to the target layer is used to extract the watermark information embedded in the target layer, so that the first target model can be restored to a target model without embedded watermark information.

[0130] Before extracting the watermark information, it is necessary to first detect whether the received first target model contains the watermark information. This is because the first target model may be attacked or tampered with during the sending process. For example, when the watermark detection key is sent from the server to the terminal, the attacker obtains the watermark detection key. , trying to use Extract the parameter weights of the first target model as The parameter weight of the first target model after the attack is ,in, . The terminal can detect whether the received first target model contains watermark information by calculating the correlation between the parameter weight of the received first target model and the received watermark detection key. For example: the correlation between the parameter weight of each target layer and the corresponding watermark detection key can be calculated first, and then the sum of the correlations of the parameter weights of all target layers can be calculated. If the sum of the correlations of the parameter weights of all target layers is greater than the preset correlation, it is determined that watermark information exists in the first target model. Among them, for each target layer, the correlation between the parameter weight and the corresponding watermark detection key can be obtained by performing a dot product operation on the parameter weight matrix of the target layer and the corresponding watermark detection key.

[0131] For example, see Figure 7 . Figure 7 FIG. 1 is a schematic diagram of the execution process of a watermark extraction method according to an embodiment of the present disclosure. Figure 7 As shown, the execution process of a watermark extraction method includes S701 to S708.

[0132] S701: The terminal sends an acquisition request, which includes a user identifier;

[0133] S702: The server obtains the user key corresponding to the user identifier according to the user identifier in the acquisition request;

[0134] S703: The server generates a watermark detection key for each target layer of the first target model according to the user key and the watermark information of each target layer;

[0135] S704: The server returns to the terminal a watermark detection key for each target layer of the first target model;

[0136] S705: The terminal calculates the correlation between the parameter weight of each target layer of the first target model and the watermark detection key of each target layer;

[0137] S706: The terminal calculates the sum of the correlations of all target layers;

[0138] S707: The terminal determines whether the sum of all correlations is greater than a preset correlation; if so, execute S708;

[0139] S708: The terminal executes a watermark extraction process: for each target layer of the first target model, the watermark information of the target layer is extracted according to the user key corresponding to the user identifier and the watermark detection key corresponding to the target layer.

[0140] Based on the above embodiment, when the sum of the correlations between the parameter weights of at least one target layer and the respective watermark detection keys of at least one target layer is less than or equal to a preset correlation, it can be determined that the parameter weights of the first target model are tampered with.

[0141] After determining that the parameter weights of the first target model have been tampered with, the tampered first target model can be sent to the server. After receiving the tampered model, the server will track the source of the tampering. For example: calculate the correlation between the parameter weights of the tampered model and all watermark detection keys generated by the server, and query the user identification information corresponding to the watermark detection key according to the watermark detection key corresponding to the maximum value of the correlation. Track the relevant users through the user identification information, and then determine the source of tampering.

[0142] Before extracting watermark information from the first target model, the correlation between the parameter weight of the received first target model and the received watermark detection key is calculated to detect whether the received first target model has watermark information. If the watermark information does not exist, it can be determined that the first target model has been tampered with, and then the source of the tampering is determined based on the watermark detection key.

[0143] Figure 8 A schematic block diagram of an electronic device that can be used to implement a watermark embedding method according to an embodiment of the present disclosure is schematically shown.

[0144] like Figure 8 As shown, the electronic device 800 according to the embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 to the random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an on-board memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for executing different actions of a watermark embedding method process according to an embodiment of the present disclosure.

[0145] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other through bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also implement a watermark embedding method provided by the embodiment of the present disclosure by executing the program stored in one or more memories.

[0146] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 808 as needed.

[0147] Fig. 9 A schematic block diagram of an electronic device that can be used to implement a watermark extraction method according to an embodiment of the present disclosure is schematically shown.

[0148] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 903 or a program loaded from a storage part 909 to a random access memory (RAM) 904. The processor 901 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an on-board memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure. The electronic device 900 also includes a trusted execution environment 902 for storing at least one watermark detection key.

[0149] In RAM 904, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 903 and RAM 904 are connected to each other through bus 905. Processor 901 executes various operations of a watermark extraction method flow according to an embodiment of the present disclosure by executing programs in ROM 903 and / or RAM 904. It should be noted that the program can also be stored in one or more memories other than ROM 903 and RAM 904. Processor 901 can also implement a watermark extraction method provided by an embodiment of the present disclosure by executing programs stored in one or more memories.

[0150] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 906, which is also connected to the bus 905. The electronic device 900 may further include one or more of the following components connected to the I / O interface 906: an input portion 907 including a keyboard, a mouse, etc.; an output portion 908 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 909 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 910 performs communication processing via a network such as the Internet. A drive 911 is also connected to the I / O interface 906 as needed. A removable medium 912, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 911 as needed, so that a computer program read therefrom is installed into the storage portion 909 as needed.

[0151] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0152] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0153] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0154] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 801 or the processor 901. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0155] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809 or the communication part 910, and / or installed from the removable medium 811 or the removable medium 912. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0156] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809 or the communication part 910, and / or installed from the removable medium 811 or the removable medium 912. When the computer program is executed by the processor 801 or the processor 901, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0157] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals. In the technical solution of this disclosure, the user's authorization or consent is obtained before obtaining or collecting user personal information.

[0158] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0159] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in a variety of ways, even if such combinations and / or couplings are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or couplings fall within the scope of the present disclosure.

[0161] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A watermark embedding method, comprising: receiving a request sent by at least one user; The acquisition request is used to acquire the target model; Randomly generating at least one watermark information for at least one target layer of the target model according to the parameter weight distribution characteristics of the target model; For each of the target layers, embedding the watermark information corresponding to the target layer into the target layer according to the watermark embedding strength and the watermark embedding position of the target layer, so as to obtain a first target model; The first target model is sent to a user corresponding to the acquisition request.

2. The method according to claim 1, wherein the acquisition request includes a user identifier, and the method further comprises: According to the user identifier, obtaining a user key corresponding to the user identifier; Generate at least one watermark detection key for at least one target layer of the first target model according to the user key and the at least one watermark information; The at least one watermark detection key is sent to a user corresponding to the acquisition request.

3. The method according to claim 1, wherein the step of embedding the watermark information corresponding to the target layer into the target layer according to the watermark embedding strength and the watermark embedding position of the target layer to obtain the first target model comprises: Adjust the watermark information corresponding to the target layer according to the watermark embedding strength of the target layer to obtain first watermark information; The first watermark information is used to adjust the parameter weight at the watermark embedding position of the target layer to obtain the first target model.

4. According to the method of claim 3, the target layer has M parameter weights, where m is an integer greater than or equal to 1 and less than or equal to M; the step of adjusting the parameter weights at the watermark embedding position of the target layer using the first watermark information comprises: In response to a contribution of the mth parameter weight to an output result of the target model being less than a first threshold, adjusting the mth parameter weight using the first watermark information.

5. The method according to claim 3, further comprising: In response to the layer contribution of the target layer being less than a second threshold, setting the watermark embedding strength of the target layer to a first strength; The layer contribution of the target layer is determined based on the contribution of each parameter weight in the target layer to the output result of the target model; In response to the layer contribution of the target layer being greater than or equal to the second threshold, the watermark embedding strength of the target layer is set to a second strength; the first strength is greater than the second strength.

6. The method according to claim 2, wherein sending the at least one watermark detection key to the user corresponding to the acquisition request comprises: encrypting the at least one watermark detection key to obtain at least one encrypted watermark detection key; The at least one encrypted watermark detection key is sent to a user corresponding to the acquisition request.

7. A watermark extraction method, comprising: receiving a first target model; For each target layer of the first target model, watermark information of the target layer is extracted according to the watermark embedding strength and the watermark embedding position of the target layer using a watermark detection key corresponding to the target layer.

8. The method according to claim 7, wherein extracting the watermark information of the target layer by using the watermark detection key corresponding to the target layer comprises: receiving at least one watermark detection key respectively for at least one target layer of the first target model; When the sum of the correlations between the parameter weight of at least one target layer and the watermark detection key of each of the at least one target layer is greater than a preset correlation, the watermark information of the target layer is extracted based on the user key corresponding to the user identifier and the watermark detection key corresponding to the target layer.

9. The method according to claim 8, further comprising: When the sum of the correlations between the parameter weight of the at least one target layer and the respective watermark detection keys of the at least one target layer is less than or equal to a preset correlation, it is determined that the parameter weight of the first target model is tampered with.

10. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the following method: Receiving an acquisition request sent by at least one user; the acquisition request is used to acquire a target model; Randomly generating at least one watermark information for at least one target layer of the target model according to the parameter weight distribution characteristics of the target model; For each of the target layers, embedding the watermark information corresponding to the target layer into the target layer according to the watermark embedding strength and the watermark embedding position of the target layer, so as to obtain a first target model; The first target model is sent to a user corresponding to the acquisition request.

Images