A neural network watermark embedding method, device, electronic device and storage medium

Abstract

The present invention relates to a neural network watermark embedding method, device, electronic device and storage medium. The neural network watermark embedding method described in the present invention comprises: obtaining a key K, wherein the key K corresponds to a unique timestamp T; randomly selecting n pictures in the original training set, wherein n is greater than 1, wherein the original training set includes multiple pictures and labels corresponding to each picture; scrambling and encrypting the pictures by a chaotic sequence generated by the key K to obtain n pictures of a trigger set, and putting the n pictures and their corresponding labels into a trigger set; putting the original training set and the trigger set into a neural network to be embedded with a watermark for training to obtain a neural network embedded with a watermark. The neural network watermark embedding method described in the present invention generates a trigger set while keeping the image features unchanged, and gives it an original label, so that the ownership of the model is protected without affecting the decision boundary of the original classification task.

Description

Technical Field

[0001] The present invention relates to the field of information hiding, and in particular to a neural network watermark embedding method, device, electronic equipment and storage medium. Background Art

[0002] As deep learning models become more and more widely used, many large companies have begun to use their large amounts of proprietary data and computing resources to train deep learning models, and then deploy and commercialize them and distribute them to other users as a way to make money. However, this may lead to copyright disputes, so a safe and reliable copyright protection method is needed to protect the interests of the owner.

[0003] In recent years, researchers have borrowed the ideas of the original digital watermarking method and proposed a watermarking method suitable for copyright protection of deep learning models. Watermarking methods based on deep learning copyright protection can be roughly divided into two types: one is called white-box watermarking, which embeds watermarks by modifying the weights in the model. This method is similar to the principle of original digital watermarking; the other method is called black-box watermarking. This method does not require access to the internal parameters of the network. It embeds watermark information by using a specific trigger set as the input of the neural network and outputting a specified label. Finally, the ownership of the network is verified by comparing the correctness of the specified label and the output label.

[0004] For white-box watermarking, Uchida et al. borrowed the idea of ​​digital watermarking to make the first attempt, and realized the embedding of watermarks by modifying some weight parameters in the network, and the original performance of the network was not affected after the watermark information was embedded. Wang and Kerschbaum et al. found that Uchida's scheme modified the statistical distribution of the model, which would cause the attacker to not only detect the watermark, but also extract its embedding length and delete the watermark through a coverage attack. In order to improve the stability and security of embedded watermarks, Wang and Wu et al. based on Uchida's method, screened out some weights that had relatively little impact on the performance of the embedded model, and used a separate neural network to generate a matrix for embedding watermark information. Wang and Kerschbaum et al. proposed a white-box watermarking method based on generative adversarial networks, which made it impossible to distinguish the parameter distribution of the neural network with embedded watermarks from that of the neural network without watermarks, making it difficult to detect. Due to the limitations of white-box watermarking, watermark embedding requires access to and extraction of the internal parameters and structure of the model. Some scholars proposed a method to add watermarks to the neural network model in a black-box manner, borrowing the idea of ​​backdoor attacks, so that details such as model parameters can be extracted when verifying the watermark.

[0005] For black-box watermarking, Adi et al. first proposed to embed abstract images as backdoor watermarks into neural networks, thereby ensuring the ownership of the model without accessing the internal parameters and structure of the model; based on Adi et al., Zhang et al. proposed a black-box method based on three different methods: text, noise, and irrelevant pictures as trigger sets; in order to resist watermark attacks such as parameter fine-tuning and adversarial fine-tuning coverage, Merrer et al. proposed a trigger set construction method based on decision boundaries, which not only protects the model but also improves the performance of the model, but this method has the problem of easily causing false positives; since the trigger set constructed by the previous method is not closely related to the original training set, it is easy to erase the watermark by retraining and fine-tuning, Li et al. A custom filter is proposed to make modifications on the original training set. The embedding process can only be performed in the initial training of the model, which also makes this method able to resist attacks that delete watermarks through fine-tuning or incremental training. However, this method has problems such as embedding watermarks requires the model to be trained from scratch and the size of the watermark is limited. In the above black-box methods, watermarks are embedded by setting trigger sets with specified labels. However, these behaviors will affect the decision boundary of the original model on the original task. Zhong et al. proposed a new black-box watermarking framework, which embeds watermarks in the model by adding new labels to the classification task of the model. However, this method has the risk of easily detecting the watermark, so an invisible watermark embedding / verification protocol is needed to resist fraud attacks. Summary of the invention

[0006] Based on this, the purpose of the present invention is to provide a neural network watermark embedding method, device, electronic device and storage medium, which generate a trigger set and give it an original label while keeping the image features unchanged, so as to protect the ownership of the model without affecting the decision boundary of the original classification task, and maintain the performance accuracy almost the same as the original model while maintaining the owner's intellectual property rights.

[0007] In a first aspect, the present invention provides a neural network watermark embedding method, comprising the following steps:

[0008] Obtain a key K, wherein the key K corresponds to a unique timestamp T;

[0009] Randomly select n pictures (P1, P2, ..., P n ), where n is greater than 1, and the original training set includes multiple pictures and labels corresponding to each picture;

[0010] The chaotic sequence generated by the key K is used to generate the pictures (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n), the n pictures (Pe1, Pe2, ..., Pe n ) and their corresponding labels (L1, L2, ..., L n ) into the trigger set;

[0011] The original training set and the trigger set are put into the neural network to be embedded with watermark for training, so that the neural network to be embedded with watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, thereby obtaining the neural network embedded with watermark.

[0012] Furthermore, when verifying the ownership of the neural network, the method further comprises:

[0013] Get the n pictures (P1, P2, ..., P n ) and the corresponding labels (L1, L2, ..., L n ) and the key K;

[0014] The chaotic sequence generated by the key K is used to generate the pictures (P1, P2, ..., P n ) is scrambled and encrypted to obtain the trigger set images (Pe1, Pe2, ..., Pe n );

[0015] The trigger set image is input into the neural network to be verified, and the labels (R1, R2, ..., R n );

[0016] By comparing the labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ), and obtain the verification result of the neural network.

[0017] Furthermore, the chaotic sequence generated by the key K is used to generate the images (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n ),include:

[0018] For each picture P i , P i Divided into multiple M×M sub-blocks {B (1,1) , …, B (w / M,h / M)}; where w and h represent the width and height of the image respectively; M is a preset value;

[0019] The first 4 digits of the key K are divided by 10000 to get k1 and the last 4 digits are divided by 10000 to get k2, which are respectively input as x0 into the following logistic chaotic mapping formula to generate pseudo-random sequences Q1 and Q2:

[0020] x n+1 =λ*x n *(1-x n )

[0021] Q1={x0,x1,…,x n}, where x0=k1

[0022] Q2={x0,x1,…,x n}, where x0 = k2

[0023] where λ∈[0,4], x n ∈[0, 1]; n is the number of iterations;

[0024] Sort the random sequences Q1 and Q2 to generate integer scrambled sequences O1 and O2;

[0025] Use sequence O1 to scramble each block, and use sequence O2 to scramble the pixel values ​​in each block to obtain the trigger set image Pe i .

[0026] Furthermore, λ=4.

[0027] Furthermore, the random sequences Q1 and Q2 are sorted to generate integer scrambled sequences O1 and O2, including:

[0028] Using the following formula, we can get the integer scrambled sequences O1 and O2:

[0029] O1={r1,r2,…,r n+1}

[0030] Among them, r i is x i Ranking value in Q1 sequence; r i is a positive integer;

[0031] O2={r1,r2,…,r n+1}

[0032] Among them, r i is x i Ranking value in Q2 sequence; r i Is a positive integer.

[0033] Furthermore, the sequence O1 is used to scramble each block, and the sequence O2 is used to scramble the pixel values ​​in each block to obtain the trigger set image Pe i ,include:

[0034] For each picture P i , for all sub-blocks {B (1,1) , …, B (w / M,h / M)} are sorted to obtain the number of each sub-block {1, 2, ..., n+1}; where n is equal to n in sequence O1;

[0035] According to the sequence O1 = {r1, r2, ..., r n+1}, filling each sub-block into the position of the sub-block number in the sequence O1, and rearranging the order of each sub-block;

[0036] For each sub-block, all pixels in it are sorted to obtain the number of each pixel {1, 2, ..., n+1}; where n is equal to n in sequence O2;

[0037] According to the sequence O2 = {r1, r2, ..., r n+1}, fill each pixel point into the position of the pixel point number in the sequence O2, and rearrange the order of each pixel point.

[0038] Furthermore, by comparing the labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ), and obtain the verification result of the neural network, including:

[0039] The labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ) consistency:

[0040] V(t={f(P k ), f(Pe k )} n k=1,τ)={True,False}

[0041] Among them, V() is the consistency function of verifying the output label and the true label, f() is the label output after the network model processes the image, and P k and Pe k represent the original training set and the trigger set respectively; k is the number of rounds of current verification; t represents f(P k ) and f(Pe k ) is the ratio of the number of identical results in the output labels to the total number; τ is the preset verification threshold;

[0042] When t≥τ is calculated, the verification result of the neural network is that the ownership is verified.

[0043] In a second aspect, the present invention further provides a neural network watermark embedding device, comprising:

[0044] A key acquisition module, used to acquire a key K, wherein the key K corresponds to a unique timestamp T;

[0045] The original image selection module is used to randomly select n images (P1, P2, ..., P n ), where n is greater than 1, and the original training set includes multiple pictures and labels corresponding to each picture;

[0046] The trigger set acquisition module is used to generate a chaotic sequence for the image (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n ), the n pictures (Pe1, Pe2, ..., Pe n ) and their corresponding labels (L1, L2, ..., L n ) into the trigger set;

[0047] The watermark embedding module is used to put the original training set and the trigger set into the neural network to be embedded with watermark for training, so that the neural network to be embedded with watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, thereby obtaining the neural network embedded with watermark.

[0048] In a third aspect, the present invention further provides an electronic device, characterized in that it includes:

[0049] at least one memory and at least one processor;

[0050] The memory is used to store one or more programs;

[0051] When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of a neural network watermark embedding method as described in any one of the first aspects of the present invention.

[0052] In a fourth aspect, the present invention further provides a computer-readable storage medium, characterized in that:

[0053] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a neural network watermark embedding method as described in any one of the first aspects of the present invention are implemented.

[0054] The present invention provides a neural network watermark embedding method, device, electronic device and storage medium. Through a scrambling method that retains image features, the original label is assigned to the image after scrambling to generate a watermark trigger set, so that the model can learn the classification task of the trigger set well under the premise of ensuring that the decision boundary of the original classification task is not distorted. In response to the black box counterfeiting attack in neural network copyright protection, a unique watermark protocol method is proposed, that is, when the attacker obtains the owner's neural network, he wants to forge a watermark belonging to himself through the same watermark construction method. When he applies for a key to forge a watermark from a third-party copyright certification company, the application time will be recorded, so that even if the forged watermark is successfully embedded, the authenticity of the watermark can be verified by comparing the application time, thereby ensuring that the watermark embedded in this way can resist the black box counterfeiting attack.

[0055] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of a process flow of a neural network watermark embedding method provided by the present invention;

[0057] Figure 2 A schematic diagram of a watermark embedding and watermark verification process in one embodiment of the present invention;

[0058] Figure 3 A schematic diagram of a process of constructing a trigger set in one embodiment of the present invention;

[0059] Figure 4 A schematic diagram of the structure of a neural network watermark embedding device provided by the present invention. DETAILED DESCRIPTION

[0060] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0061] It should be clear that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present application.

[0062] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0063] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0064] In addition, in the description of this application, unless otherwise specified, "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0065] In response to the problems in the background technology, the embodiments of the present application provide a neural network watermark embedding method. For the neural network intellectual property to be protected, we propose a new watermark protocol based on logistic chaotic mapping. The key is distributed through a third-party certification center. Two pairs of scrambled sequences are generated by the key to process the image to generate a trigger set. The original training set and the trigger set are put into the network for training together, so that the network can learn the classification task and the watermark task at the same time, that is, the watermark embedding process is completed.

[0066] like Figure 1 As shown, the method comprises the following steps:

[0067] S01: Obtain a key K, where the key K corresponds to a unique timestamp T.

[0068] In a specific embodiment, the key K is a string of random numbers of at least 8 digits.

[0069] Timestamp is data generated using digital signature technology. The object of the signature includes information such as the original file information, signature parameters, and signature time. The timestamp system is used to generate and manage timestamps. It digitally signs the signature object to generate a timestamp to prove that the original file existed before the signature time.

[0070] Preferably, the timestamp T includes the time information of applying for the key from the third-party copyright certification company.

[0071] S02: Randomly select n pictures (P1, P2, ..., P n ), where n is greater than 1, and the original training set includes multiple pictures and a label corresponding to each picture.

[0072] In a specific embodiment, the selected n original images will be saved by the copyright owner of the neural network and used to generate trigger set images when embedding watermarks and subsequently verifying the ownership of the neural network.

[0073] S03: The chaotic sequence generated by the key K is used to generate the image (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n ), the n pictures (Pe1, Pe2, ..., Pe n ) and their corresponding labels (L1, L2, ..., L n ) into the trigger set.

[0074] S04: Put the original training set and the trigger set together into the neural network to be watermarked for training, so that the neural network to be watermarked learns the mapping between the trigger set and the corresponding label while learning the original classification task, thereby obtaining the neural network embedded with watermark.

[0075] The present invention provides a neural network watermark embedding method, which generates a trigger set and assigns it an original label while keeping the image features unchanged, so as to protect the ownership of the model without affecting the decision boundary of the original classification task, and maintain the intellectual property rights of the owner while maintaining the performance accuracy almost the same as the original model.

[0076] The neural network embedded with watermark obtained in this way is as follows: Figure 2 As shown in the figure, in the watermark verification stage, the image to be processed (the trigger set image before generation) is submitted to a third-party authoritative authentication center. The authoritative center processes the image according to the corresponding key stored in the database and authenticates it without disclosure to prevent attackers from making more targeted attacks. Finally, when the network label hit rate reaches the threshold, the ownership verification is successful, otherwise, the verification fails.

[0077] In a preferred embodiment, when verifying the ownership of the neural network, the method further comprises:

[0078] S05: Obtain the n pictures (P1, P2, ..., P n ) and the corresponding labels (L1, L2, ..., L n ) and the key K.

[0079] In a specific embodiment, when a copyright dispute occurs, the owner can extract n pictures (P1, P2, ..., P n ) and its original labels (L1, L2, …, L n ) and submit it to a third-party authoritative center for certification.

[0080] S06: The chaotic sequence generated by the key K is used to generate the image (P1, P2, ..., P n ) is scrambled and encrypted to obtain the trigger set images (Pe1, Pe2, ..., Pe n ).

[0081] In a specific embodiment, the third-party authoritative center uses the key K retained in the database to combine the pictures (P1, P2, ..., P n ) to obtain the trigger set (Pe1, Pe2, ..., Pe n ).

[0082] S07: Input the trigger set image into the neural network to be verified, and obtain the labels (R1, R2, ..., R n ).

[0083] S08: By comparing the labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ), and obtain the verification result of the neural network.

[0084] In a preferred embodiment, the following formula is used to calculate the labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ) consistency:

[0085] V(t={f(P k ), f(Pe k )} n k=1,τ)={True,False}

[0086] Among them, V() is the consistency function of verifying the output label and the true label, f() is the label output after the network model processes the image, and P kand Pe k represent the original training set and the trigger set respectively; k is the number of rounds of current verification; t represents f(P k ) and f(Pe k ) is the ratio of the number of identical results in the output labels to the total number; τ is the preset verification threshold.

[0087] In a specific embodiment, the threshold can be set to τ=0.9, which is nine times the random hit rate (0.1) in a normal 10-class task, which is sufficient to prove network ownership. Therefore, to prove its ownership of the network model, its accuracy needs to be greater than or equal to 90%.

[0088] When t≥τ is calculated, the verification result of the neural network is that the ownership is verified.

[0089] With respect to black-box counterfeiting attacks in neural network copyright protection, due to the uniqueness of the watermark protocol method proposed in the present invention, that is, when the attacker obtains the owner's neural network, he wants to forge a watermark of his own through the same watermark construction method. When he applies for a key to forge a watermark from a third-party copyright certification company, the application time will be recorded. Even if the forged watermark is successfully embedded, the authenticity of the watermark can be verified by comparing the application time, thereby ensuring that the watermark embedded in this way can resist black-box counterfeiting attacks.

[0090] In a preferred embodiment, Figure 3 As shown in the figure, the chaotic sequence generated by the key K has an effect on the pictures (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n ), including the following sub-steps:

[0091] S031: For each image P i , P i Divided into multiple M×M sub-blocks {B (1,1) , …, B (w / M,h / M)}; where w and h represent the width and height of the image respectively; M is a preset value.

[0092] S032: Divide the first 4 digits of the key K by 10000 to obtain k1 and the last 4 digits by 10000 to obtain k2, and input them into the following logistic chaotic mapping formula as x0 to generate pseudo-random sequences Q1 and Q2:

[0093] x n+1 =λ*x n *(1-x n )

[0094] Q1={x0,x1,…,x n}, where x0 = k1

[0095] Q2={x0,x1,…,x n}, where x0 = k2

[0096] where λ∈[0,4], x n ∈[0, 1]; n is the number of iterations.

[0097] Preferably, λ=4.

[0098] In an illustrative example, the key K = 21936339, then the input x0 of the sequence Q1 = k1 = 0.2193, and the input x0 of the sequence Q2 = k2 = 0.6399. After 3 iterations, the obtained sequence Q1 = {0.2193, 0.6848, 0.8634, 0.4718}, and the sequence Q2 = {0.6399, 0.9217, 0.2886, 0.8213}.

[0099] S033: Sort the random sequences Q1 and Q2 to generate integer scrambled sequences O1 and O2.

[0100] Preferably, the integer scrambled sequences O1, O2 are obtained using the following formula:

[0101] O1={r1,r2,…,r n+1}

[0102] Among them, r i is x i Ranking value in Q1 sequence; r i is a positive integer;

[0103] O2={r1,r2,…,r n+1}

[0104] Among them, r i is x i Ranking value in Q2 sequence; r i Is a positive integer.

[0105] In an illustrative example, for the above sequence Q1={0.2193, 0.6848, 0.8634, 0.4718}, sequence Q2={0.6399, 0.9217, 0.2886, 0.8213}, it can be calculated that O1={1, 3, 4, 2}, O2={2, 4, 1, 3}.

[0106] S034: Use sequence O1 to scramble each block, and use sequence O2 to scramble the pixel values ​​in each block to obtain the trigger set image Pe i .

[0107] In a specific embodiment, the sub-step of scrambling each block using the sequence O1 includes:

[0108] S0341: For each image P i , for all sub-blocks {B (1,1) , …, B (w / M,h / M)} are sorted to obtain the number of each sub-block {1, 2, ..., n+1}; where n is equal to n in sequence O1;

[0109] S0342: According to the sequence O1 = {r1, r2, ..., r n+1}, fill each sub-block into the position of the sub-block number in the sequence O1, and rearrange the order of each sub-block.

[0110] After rearranging the order of the sub-blocks, a scrambled sub-block image is obtained.

[0111] In a specific embodiment, the step of scrambling the pixel values ​​in each block using the sequence O2 includes:

[0112] S0343: For each sub-block, sort all the pixels therein to obtain a number {1, 2, ..., n+1} for each pixel; wherein n is equal to n in sequence O2.

[0113] S0344: According to the sequence O2 = {r1, r2, ..., r n+1}, fill each pixel point into the position of the pixel point number in the sequence O2, and rearrange the order of each pixel point.

[0114] After rearranging the order of pixel values, the scrambled image is obtained, namely the trigger set image.

[0115] Since most current black-box watermarking methods construct trigger sets by adding backdoor attacks such as noise and signatures to the original image and then assigning them wrong labels, or assigning custom labels to abstract images, these trigger sets will affect the decision boundary of the original model when they are put into network training. The present invention uses a scrambling method that retains image features, assigning the original labels to the images after scrambling to generate watermark trigger sets, which ensures that the decision boundary of the original classification task will not be distorted, so that the model can learn the classification task of the trigger set well.

[0116] The present application also provides a neural network watermark embedding device, such as Figure 4 As shown, the neural network watermark embedding device 400 includes:

[0117] A key acquisition module 401 is used to acquire a key K, where the key K corresponds to a unique timestamp T;

[0118] The original picture selection module 402 is used to randomly select n pictures (P1, P2, ..., P n ), where n is greater than 1, and the original training set includes multiple pictures and labels corresponding to each picture;

[0119] The trigger set acquisition module 403 is used to generate a chaotic sequence of pictures (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n ), the n pictures (Pe1, Pe2, ..., Pe n ) and their corresponding labels (L1, L2, ..., L n ) into the trigger set;

[0120] The watermark embedding module 404 is used to put the original training set and the trigger set into the neural network to be embedded with watermark for training, so that the neural network to be embedded with watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, thereby obtaining the neural network embedded with watermark.

[0121] Preferably, when verifying the ownership of the neural network, the device further comprises:

[0122] The original data acquisition module is used to acquire the n pictures (P1, P2, ..., P n ) and the corresponding labels (L1, L2, ..., L n ) and the key K;

[0123] The trigger set generation module is used to generate a chaotic sequence for the image (P1, P2, ..., P n ) is scrambled and encrypted to obtain the trigger set images (Pe1, Pe2, ..., Pe n );

[0124] The label acquisition module is used to input the trigger set image into the neural network to be verified, and obtain the label (R1, R2, ..., R n );

[0125] A verification module is used to compare the labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ), and obtain the verification result of the neural network.

[0126] Preferably, the trigger set acquisition module includes:

[0127] The image division unit is used to divide each image P into i , Pi Divided into multiple M×M sub-blocks {B (1,1) , …, B (w / M,h / M)}; where w and h represent the width and height of the image respectively; M is a preset value;

[0128] The pseudo-random sequence generation unit is used to divide the first 4 digits of the key K by 10000 to obtain k1 and the last 4 digits by 10000 to obtain k2, and input the following logistic chaotic mapping formula as x0 to generate pseudo-random sequences Q1 and Q2:

[0129] x n+1 =λ*x n *(1-x n )

[0130] Q1={x0,x1,…,x n}, where x0 = k1

[0131] Q2={x0,x1,…,x n}, where x0 = k2

[0132] where λ∈[0,4], x n ∈[0, 1]; n is the number of iterations;

[0133] A scrambled sequence generation unit, used for sorting random sequences Q1, Q2 to generate integer scrambled sequences O1, O2;

[0134] The scrambling unit is used to scramble each block using sequence O1 and scramble the pixel values ​​in each block using sequence O2 to obtain the trigger set image Pe i .

[0135] Preferably, λ=4.

[0136] Preferably, the scrambled sequence generation unit uses the following formula to obtain integer scrambled sequences O1, O2:

[0137] O1={r1,r2,…,r n+1}

[0138] Among them, r i is x i Ranking value in Q1 sequence; r i is a positive integer;

[0139] O2={r1,r2,…,r n+1}

[0140] Among them, r i is x i Ranking value in Q2 sequence; r i Is a positive integer.

[0141] Preferably, the scrambling unit comprises:

[0142] Sub-block sorting element, used for each picture P i , for all sub-blocks {B (1,1) , …, B (w / M,h / M)} are sorted to obtain the number of each sub-block {1, 2, ..., n+1}; where n is equal to n in sequence O1;

[0143] The sub-block scrambling element is used to scramble the sub-blocks according to the sequence O1 = {r1, r2, ..., r n+1}, filling each sub-block into the position of the sub-block number in the sequence O1, and rearranging the order of each sub-block;

[0144] A pixel value sorting component is used to sort all the pixels in each sub-block to obtain a number {1, 2, ..., n+1} for each pixel; wherein n is equal to n in the sequence O2;

[0145] The pixel value scrambling element is used to scramble the pixels according to the sequence O2 = {r1, r2, ..., r n+1}, fill each pixel point into the position of the pixel point number in the sequence O2, and rearrange the order of each pixel point.

[0146] Preferably, the verification module includes:

[0147] A consistency calculation unit is used to calculate the labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ) consistency:

[0148] V(t={f(P k ), f(Pe k )} n k=1,τ)={True,False}

[0149] Among them, V() is the consistency function of verifying the output label and the true label, f() is the label output after the network model processes the image, and P k and Pe k represent the original training set and the trigger set respectively; k is the number of rounds of current verification; t represents f(P k ) and f(Pe k ) is the ratio of the number of identical results in the output labels to the total number; τ is the preset verification threshold;

[0150] When t≥τ is calculated, the verification result of the neural network is that the ownership is verified.

[0151] The present application also provides an electronic device, including:

[0152] at least one memory and at least one processor;

[0153] The memory is used to store one or more programs;

[0154] When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of a neural network watermark embedding method as described above.

[0155] The present application also provides a computer-readable storage medium.

[0156] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a neural network watermark embedding method as described above are implemented.

[0157] The present invention provides a neural network watermark embedding method, device, electronic device and storage medium. In a specific embodiment, the present invention provides a neural network watermark embedding method, which will introduce a third-party authoritative center. The scrambling key used will be issued by the third-party authoritative center. The authoritative center only needs to store the key without storing the entire trigger set, thereby greatly reducing the pressure of storing large amounts of data. The verification process is also independently performed by the third-party authoritative center, so that the owner's ownership can be verified without the need to disclose the watermark, and more purposeful attacks by attackers are avoided. When the copyright owner applies for the encryption key, the third-party authoritative center will generate a timestamp to record the application time. Therefore, the algorithm can resist forgery attacks and is more robust.

[0158] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A neural network watermark embedding method, characterized in that: The following steps are involved: Obtain a key K, wherein the key K corresponds to a unique timestamp T; Randomly select n pictures (P1, P2, ..., P n ), where n is greater than 1, and the original training set includes multiple pictures and labels corresponding to each picture; The chaotic sequence generated by the key K is used to generate the pictures (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n ), the n pictures (Pe1, Pe2, ..., Pe n ) and their corresponding labels (L1, L2, ..., L n ) into the trigger set; Among them, for each trigger set picture Pe i ,include: For each picture P i , for all sub-blocks {B (1,1) , …, B (w / M,h / M) } to obtain the number of each sub-block {1, 2, ..., n+1}; wherein n is equal to the number of iterations n in the sequence o1; the sequence o1 = {r1, r2, ..., r n+1 } is the chaotic sequence; w and h represent the image P respectively. i The width and height of ; M is the preset value; According to the sequence O1 = {r1, r2, ..., r n+1 }, filling each sub-block into the position of the sub-block number in the sequence O1, and rearranging the order of each sub-block; For each sub-block, all pixels in it are sorted to obtain the number of each pixel {1, 2, ..., n+1}; where n is equal to the number of iterations n in the sequence O2, and the sequence O2 = {r1, r2, ..., r n+1 } is the chaotic sequence; According to the sequence O2 = {r1, r2, ..., r n+1 }, fill each pixel point with the position of the pixel point number in the sequence O2, rearrange the order of each pixel point, and obtain the trigger set image Pe i ; Putting the original training set and the trigger set together into the neural network to be embedded with watermark for training, so that the neural network to be embedded with watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, thereby obtaining the neural network embedded with watermark; Wherein, when verifying the ownership of the neural network embedded with the watermark, the method includes: Authenticate the trigger set image before generation with the key K: When the label hit rate of the neural network embedded with the watermark reaches a threshold, the ownership verification is successful, otherwise, the verification fails.

2. A neural network watermark embedding method according to claim 1, characterized in that: When verifying ownership of the neural network, the method further comprises: Get the n pictures (P1, P2, ..., P n ) and the corresponding labels (L1, L2, ..., L n ) and the key K; The chaotic sequence generated by the key K is used to generate the pictures (P1, P2, ..., P n ) is scrambled and encrypted to obtain the trigger set images (Pe1, Pe2, ..., Pe n ); The trigger set image is input into the neural network to be verified, and the labels (R1, R2, ..., R n ); By comparing the labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ), and obtain the verification result of the neural network.

3. A neural network watermark embedding method according to claim 1, characterized in that: The chaotic sequence generated by the key K is used to generate the pictures (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n ),include: For each picture P i , P i Divided into multiple M×M sub-blocks {B (1,1) , …, B (w / M,h / M) }; where w and h represent the width and height of the image respectively; M is a preset value; The first 4 digits of the key K are divided by 10000 to get k1 and the last 4 digits are divided by 10000 to get k2, which are respectively input as x0 into the following logistic chaotic mapping formula to generate pseudo-random sequences Q1 and Q2: x n+1 =λ*x n *(1-x n ) Q1={x0,x1,…,x n }, where x0 = k1 Q2={x0,x1,…,x n }, where x0 = k2 where λ∈[0,4], x n ∈[0, 1]; n is the number of iterations; Sort the random sequences Q1 and Q2 to generate integer scrambled sequences O1 and O2; Use sequence O1 to scramble each block, and use sequence O2 to scramble the pixel values ​​in each block to obtain the trigger set image Pe i .

4. A neural network watermark embedding method according to claim 3, characterized in that: λ=4.

5. A neural network watermark embedding method according to claim 3, characterized in that: Sort the random sequences Q1 and Q2 to generate integer scrambled sequences O1 and O2. include: Using the following formula, we can get the integer scrambled sequences O1 and O2: O1＝{r1,r2,…,r n+1} Among them, r i is x i Ranking value in Q1 sequence; r i is a positive integer; O2＝{r1,r2,…,r n+1} Among them, r i is x i Ranking value in Q2 sequence; r i Is a positive integer.

6. A neural network watermark embedding method according to claim 2, characterized in that: By comparing the labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ), and obtain the verification result of the neural network, including: The labels (R1, R2, ..., R n ) and the original labels (L1, L2, ..., L n ) consistency: V(t={f(P k ),f(Pe k )} n k=1, τ)={True, False} Among them, V() is the consistency function of verifying the output label and the true label, f() is the label output by the network model after processing the image, and P k and Pe k represent the original training set and the trigger set respectively; k is the number of rounds of current verification; t represents f(P k ) and f(Pe k ) is the ratio of the number of identical results in the output labels to the total number; τ is the preset verification threshold; When t≥τ is calculated, the verification result of the neural network is that the ownership is verified.

7. A neural network watermark embedding device, characterized in that: include: A key acquisition module, used to acquire a key K, wherein the key K corresponds to a unique timestamp T; The original image selection module is used to randomly select n images (P1, P2, ..., P n ), where n is greater than 1; A key acquisition module, used to acquire a key K, wherein the key K corresponds to a unique timestamp T; The original image selection module is used to randomly select n images 9P1, P2, ..., P in the original training set n ), where n is greater than 1, and the original training set includes multiple pictures and labels corresponding to each picture; The trigger set acquisition module is used to generate a chaotic sequence for the image (P1, P2, ..., P n ) are scrambled and encrypted to obtain n trigger set images (Pe1, Pe2, ..., Pe n ), the n pictures (Pe1, Pe2, ..., Pe n ) and their corresponding labels (L1, L2, ..., L n ) into the trigger set; Among them, for each trigger set picture Pe i ,include: For each picture P i , for all sub-blocks {B (1,1) , …, B (w / M,h / M) } to obtain the number of each sub-block {1, 2, ..., n+1}; wherein n is equal to the number of iterations n in the sequence o1, and the sequence o1 = {r1, r2, ..., r n+1 } is the chaotic sequence; w and h represent the image P respectively. i The width and height of ; M is the preset value; According to the sequence O1 = {r1, r2, ..., r n+1 }, filling each sub-block into the position of the sub-block number in the sequence O1, and rearranging the order of each sub-block; For each sub-block, all pixels in it are sorted to obtain the number of each pixel {1, 2, ..., n+1}; where n is equal to the number of iterations n in the sequence O2, and the sequence O2 = {r1, r2, ..., r n+1 } is the chaotic sequence; According to the sequence O2 = {r1, r2, ..., r n+1 }, fill each pixel point with the position of the pixel point number in the sequence O2, rearrange the order of each pixel point, and obtain the trigger set image Pe i ; A watermark embedding module, used for putting the original training set and the trigger set into the neural network to be embedded with watermark for training, so that the neural network to be embedded with watermark learns the mapping between the trigger set and the corresponding label while learning the original classification task, thereby obtaining the neural network embedded with watermark; Wherein, when verifying the ownership of the neural network embedded with the watermark, the method includes: Authenticate the trigger set image before generation with the key K: When the label hit rate of the neural network embedded with the watermark reaches a threshold, the ownership verification is successful, otherwise, the verification fails.

8. An electronic device, characterized in that: include: at least one memory and at least one processor; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of a neural network watermark embedding method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a neural network watermark embedding method as described in any one of claims 1-6 are implemented.

Images