Model data ownership protection and verification method based on model coupling

Abstract

The invention relates to a model data ownership protection and verification method based on model coupling, and belongs to the field of deep neural network ownership protection. The method comprises the following steps that: 1) a model owner trains a target model and a license model in parallel; 2) the model owner uses a model coupling algorithm to merge the target model and the license model; 3) releasing a coupling model by a model owner; and 4) the model owner performs ownership verification. According to the method, an existing model copyright verification method is supplemented for the right confirmation requirement of the deep neural network, and the problem of data ownership protection is solved while the efficiency and the overhead are considered.

Description

Technical Field

[0001] The present invention belongs to the field of deep neural network ownership protection, and in particular to a model data ownership protection and verification method based on model coupling. Background Art

[0002] Deep neural networks (DNNs) have been widely used in many cutting-edge fields, such as face recognition, natural language processing, and autonomous driving. Since training a high-performance deep neural network model consumes a lot of resources and has become an increasingly profitable business, attackers may try various attacks to steal or extract these high-value models. Therefore, when releasing industrial-grade DNN models, one of the owners' biggest concerns is copyright protection.

[0003] Model fingerprinting and model watermarking are two approaches to protecting ownership, but both have limitations when used alone. Fingerprinting leverages the inherent uniqueness of a model to verify ownership, such as its unique decision boundary. However, exploring this uniqueness is not only time-consuming, but has also been shown to be easily circumvented through model fine-tuning or sample preprocessing. Another more robust approach is model watermarking, which signals ownership by leveraging artificially constructed uniqueness, typically through modified parameters in white-box watermarking or specific backdoors in black-box watermarking.

[0004] White-box watermarking requires access to the parameters of the suspicious model when verifying ownership, which is usually impractical in real-world scenarios. Although most black-box watermarking methods do not require access to the internal structure of the model, they usually perform tedious fine-tuning on the protected model. Our invention can overcome the shortcomings of the above-mentioned previous ownership protection methods and achieve lightweight, efficient, and low-loss data ownership verification by combining the technical features of model fingerprints and model watermarks.

[0005] For the convenience of describing the present invention, some related technologies related to the present invention are introduced below.

[0006] (1) Model fingerprint:

[0007] Model fingerprinting focuses on generating special samples to capture the decision boundary of the model. Fingerprints are generated in large quantities near the decision boundary of the model through projected gradient descent (PGD) and greedy algorithms or noise addition. However, in ownership protection, even with relatively robust fingerprinting methods, it still fails when facing attackers who preprocess inputs or even compress models to obscure ownership during black-box query. Fragile fingerprints can effectively verify model integrity against attackers' model integrity violations due to their easy-to-destruct nature and the uniqueness generated during verification.

[0008] (2) Model watermark:

[0009] The model owner can trigger the backdoor in the proxy model to claim ownership. Existing watermarking methods can be divided into two categories according to their strategies: white-box watermarking and black-box watermarking.

[0010] White-box watermarking injects weight parameters or special signatures into the model before it is distributed or deployed, so that this information can be extracted for ownership verification when ownership disputes occur. However, these methods involve the internal parameters of the model and are not practical in real-world applications. Black-box watermarking overcomes the practical problems of white-box watermarking. Some studies have proposed using black and white squares as triggers for watermark embedding, but this method is not sufficiently concealed. Another study proposed using sample-specific perturbations as triggers, but in practice these methods are still difficult to evade the attacker's perception and involve a lot of computational consumption. Summary of the invention

[0011] In response to the demand for deep neural network ownership confirmation, the present invention provides a model data ownership protection and verification method based on model coupling, which solves the shortcomings of the above-mentioned model fingerprint and model black box in practical applications.

[0012] To achieve the above purpose, the technical solution of the present invention is: a model data ownership protection and verification method based on model coupling, comprising:

[0013] (1) Model developer O creates the original model training dataset Select the model architecture S;

[0014] (2) Model developer O inputs random data x into watermark generator G, generating D g ;

[0015] (3) Model developer O divides the training dataset D into N t 、N s are the number of data in the two data sets, D t middle y i ∈[0, ..., Kn-1], where K represents the total number of data classes in D, n represents the number of data classes in the split data set, and D s middle y i ∈[0, ..., n-1], where the smaller data set D is split s With the trigger added to the dataset D g Synthetically generate a new data set Dp;

[0016] (4) Model developer O trains the target model y in parallel t With the ownership license model OL:

[0017]

[0018] The former represents the optimization process of the target model, and the latter represents the optimization process of the ownership licensing model OL. represents the loss function, f t Represents y t The initial model function to be optimized, f p Represents the initial model function of OL to be optimized, N p D p The amount of data in

[0019] (5) Model developer O uses model coupling algorithm A to process target model y t and the ownership licensing model OL: the prediction output of the ownership licensing model OL is the predicted value Multiply by weight α and assign to y t The other classes remain unchanged:

[0020]

[0021] where y d is the final output result of the ownership licensing model OL, λ is a hyperparameter, for y t Output results in different dimensions; use mask β to record the output of OL in the watermark trigger dimension, and the final output y of model F passes through the priority function P:

[0022]

[0023] Finally, a coupling model F is generated that can protect and verify data ownership;

[0024] (6) Model developer O distributes the coupled model F to model user U and retains the model ownership and model watermark related knowledge locally; when verifying the model ownership, model developer O adds trigger T to the randomly selected test sample for random data, generates a watermark test sample W, and compares W with D s The fingerprint samples B are input into the suspect model, i.e. the generated coupling model F, in sequence, and the output results are collected.

[0025] In one embodiment of the present invention, in step (6), W and D s The fingerprint samples B are input into the suspect model in sequence, and the specific implementation process of collecting the output results is as follows:

[0026] (6-1) If the output result is consistent with the content displayed by the watermark, the suspected model is no longer suspected and becomes an independent training model;

[0027] (6-2) If W successfully triggers the target watermark preset by the model developer O during verification, that is, the output result shows a predetermined deviation, the ownership of the model will be charged;

[0028] (6-3) When W verifies, the model predicts normally, but the output result of B verification is different from the expected output result of the model developer O, indicating that the attacker has attacked the illegally obtained model, and the model developer O accuses the model of ownership.

[0029] In one embodiment of the present invention, the process of step (1) is as follows: the model developer O privately owns the original data set where x i ∈R d , with K types of data, denoted as y i ∈[0, ..., K-1]; model developer O has white-box access to model training and selects model architecture S according to requirements to perform model training.

[0030] In one embodiment of the present invention, the process of step (2) is as follows: Model developer O presets an empty data set Input random data x that has no intersection with D into the watermark generator G to generate D g :D g ←G(x).

[0031] In one embodiment of the present invention, random data x is input into a watermark generator G to generate D g The watermark generation method relies on poisoning methods including BadNets or Sleeper agent.

[0032] In one embodiment of the present invention, the ownership permission model OL and the target model y t The coupling makes the watermark embedded in the ownership licensing model OL a triggerable backdoor of the coupling model F, and the triggering of the backdoor is used as a means to verify the ownership of F.

[0033] In one embodiment of the present invention, the process of step (3) is as follows: the model developer O divides the training data set into Then D s With D g Synthesize, generate D p :D p ←D s ∪D g , D p middle y i ∈[0,...,n].

[0034] The present invention also provides a model data ownership protection and verification system based on model coupling, which executes the method steps as described above, and the system includes:

[0035] The first module is to create the original training data set of the model Select the model architecture S;

[0036] The second module inputs random data x into the watermark generator G to generate D g ;

[0037] The third module divides the training data set D into N t 、N s are the number of data in the two data sets, respectively, where K represents the total number of data classes in D, n represents the number of data classes in the split data set, and D t middle y i ∈[0,...,Kn-1],D s middle y i ∈[0, ..., n-1], where the smaller data set D is split s With the trigger added to the dataset D g Synthetically generate a new data set Dp;

[0038] The fourth module is to train the target model y in parallel t With the ownership license model OL:

[0039]

[0040] The former represents the optimization process of the target model, and the latter represents the optimization process of the ownership licensing model OL. represents the loss function, ft represents y t The initial model function to be optimized, f p Represents the initial model function of OL to be optimized, N p D p The amount of data in

[0041] The fifth module uses the model coupling algorithm A to process the target model y t and the ownership licensing model OL: the prediction output of the ownership licensing model OL is the predicted value Multiply by weight α and assign to y t The other classes remain unchanged:

[0042]

[0043] Where yd is the final output result of the ownership licensing model OL, λ is a hyperparameter, for y t Output results in different dimensions; use mask β to record the output of OL in the watermark trigger dimension, and the final output y of model F passes through the priority function P:

[0044]

[0045] Finally, a coupling model F is generated that can protect and verify data ownership;

[0046] The sixth module distributes the coupled model F to the model user U and retains the model ownership and model watermark related knowledge locally; when verifying the model ownership, the model developer O adds a trigger T to the randomly selected test sample for random data, generates a watermark test sample W, and compares W with D s The fingerprint samples B are input into the suspect model, i.e. the generated coupling model F, in sequence, and the output results are collected.

[0047] The present invention also provides a computer system, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0048] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0049] Compared with the prior art, the present invention has the following beneficial effects: This method solves the basic conflict in model ownership protection. It is a lightweight solution that can continuously protect model ownership without sacrificing model performance, and only requires moderate time and computing resource overhead. Its main advantage is that it proposes a two-layer model ownership embedding and verification framework, which includes an independent ownership model and an algorithm to entangle it with the protected DNN model, combining the advantages of model fingerprint recognition and watermarking technology, while achieving ownership protection and ensuring the balance between model performance and resource overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention provides a flowchart of the data ownership protection and verification method for model coupling in the method of the present invention.

[0051] Figure 2 This is a schematic diagram of the operation of the data ownership protection and verification method coupled with the model in the method of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0053] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0054] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0055] The present invention provides a model data ownership protection and verification method based on model coupling, comprising:

[0056] (1) Model developer O creates the original model training dataset Select model architecture S; : Model developer O private original dataset where x i ∈R d , with K types of data, denoted as y i ∈[0, ..., K-1]; model developer O has white-box permission for model training and selects model architecture S according to requirements to perform model training;

[0057] (2) Model developer O inputs random data x into watermark generator G, generating D g ; Model developer O presets an empty dataset Input random data x that has no intersection with D into the watermark generator G to generate D g :D g ←G(x); generate D g The watermark generation method uses poisoning methods such as BadNets or Sleeper agent; the ownership licensing model OL and the target model y t The coupling makes the watermark embedded in the ownership licensing model OL become a triggerable backdoor of the coupling model F, and the triggering of the backdoor is used as a means to verify the ownership of F;

[0058] (3) Model developer O divides the training dataset D into N t 、N s are the number of data in the two data sets, respectively, where K represents the total number of data classes in D, n represents the number of data classes in the split data set, and D t middle y i ∈[0,...,Kn-1],D s middle y i ∈[0, ..., n-1], where the smaller data set D is split s With the trigger added to the dataset D g Synthetically generate a new data set D p :D p ←Ds ∪D g , D p middle y i ∈[0, ..., n];

[0059] (4) Model developer O trains the target model y in parallel t With the ownership license model OL:

[0060]

[0061] The former represents the optimization process of the target model, and the latter represents the optimization process of the ownership licensing model OL. represents the loss function, f t Represents y t The initial model function to be optimized, f p Represents the initial model function of OL to be optimized, N p D p The amount of data in

[0062] (5) Model developer O uses model coupling algorithm A to process target model y t and the ownership licensing model OL: the prediction output of the ownership licensing model OL is the predicted value Multiply by weight α and assign to y t The other classes remain unchanged:

[0063]

[0064] where y d is the final output result of the ownership licensing model OL, λ is a hyperparameter, for y t Output results in different dimensions; use mask β to record the output of OL in the watermark trigger dimension, and the final output y of model F passes through the priority function P:

[0065]

[0066] Finally, a coupling model F is generated that can protect and verify data ownership;

[0067] (6) Model developer O distributes the coupled model F to model user U and retains the model ownership and model watermark related knowledge locally; when verifying the model ownership, model developer O adds trigger T to the randomly selected test sample for random data, generates a watermark test sample W, and compares W with D s The fingerprint samples B are input into the suspect model, that is, the generated coupling model F, and the output results are collected:

[0068] (6-1) If the output result is consistent with the content displayed by the watermark, the suspected model is no longer suspected and becomes an independent training model;

[0069] (6-2) If W successfully triggers the target watermark preset by the model developer O during verification, that is, the output result shows a predetermined deviation, the ownership of the model will be charged;

[0070] (6-3) When W verifies, the model predicts normally, but the output result of B verification is different from the expected output result of the model developer O, indicating that the attacker has attacked the illegally obtained model, and the model developer O accuses the model of ownership.

[0071] The present invention also provides a model data ownership protection and verification system based on model coupling, which executes the method steps as described above, and the system includes:

[0072] The first module is to create the original training data set of the model Select the model architecture S;

[0073] The second module inputs random data x into the watermark generator G to generate D g ;

[0074] The third module divides the training data set D into N t 、N s are the number of data in the two data sets, respectively, K represents the total number of data classes in D, n represents the number of data classes in the split data set, D t middle y i ∈[0,...,Kn-1],D s middle y i ∈[0, ..., n-1], where the smaller data set D is split s With the trigger added to the dataset D g Synthetically generate a new data set Dp;

[0075] The fourth module is to train the target model y in parallel t With the ownership license model OL:

[0076]

[0077] The former represents the optimization process of the target model, and the latter represents the optimization process of the ownership licensing model OL. represents the loss function, f t Represents y t The initial model function to be optimized, f p Represents the initial model function of OL to be optimized, N p D p The amount of data in

[0078] The fifth module uses the model coupling algorithm A to process the target model y t and the ownership licensing model OL: the prediction output of the ownership licensing model OL is the predicted value Multiply by weight α and assign to y t The other classes remain unchanged:

[0079]

[0080] where y d is the final output result of the ownership licensing model OL, λ is a hyperparameter, for y t Output results in different dimensions; use mask β to record the output of OL in the watermark trigger dimension, and the final output y of model F passes through the priority function P:

[0081]

[0082] Finally, a coupling model F is generated that can protect and verify data ownership;

[0083] The sixth module distributes the coupled model F to the model user U and retains the model ownership and model watermark related knowledge locally; when verifying the model ownership, the model developer O adds a trigger T to the randomly selected test sample for random data, generates a watermark test sample W, and compares W with D s The fingerprint samples B are sequentially input into the suspect model, i.e., the generated coupling model F, and the output results are collected.

[0084] The present invention also provides a computer system, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0085] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0086] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0087] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0090] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A model data ownership protection and verification method based on model coupling, characterized in that: include: (1) Model developer O creates the original model training dataset Select the model architecture S; (2) Model developer O inputs random data x into watermark generator G, generating D g ; (3) Model developer O divides the training dataset D into N t 、N s are the number of data in the two data sets, D t middle y i ∈[0,…,Kn-1], where K represents the total number of data classes in D, n represents the number of data classes in the split data set, and D s middle y i ∈[0,…,n-1], where a smaller data set D is split s With the trigger added to the dataset D g Synthetically generate a new data set D p ; (4) Model developer O trains the target model y in parallel t With the ownership license model OL: The former represents the optimization process of the target model, and the latter represents the optimization process of the ownership licensing model OL. represents the loss function, f t Represents y t The initial model function to be optimized, f p Represents the initial model function of OL to be optimized, N p D p The amount of data in (5) Model developer O uses model coupling algorithm A to process target model y t and the ownership licensing model OL: the prediction output of the ownership licensing model OL is the predicted value Multiply by weight α and assign to y t The other classes remain unchanged: where y d is the final output result of the ownership licensing model OL, λ is a hyperparameter, for y t Output results in different dimensions; use mask β to record the output of OL in the watermark trigger dimension, and the final output y of model F passes through the priority function P: Finally, a coupling model F is generated that can protect and verify data ownership; (6) Model developer O distributes the coupled model F to model user U and retains the model ownership and model watermark related knowledge locally; When verifying the ownership of the model, the model developer O adds a trigger T to the randomly selected test sample for random data, generates a watermark test sample W, and compares W with D s The fingerprint samples B are input into the suspect model, i.e. the generated coupling model F, in sequence, and the output results are collected.

2. According to the model data ownership protection and verification method based on model coupling according to claim 1, it is characterized in that: In step (6), W and D s The fingerprint samples B are input into the suspect model in sequence, and the specific implementation process of collecting the output results is as follows: (6-1) If the output result is consistent with the content displayed by the watermark, the suspected model is no longer suspected and becomes an independent training model; (6-2) If W successfully triggers the target watermark preset by the model developer O during verification, that is, the output result shows a predetermined deviation, the ownership of the model will be charged; (6-3) When W verifies, the model predicts normally, but the output result of B verification is different from the expected output result of the model developer O, indicating that the attacker has attacked the illegally obtained model, and the model developer O accuses the model of ownership.

3. According to the model data ownership protection and verification method based on model coupling according to claim 1, it is characterized in that: The process of step (1) is as follows: Model developer O has a private original dataset where x i ∈R d , with K types of data, denoted as y i ∈[0,…,K-1]; model developer O has white-box permission for model training and selects model architecture S according to requirements to perform model training.

4. According to the model data ownership protection and verification method based on model coupling according to claim 1, it is characterized in that: The process of step (2) is as follows: Model developer O presets an empty data set Input random data x that has no intersection with D into the watermark generator G to generate D g :D g ←G(x).

5. A method for protecting and verifying model data ownership based on model coupling according to claim 4, characterized in that: Random data x is input into watermark generator G, generating D g The watermark generation method relies on poisoning methods including BadNets or Sleeperagent.

6. A method for protecting and verifying model data ownership based on model coupling according to claim 5, characterized in that: Ownership licensing model OL and target model y t The coupling makes the watermark embedded in the ownership licensing model OL a triggerable backdoor of the coupling model F, and the triggering of the backdoor is used as a means to verify the ownership of F.

7. The model data ownership protection and verification method based on model coupling according to claim 1 is characterized in that: The process of step (3) is as follows: Model developer O divides the training data set into Then D s With D g Synthesize, generate D p :D p ←D s ∪D g , D p middle y i ∈[0,…,n].

8. A model data ownership protection and verification system based on model coupling, characterized in that: Executing the method steps of any one of claims 1 to 7, the system comprises: The first module is to create the original training data set of the model Select the model architecture S; The second module inputs random data x into the watermark generator G to generate D g ; The third module divides the training data set D into N t 、N s are the number of data in the two data sets, respectively, where K represents the total number of data classes in D, n represents the number of data classes in the split data set, and D t middle y i ∈]0,…,Kn-1],D s middle y i ∈[0,…,n-1], where a smaller data set D is split s With the trigger added to the dataset D g Synthetically generate a new data set D p ; The fourth module is to train the target model y in parallel t With the ownership license model OL: The former represents the optimization process of the target model, and the latter represents the optimization process of the ownership licensing model OL. represents the loss function, f t Represents y t The initial model function to be optimized, f p Represents the initial model function of OL to be optimized, N p D p The amount of data in The fifth module uses the model coupling algorithm A to process the target model y t and the ownership licensing model OL: the prediction output of the ownership licensing model OL is the predicted value Multiply by weight α and assign to y t The other classes remain unchanged: where y d is the final output result of the ownership licensing model OL, λ is a hyperparameter, for y t Output results in different dimensions; use mask β to record the output of OL in the watermark trigger dimension, and the final output y of model F passes through the priority function P: Finally, a coupling model F is generated that can protect and verify data ownership; The sixth module distributes the coupled model F to the model user U and retains the model ownership and model watermark related knowledge locally; when verifying the model ownership, the model developer O adds a trigger T to the randomly selected test sample for random data, generates a watermark test sample W, and compares W with D s The fingerprint samples B are sequentially input into the suspect model, i.e., the generated coupling model F, and the output results are collected.

9. A computer system, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method steps as claimed in any one of claims 1 to 7 can be implemented.

10. A computer-readable storage medium storing computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 7 can be implemented.

Images