Black box model watermarking method and device for resisting self-supervised model stealing attack
By introducing a shadow model into the self-supervised model, guiding the generation and embedding of watermarks, and verifying them in downstream tasks, the problem of watermarks being difficult to retain in self-supervised model theft attack is solved, and the robustness and effective verification of watermarks in different tasks are achieved.
Patent Information
- Application Number
- CN202411877842.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-05-02
AI Technical Summary
The prior art cannot effectively resist self-supervised model theft attacks, especially in black box model watermark verification scenarios, where watermarks are difficult to retain in the stolen model and are difficult to verify in different downstream tasks.
By introducing a shadow model, we guide the generation and embedding process of model watermarks, use the shadow model to mimic the output results of the original model on the pre-trained set, generate a watermark, and embed the watermark into the original model. At the same time, the classification consistency of carrying watermarks and clean samples is verified in downstream tasks, ensuring the robustness of watermarks in different tasks.
It effectively solves the problem that watermarks are difficult to retain after self-supervised model stealing, improves the robustness of watermarks in different downstream tasks, and ensures the effectiveness of black box watermark verification.
Smart Images
Figure CN119918031A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence security technology, and in particular to a black box model watermarking method and device for resisting self-supervisory model theft attacks. Background Art
[0002] Self-supervised learning has gradually gotten rid of its dependence on data labels in its development and has demonstrated strong adaptability in various downstream tasks. The main goal of self-supervised learning is to train a robust model that can convert input data into feature vectors, which is equivalent to a feature extractor that is not affected by downstream tasks or human intervention, thereby improving the performance of downstream classifiers in different tasks. The performance of the model depends on a large unlabeled dataset and sufficient computing resources. These stringent requirements make self-supervised models valuable, but also expose these models to potential self-supervised model stealing attacks, where attackers can steal the performance of the target model with just a few queries.
[0003] Currently, the main technology for protecting model copyright is model watermarking technology, which aims to embed a robust watermark into the model to identify illegally stolen versions. White-box watermarking requires access to the internal structure of the model during the verification phase, while black-box watermarking usually uses the output results of the model for verification, which is more convenient for practical application.
[0004] Since self-supervised learning uses unlabeled data to train the model and retrains it according to different downstream tasks, such as adding a classification layer to the model and using downstream data for training when facing a classification task. The lack of prior knowledge of downstream tasks makes the traditional black box model watermark in supervised learning cannot be directly applied to self-supervised models. In the prior art, a square pigment block is used as a trigger, and a category of data is selected as the target class to embed a backdoor-based watermark into the model, so that the watermark model produces a large difference in output for clean samples and samples with triggers, while the ordinary model will give similar results. This method can verify whether the model is a copyright infringing model downstream, but the distribution of the backdoor-based watermark is independent of the distribution of normal samples. When the model is attacked by theft, the watermark is not easy to be retained in the stolen model. There is also an existing technology for watermarking, which can retain part of the watermark information in the face of theft attacks, but the performance decreases when the attacker uses other downstream tasks to perform theft attacks.
[0005] In practical applications, model watermarks may be subject to various watermark removal attacks, while model stealing attacks only imitate the output results of the original model for a task selected by the attacker, making it more difficult to verify the watermark. The above existing technologies all have limitations. In the self-supervised model black box watermark verification scenario, they cannot effectively resist the self-supervised model stealing attack. Since this attack requires low cost and can easily obtain a high-performance model without a watermark, it poses a great threat to the existing self-supervised model black box watermark verification.
[0006] Therefore, in order to solve the above problems, the present invention proposes a black box model watermarking method for resisting self-supervisory model stealing attacks. Summary of the invention
[0007] Each exemplary embodiment of the present application provides a black box model watermark method and device, storage medium and electronic device for resisting self-supervised model theft attacks, so as to at least have the technical effect of not requiring prior knowledge about downstream tasks, and solving the problem that it is difficult to perform black box watermark verification downstream after the self-supervised model is attacked by theft.
[0008] According to one aspect of the present application, each exemplary embodiment of the present application provides a black box model watermarking method for resisting self-supervisory model stealing attacks;
[0009] The method comprises the following steps:
[0010] Get the original model E, and create its corresponding shadow model E according to the original model E s ;
[0011] The shadow model E is trained by imitating the output of the original model E on the pre-training set s ;
[0012] Select a key sample K and use it as the original model E and the shadow model E respectively s The output results of the two models are combined to generate and obtain the watermark m;
[0013] The watermark m is embedded into the original model E to obtain the watermark model E wm , the watermark model E wm , configured so that the output results remain similar for the watermarked input and the corresponding clean input without watermark;
[0014] A downstream suspicious model M is established to simulate the object of watermark verification, wherein the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack. It adds a classifier model of a linear classification layer for specific downstream tasks, and performs watermark verification through its classification consistency between samples carrying watermarks and clean samples.
[0015] According to another aspect of the present application, a black box model watermarking device for resisting self-supervisory model stealing attacks is also disclosed, comprising:
[0016] The shadow model generation module is configured to obtain the original model E and establish its corresponding shadow model E according to the original model E. s ; Imitate the output of the original model E on the pre-training set to train the shadow model E s ;
[0017] The watermark generation module is configured to select a key sample K and use it as the original model E and the shadow model E respectively. s The output results of the two models are combined to generate and obtain the watermark m;
[0018] The watermark model generation module embeds the watermark m into the original model E to obtain the watermark model E wm , the watermark model E wm , configured so that the output results remain similar for the watermarked input and the corresponding clean input without watermark;
[0019] The verification module is configured to establish a downstream suspicious model M to simulate the object of watermark verification, wherein the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack, and a classifier model with a linear classification layer is added for a specific downstream task, and the watermark verification is performed through the classification consistency of the samples carrying the watermark and the clean samples.
[0020] According to another aspect of the present application, a computer-readable storage medium is also disclosed, in which a computer program is stored, wherein the computer program is configured to execute any of the above methods when run.
[0021] According to another aspect of the present application, an electronic device is also disclosed, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any of the above-mentioned methods.
[0022] This application has the following beneficial effects:
[0023] The present invention introduces a shadow model to guide the generation and embedding process of the model watermark, thereby solving the problem that the watermark is difficult to retain in the pirated model after the self-supervised model is attacked by theft; by utilizing the consistency of the output results of the model for clean samples and watermark samples in downstream tasks, black-box watermark verification can be performed more effectively in different downstream tasks, thereby improving the robustness of the watermark. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0025] Figure 1 It is a schematic diagram of the overall process of the present invention;
[0026] Figure 2 It is a schematic diagram of the basic flow of an embodiment of the present invention;
[0027] Figure 3 is a schematic diagram of watermark generation and embedding according to an embodiment of the present invention;
[0028] Figure 4 It is a schematic diagram of the watermark verification process according to an embodiment of the present invention;
[0029] Figure 5 It is a schematic diagram of an optional electronic device structure of the present invention. DETAILED DESCRIPTION
[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the preferred embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.
[0031] Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of this application.
[0032] like Figures 1 to 3 As shown, the present invention provides a black box model watermarking method for resisting self-supervisory model stealing attacks, comprising the following steps:
[0033] S101, obtaining an original model E, and establishing a corresponding shadow model E according to the original model E s ;
[0034] S102, training the shadow model E by imitating the output result of the original model E on the pre-training set s ;
[0035] S103, selecting a key sample K and using it as the original model E and the shadow model E respectively s The output results of the two models are combined to generate and obtain the watermark m;
[0036] S104, embedding the watermark m into the original model E to obtain the watermark model E wm , the watermark model E wm , configured so that the output results remain similar for the watermarked input and the corresponding clean input without watermark;
[0037] S105, establish a downstream suspicious model M to simulate the object of watermark verification, wherein the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack, and a classifier model with a linear classification layer is added for a specific downstream task, and the watermark verification is performed through the classification consistency of the samples carrying the watermark and the clean samples.
[0038] It should be noted that the original model E is a model trained by a self-supervised learning method. An existing trained model can be used. Its output is a high-dimensional feature vector, which can be applied to different downstream tasks by adding a linear classification layer. It is the object protected by the watermark method. The watermark model is obtained by embedding a watermark into it to resist self-supervised model stealing attacks.
[0039] Get the original model E, and use the output of the original model E for its pre-training data set to train the corresponding shadow model E s , using the pre-training dataset as input, let the shadow model E s Imitate the output of the original model E for the pre-training set;
[0040] The key sample K is selected from data outside the distribution of the original model pre-training dataset, preferably from a private dataset that cannot be obtained by an attacker.
[0041] A downstream suspicious model M is established to simulate the object of watermark verification, where the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack. It adds a classifier model with a linear classification layer for a specific downstream task. In the black box scenario, the verifier can only obtain the final suspicious classifier model M instead of the substitute model directly obtained by the attacker through the self-supervised model stealing attack. The distribution of the downstream tasks targeted by the suspicious model M is not necessarily the same as the distribution of the pre-training data set of the original model. If the classification results of M for samples carrying watermarks and clean samples are highly consistent, it is considered that the watermark is successfully verified on M, and M is a pirated model that infringes the copyright of the original model.
[0042] Compared with direct verification in the upstream stage, the classification results given by the downstream classifier are not accurate feature vectors, the verification effect is affected by the downstream tasks used by the attacker, and the verifier has no knowledge of its downstream tasks, which brings difficulties to the downstream black box verification of pirated models.
[0043] The input of the downstream classifier is the feature vector output by the upstream model, and the newly added linear classification layer is trained using the dataset of the downstream task (the weight of the upstream model is fixed). The self-supervised model stealing attack occurs in the upstream stage. The substitute model obtained by the attacker belongs to the upstream model. The downstream classifier model is then trained and released according to the downstream task, and is the object of downstream black box watermark verification.
[0044] In this embodiment, the present invention specifically adopts the following preferred solutions:
[0045] The original model E and the shadow model E s , malicious model They are all multi-layer convolutional neural networks, among which the original model E is a pre-trained model using the self-supervised learning method.
[0046] like Figure 2-3 As shown, in this embodiment, the following preferred solutions can be adopted:
[0047] Training the shadow model E s The loss function used is:
[0048]
[0049] in, represents data distribution, l cos is the cosine similarity loss, E(X) and E s (X) is the original model E and the shadow model E s The output vector for the original image X.
[0050] In this embodiment, the following preferred solutions can be adopted:
[0051] The loss function used to generate the watermark m is:
[0052]
[0053] in, represents data distribution, l cos is the cosine similarity loss, E(K) and E(X+m) are the output vectors of the original model E for the key sample K and the image after adding watermark m to the original image X.
[0054] In this embodiment, the following preferred solutions can be adopted:
[0055] The loss function used for watermark embedding is:
[0056]
[0057] in, represents data distribution, l cos is the cosine similarity loss, E wm (X) and E wm (X+m) is the watermark model E wm For the output vector of the original image X and the image after adding watermark m to the original image X, E s (X+m) is the shadow model E s The output vector of the image after adding watermark m to the original image X.
[0058] Keep watermark model E wm The fidelity loss function of the performance is:
[0059]
[0060] in, represents data distribution, l cos is the cosine similarity loss, E(X) and E wm (X) is the original model E and the watermark model E wm The output vector for the original image X.
[0061] like Figure 3 As shown in the figure, after the model owner releases the trained self-supervised model, a malicious third party can use a smaller query dataset to conduct a self-supervised model theft attack on the target model and add a linear classification layer to the pirated model for different downstream tasks. Suspicious models in the downstream task stage usually only have access to black box access rights. The verifier can determine whether the model is a pirated version of the protected model by the consistency of the output results of the suspicious model for clean samples and watermarked samples.
[0062] According to another aspect of the present application, a black box model watermarking device for resisting self-supervisory model stealing attacks is also provided, comprising:
[0063] The shadow model generation module is configured to obtain the original model E and establish its corresponding shadow model E according to the original model E. s ; Imitate the output of the original model E on the pre-training set to train the shadow model E s ;
[0064] The watermark generation module is configured to select a key sample K and use it as the original model E and the shadow model E respectively. s The output results of the two models are combined to generate and obtain the watermark m;
[0065] The watermark model generation module embeds the watermark m into the original model E to obtain the watermark model E wm , the watermark model E wm , configured so that the output results remain similar for the watermarked input and the corresponding clean input without watermark;
[0066] The verification module is configured to establish a downstream suspicious model M to simulate the object of watermark verification, wherein the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack, and a classifier model with a linear classification layer is added for a specific downstream task, and the watermark verification is performed through the classification consistency of the samples carrying the watermark and the clean samples.
[0067] According to another aspect of the embodiment of the present application, an electronic device for a black box model watermarking method for resisting self-supervisory model stealing attacks is also provided, and the electronic device can be but is not limited to being applied in a server. Figure 5As shown, the electronic device includes a memory 302 and a processor 304. The memory 302 stores a computer program, and the processor 304 is configured to execute the steps in any of the above method embodiments through the computer program.
[0068] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.
[0069] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0070] S1, obtain the original model E, and establish its corresponding shadow model E according to the original model E s ;
[0071] S2, train the shadow model E by imitating the output result of the original model E on the pre-training set s ;
[0072] S3, select a key sample K and use it as the original model E and the shadow model E respectively s The output results of the two models are combined to generate and obtain the watermark m;
[0073] S4, embedding the watermark m into the original model E to obtain the watermark model E wm , the watermark model E wm , configured so that the output results remain similar for the watermarked input and the corresponding clean input without watermark;
[0074] S5, establish a downstream suspicious model M to simulate the object of watermark verification, wherein the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack, which adds a classifier model of a linear classification layer for a specific downstream task, and performs watermark verification through its classification consistency between the samples carrying the watermark and the clean samples.
[0075] Alternatively, a person skilled in the art may understand that: Figure 5 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a mobile Internet device (MID) or a PAD. Figure 5 The structure of the electronic device is not limited. Figure 5 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 5 Different configurations are shown.
[0076] The memory 302 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 302 may further include a memory remotely located relative to the processor 304, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. As an example, Figure 5 As shown, the memory 302 may include, but is not limited to, a shadow model generation module, a watermark generation module, etc., which will not be described in detail in this example.
[0077] Optionally, the transmission device 306 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 306 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 306 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0078] In addition, the electronic device further comprises: a display 308 for displaying the current verification result; and a connection bus 310 for connecting various module components in the electronic device.
[0079] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.
[0080] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:
[0081] S1, obtain the original model E, and establish its corresponding shadow model E according to the original model E s ;
[0082] S2, train the shadow model E by imitating the output result of the original model E on the pre-training set s ;
[0083] S3, select a key sample K and use it as the original model E and the shadow model E respectively s The output results of the two models are combined to generate and obtain the watermark m;
[0084] S4, embedding the watermark m into the original model E to obtain the watermark model Ewm , the watermark model E wm , configured so that the output results remain similar for the watermarked input and the corresponding clean input without watermark;
[0085] S5, establish a downstream suspicious model M to simulate the object of watermark verification, wherein the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack, which adds a classifier model of a linear classification layer for a specific downstream task, and performs watermark verification through its classification consistency between the samples carrying the watermark and the clean samples.
[0086] Optionally, the storage medium is also configured to store a computer program for executing the steps included in the method in the above embodiment, which will not be described in detail in this embodiment.
[0087] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.
[0088] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0089] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
[0090] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0091] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0092] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0093] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0094] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
[0095] Although the preferred embodiments of the present application have been described, those skilled in the art, once knowing the basic creative concept, can make other changes and modifications according to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A black box model watermarking method for resisting self-supervised model stealing attacks, characterized in that: The method comprises the following steps: Get the original model E, and create its corresponding shadow model E according to the original model E s ; Imitate the output of the original model E on the pre-training set to train the shadow model E s ; Select a key sample K and use it as the original model E and the shadow model E respectively s The output results of the two models are combined to generate and obtain the watermark m; The watermark m is embedded into the original model E to obtain the watermark model E wm , the watermark model E wm , configured so that the output results remain similar for the watermarked input and the corresponding clean input without watermark; A downstream suspicious model M is established to simulate the object of watermark verification, wherein the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack. It adds a classifier model of a linear classification layer for specific downstream tasks, and performs watermark verification through its classification consistency between samples carrying watermarks and clean samples.
2. The black box model watermarking method for resisting self-supervised model stealing attack according to claim 1 is characterized in that: Training the shadow model E s The loss function used is: in, represents data distribution, l cos is the cosine similarity loss, E(X) and E s (X) is the original model E and the shadow model E s The output vector for the original image X.
3. The black box model watermarking method for resisting self-supervised model stealing attack according to claim 2 is characterized in that: The loss function used to generate and obtain the watermark m is: in, represents data distribution, l cos is the cosine similarity loss, E(K) and E(X+m) are the output vectors of the original model E for the key sample K and the image after adding watermark m to the original image X.
4. The black box model watermarking method for resisting self-supervisory model stealing attack according to claim 3 is characterized in that: The loss function used for watermark embedding is: in, represents data distribution, l cos is the cosine similarity loss, E wm (X) and E wm (X+m) is the watermark model E wm For the output vector of the original image X and the image after adding watermark m to the original image X, E s (X+m) is the shadow model E s The output vector of the image after adding watermark m to the original image X.
5. The black box model watermarking method for resisting self-supervisory model stealing attack according to claim 4 is characterized in that: The watermark model E wm Use the fidelity loss function to maintain the watermark model E wm The performance of the watermark model E wm The fidelity loss function of the performance is: in, represents data distribution, l cos is the cosine similarity loss, E(X) and E wm (X) is the original model E and the watermark model E wm The output vector for the original image X.
6. The black box model watermarking method for resisting self-supervised model stealing attack according to claim 1 is characterized in that: The original model E and the shadow model E s , malicious model M are all multi-layer convolutional neural networks, where the original model E is a model pre-trained by the self-supervised learning method.
7. A black box model watermarking device for resisting self-supervised model stealing attacks, characterized in that: include: The shadow model generation module is configured to obtain the original model E and establish its corresponding shadow model E according to the original model E. s ; Imitate the output of the original model E on the pre-training set to train the shadow model E s ; The watermark generation module is configured to select a key sample K and use it as the original model E and the shadow model E respectively. s The output results of the two models are combined to generate and obtain the watermark m; The watermark model generation module embeds the watermark m into the original model E to obtain the watermark model E wm , the watermark model E wm , configured so that the output results remain similar for the watermarked input and the corresponding clean input without watermark; The verification module is configured to establish a downstream suspicious model M to simulate the object of watermark verification, wherein the suspicious model M is a substitute model obtained after the suspected original model is attacked by the self-supervised model stealing attack, and a classifier model with a linear classification layer is added for a specific downstream task, and the watermark verification is performed through the classification consistency of the samples carrying the watermark and the clean samples.
8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 6 when executed.
9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.