An image dataset copyright protection method, device, equipment and medium
By embedding copyright watermarks in image datasets and generating reversible, non-learnable perturbations, and utilizing generative adversarial networks and dual watermark decoders, this approach addresses the problem in existing technologies that cannot simultaneously prevent unauthorized model training and achieve watermark extraction, thereby improving the robustness and adaptability of copyright protection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MACAU UNIV OF SCI & TECH
- Filing Date
- 2026-05-07
- Publication Date
- 2026-07-24
AI Technical Summary
Existing technologies cannot effectively protect the copyright of image datasets while preventing unauthorized model training, and non-learnable perturbations can interfere with watermark extraction, leading to the loss of watermark information.
By embedding copyright watermark information into an image dataset and generating reversible, non-learnable perturbations, a watermark encoder and decoder are trained using a generative adversarial network to generate a reversible, non-learnable perturbation dataset. A perturbation dictionary is generated by minimizing the input-output mutual information, and a dual-watermark decoder is used to extract the watermark information.
This approach prevents data use during unauthorized model training while ensuring reliable watermark extraction under any circumstances, thus improving the robustness and adaptability of copyright protection and reducing implementation costs.
Smart Images
Figure CN122453587A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and information security technology, and in particular to a method, apparatus, device and medium for copyright protection of image datasets. Background Technology
[0002] With the rapid development of deep learning technology, the value of large-scale datasets in model training is becoming increasingly prominent. However, dataset owners face two serious threats when publicly sharing image data:
[0003] Unauthorized model training: Unauthorized users steal high-quality data to train commercial models, resulting in the leakage of semantic information;
[0004] Malicious data breach: Authorized users illegally disseminate undisturbed raw data, making it impossible to verify copyright ownership.
[0005] Existing copyright protection methods can be divided into two categories: (1) protecting semantic information through unlearnable examples; (2) verifying ownership through digital watermarks.
[0006] However, when the two methods are directly combined, the perturbation will interfere with watermark extraction, resulting in the loss of watermark information; and relying on a single method cannot simultaneously achieve "preventing model learning" and "copyright tracing". Summary of the Invention
[0007] This invention provides a method, apparatus, device, and medium for copyright protection of image datasets, which addresses the technical shortcomings of existing technologies that cannot effectively prevent unauthorized model training by utilizing non-learnable perturbations while providing a reliable watermark verification mechanism for legitimate owners, thus achieving efficient synergy between perturbation protection and copyright verification.
[0008] This invention provides a method for copyright protection of image datasets, comprising:
[0009] The copyright watermark information is embedded into the original image dataset to be protected by the watermark encoder to generate a watermarked image dataset.
[0010] Based on the watermarked image dataset, a reversible, non-learnable perturbation dataset is generated by minimizing the mutual information between the input and output of the victim model.
[0011] The non-learnable perturbation dataset is added to the watermarked image dataset to generate a protected image dataset.
[0012] The copyright watermark information is extracted from the watermarked image dataset using a first watermark decoder, and the copyright watermark information is extracted from the protected image dataset using a second watermark decoder.
[0013] The first watermark decoder is trained based on the watermarked image dataset; the second watermark decoder is trained based on the protected image dataset.
[0014] According to the image dataset copyright protection method provided by the present invention, the non-learnable perturbation dataset is a perturbation dictionary generated according to image categories, and all images under the same image category share the same perturbation template.
[0015] According to the image dataset copyright protection method provided by the present invention, the watermark encoder and the first watermark decoder are obtained through joint training, the joint training including:
[0016] The watermark encoder and the first watermark decoder are trained by using a joint loss function and a discriminator.
[0017] The joint loss function includes: image fidelity loss between the watermarked image dataset and the original image dataset; extraction loss of the first watermark decoder for extracting the copyright watermark information; and discrimination loss of the discriminator for the watermarked image dataset.
[0018] The discriminator and the watermark encoder form a generative adversarial network, and the discriminator is used to distinguish the watermarked image dataset generated by the watermark encoder from the original image dataset.
[0019] According to the image dataset copyright protection method provided by the present invention, the process of generating the perturbation dictionary adopts an alternating training strategy and repeatedly executes the following steps until a preset stopping condition is met:
[0020] With the current perturbation dictionary fixed, the victim model is trained using the protected image dataset;
[0021] The victim model, after being trained, generates a corresponding personalized perturbation for each image in the watermarked image dataset by minimizing the mutual information between the input and output of the victim model.
[0022] The personalized perturbations of all images under the same image category are averaged to obtain the perturbation template of the image category, and the perturbation dictionary is updated.
[0023] According to the image dataset copyright protection method provided by the present invention, the protected image dataset is JPEG compressed before training the victim model using the protected image dataset during the generation of the perturbation dictionary.
[0024] According to the image dataset copyright protection method provided by the present invention, the watermarked image dataset is JPEG compressed before the first decoder extracts the copyright watermark information; and / or, the protected image dataset is JPEG compressed before the second decoder extracts the copyright watermark information.
[0025] According to the image dataset copyright protection method provided by the present invention, minimizing the mutual information between the input and output of the victim model specifically includes:
[0026] Achieve this through the following objective function:
[0027]
[0028] in, Represents a watermarked image dataset The i-th watermarked image in the image; express Corresponding personalized perturbations; For the victim model; ε represents the differentiable JPEG compression module; ε represents the perturbation amplitude constraint.
[0029] According to the image dataset copyright protection method provided by the present invention, the first watermark decoder and the second watermark decoder have the same structure; and the second watermark decoder is trained with the parameters of the first watermark decoder as the initial state.
[0030] The present invention also provides an image dataset copyright protection device, comprising:
[0031] The watermark embedding module is used to embed copyright watermark information into the original image dataset to be protected through the watermark encoder, generating a watermarked image dataset.
[0032] The first generation module is used to generate a reversible, non-learnable perturbation dataset based on the watermarked image dataset by minimizing the mutual information between the input and output of the victim model.
[0033] The second generation module is used to add the non-learnable perturbation dataset to the watermarked image dataset to generate a protected image dataset.
[0034] The extraction module is used to extract the copyright watermark information from the watermarked image dataset using a first watermark decoder, and to extract the copyright watermark information from the protected image dataset using a second watermark decoder.
[0035] The first watermark decoder is trained based on the watermarked image dataset; the second watermark decoder is trained based on the protected image dataset.
[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method as described above.
[0037] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image dataset copyright protection method as described above.
[0038] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image dataset copyright protection method as described above.
[0039] Compared with the prior art, the beneficial effects of the present invention are:
[0040] (1) By introducing a reversible non-learnable perturbation and a collaborative mechanism of dual watermark extractor, this invention constructs a complete copyright protection framework that simultaneously covers "preventing unauthorized training" and "realizing ownership traceability", which solves the problem of prior art being unable to address both aspects and improves copyright protection capabilities.
[0041] (2) This invention, from the perspective of information theory, generates unlearnable perturbations by minimizing the mutual information between input and output, so that it has good generalization attack capabilities against different structures and unknown victim models.
[0042] (3) The present invention effectively overcomes the masking effect of unlearnable perturbation on watermark information through the dual watermark extractor strategy, ensuring that the watermark can be accurately extracted in any state (with or without perturbation), thus improving robustness.
[0043] (4) By adopting a class-wide perturbation and an integrated JPEG compression robust design, the present invention significantly reduces the implementation cost of the scheme and improves its adaptability in real network environments. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0045] Figure 1 This is a flowchart illustrating an image dataset copyright protection method according to an embodiment of the present invention.
[0046] Figure 2This is an overall framework diagram of the image dataset copyright protection method according to an embodiment of the present invention.
[0047] Figure 3 This is a flowchart illustrating the non-learnable perturbation generation algorithm according to an embodiment of the present invention.
[0048] Figure 4 This is a schematic diagram of an application scenario according to an embodiment of the present invention.
[0049] Figure 5 This is a schematic diagram of the structure of an image dataset copyright protection device according to an embodiment of the present invention. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0051] This invention provides a method for protecting the copyright of image datasets, comprising the following steps:
[0052] Step 100: Embed the copyright watermark information into the original image dataset to be protected using a watermark encoder to generate a watermarked image dataset.
[0053] Step 101: Based on the watermarked image dataset, generate a reversible, non-learnable perturbation dataset by minimizing the mutual information between the input and output of the victim model.
[0054] Step 102: Add the non-learnable perturbation dataset to the watermarked image dataset to generate a protected image dataset.
[0055] Step 103: Extract the copyright watermark information from the watermarked image dataset using the first watermark decoder, and extract the copyright watermark information from the protected image dataset using the second watermark decoder.
[0056] The first watermark decoder is trained based on the watermarked image dataset; the second watermark decoder is trained based on the protected image dataset.
[0057] Specifically, in step 100 above, the watermark encoder can be a deep learning-based watermark encoder, such as a convolutional neural network, or a traditional non-deep learning watermark encoder, such as an encoder based on discrete cosine transform; no limitation is made here. The copyright watermark information represents a piece of information indicating ownership; it can be a binary sequence and may contain owner identifiers, dates, etc., such as watermark information. It is a length of binary sequences (e.g.) (bit).
[0058] Copyright watermark information is imperceptibly embedded in every original image in the original image dataset. To improve imperceptibility, a watermark encoder can be trained.
[0059] In step 101 above, the core idea of adding perturbation is to generate noise that can effectively disrupt the training process of deep learning models, thereby preventing unauthorized users from conducting illegal model training; the noise itself is also reversible, so that authorized users can recover a clean image from the perturbated image for legitimate model training.
[0060] From an information theory perspective, the victim model completely destroys the inherent patterns and information within the data itself, which could be learned by any model. The victim model can be a classification model simulating a potential attacker. Its input is a perturbated protected image, and its output is the classification prediction probability distribution of the input image. When the mutual information between the input and output is minimized, it means that the perturbation has maximally destroyed the classifiability of the image data, preventing the model from learning effective features from the protected image and thus preventing unauthorized users from illegally training the model. The network architecture of the victim model can be ResNet-18 or VGG-11, without restriction. The victim model and the perturbation can be trained alternately, allowing them to continuously evolve in a loop, ultimately resulting in a powerful, general, and unlearnable perturbation.
[0061] In step 102 above, the protective perturbation generated in step 101 is applied to the watermarked image to obtain a protected image dataset that can be directly distributed.
[0062] "Addition" refers to applying a non-learnable perturbation to each watermarked image in the watermarked dataset. Addition can be a pixel-by-pixel addition operation. It's important to note that "addition" here is a deterministic, reversible arithmetic operation, not a training or optimization process. Authorized users can approximate the clean watermarked image from the protected images by performing the reverse operation, thus enabling legitimate use of the image dataset. The generated protected image dataset contains both imperceptible copyright watermark information and strong non-learnable perturbations, giving it the dual capabilities of ownership verification and preventing unauthorized model training.
[0063] In step 103 above, the watermark is extracted from the watermarked image dataset using the first watermark decoder, verifying whether the watermark is successfully embedded and can be accurately read in its original, undisturbed state. The watermark is then extracted from the protected image dataset (i.e., the image set with added non-learnable perturbations) using the second watermark decoder, verifying that the copyright information can still be reliably extracted even after strong perturbations are applied, thus demonstrating the effectiveness of the entire protection framework. The first watermark decoder is trained based on the watermarked image dataset; the second watermark decoder is trained based on the protected image dataset.
[0064] This invention constructs a complete copyright protection framework that simultaneously covers "preventing unauthorized training" and "achieving ownership tracing" by introducing a collaborative mechanism of reversible non-learnable perturbation and a dual watermark extractor. This solves the problem of prior art failing to address both aspects and improves copyright protection capabilities. By minimizing the input-output mutual information to generate non-learnable perturbations, it has good generalization attack capabilities against different structures and unknown victim models. Through the dual watermark extractor strategy, it effectively overcomes the masking effect of non-learnable perturbations on watermark information, ensuring that watermarks can be accurately extracted in any state (with or without perturbation), thus improving robustness.
[0065] Optionally, the non-learnable perturbation dataset is a perturbation dictionary generated according to image categories, and all images under the same image category share the same perturbation template.
[0066] Specifically, the perturbation dictionary can be a key-value pair data structure, where the key is the image category label and the value is the non-learnable perturbation template corresponding to that category. The categories in this dictionary are completely consistent with the categories in the watermarked image dataset. When generating protected images, the corresponding perturbation templates can be retrieved from this dictionary and added based on the category labels of the watermarked images.
[0067] Generate a universal perturbation template for each image (semantic) category, and share this template with all images in that category. An aggregation computation method can be used to transform a set containing N perturbations into a perturbation dictionary containing only C perturbation templates (where C is the number of categories, usually C << N).
[0068] The methods described above reduce the storage overhead required to protect N images from O(N) to O(C). For example, for ImageNet-1K (1.27 million images, 1000 categories), the storage overhead is reduced by approximately 1270 times, making it feasible to protect extremely large datasets. Since the perturbation is obtained by averaging multiple samples of the same type of image, it captures the common features of that type of image, enhancing the generalization ability of the method. Only a small dictionary file needs to be maintained, making it easy to distribute and authorize verification.
[0069] Optionally, the watermark encoder and the first watermark decoder are obtained through joint training, wherein the joint training includes:
[0070] The watermark encoder and the first watermark decoder are trained by using a joint loss function and a discriminator.
[0071] The joint loss function includes: image fidelity loss between the watermarked image dataset and the original image dataset; extraction loss of the first watermark decoder for extracting the copyright watermark information; and discrimination loss of the discriminator for the watermarked image dataset.
[0072] The discriminator and the watermark encoder form a generative adversarial network, and the discriminator is used to distinguish the watermarked image dataset generated by the watermark encoder from the original image dataset.
[0073] Specifically, such as Figure 2 As shown, a watermarking network consisting of a watermark encoder E and a first watermark decoder D is used for watermark embedding. The watermark encoder E receives the original image. and watermark information Output watermarked images The goal of training this network is to minimize the joint loss function. The function contains:
[0074] (1) Image fidelity loss (Mean Squared Error, MSE): Guarantee and Visual similarity.
[0075] (2) Discriminant loss of the discriminator on the watermarked image dataset, i.e. adversarial loss (with discriminator A): further improves the imperceptibility of the watermark.
[0076] (3) Watermark extraction loss: Characterized by binary cross-entropy (BCE), ensuring that the first decoder D can extract watermarks from the target area. Accurately extract watermarks .in, These are the trainable parameters for the first decoder; It is a differentiable JPEG module, which can be used to improve robustness.
[0077] In summary, by minimizing Loss is updated during training, and the joint loss function is used. as follows:
[0078]
[0079] in, For image fidelity loss, To combat the losses, This represents binary cross-entropy.
[0080] Using only mean squared error loss can only guarantee pixel-level average similarity. It may produce blurring or modifications that do not conform to the statistical characteristics of natural images, and the human eye may perceive unnatural textures or artifacts. Discriminator A is a deep neural network that can learn and capture the high-level, semantic feature distribution of natural images (such as the statistical regularities of texture, edge, and color distribution).
[0081] Discriminator A is trained to distinguish between the "original image" and the "watermarked image," outputting the probability of the watermarked image. Its goal is to classify them as accurately as possible. Encoder E is trained to generate watermarked images that are visually indistinguishable from the original image; this corresponds to the adversarial loss of encoder E. By minimizing this loss term (i.e., letting...) The encoder E is driven to generate an output that appears identical to the original image to the discriminator.
[0082] Optionally, the process of generating the perturbation dictionary employs an alternating training strategy and repeatedly executes the following steps until a preset stopping condition is met:
[0083] With the current perturbation dictionary fixed, the victim model is trained using the protected image dataset;
[0084] The victim model, after being trained, generates a corresponding personalized perturbation for each image in the watermarked image dataset by minimizing the mutual information between the input and output of the victim model.
[0085] The personalized perturbations of all images under the same image category are averaged to obtain the perturbation template of the image category, and the perturbation dictionary is updated.
[0086] Specifically, such as Figure 3 As shown, the non-learnable perturbation generation algorithm employs an alternating optimization framework. Its process begins with initializing class-level (image category level, or simply class-level) perturbations and a victim model, followed by a loop of two core steps: First, the perturbations are fixed, and the victim model is trained using protected data to minimize the classification loss. Then, the model is fixed, gradients are calculated based on the principle of minimizing mutual information, and the personalized perturbation for each image is updated. The perturbation amplitude is then constrained by cropping, and perturbations of the same category are aggregated into a general class-level noise. This loop continues until the victim model's accuracy on noisy data exceeds a threshold, ultimately outputting a reversible, transferable, and robust class-level perturbation dictionary that is robust to JPEG compression, achieving efficient semantic information protection. The set threshold is 99%, meaning the loop stops when the victim model's accuracy on noisy data exceeds 99%.
[0087] The pseudocode for the algorithm is as follows:
[0088]
[0089] Optionally, the generation process of the unlearnable perturbation adopts an optimization objective function based on minimizing mutual information.
[0090]
[0091] That is, minimizing the mutual information between the input and output of the victim model, specifically including:
[0092] Achieve this through the following objective function:
[0093]
[0094] in, Represents a watermarked image dataset The i-th watermarked image in the image; express Corresponding personalized perturbations; For the victim model; ε represents the differentiable JPEG compression module; ε represents the perturbation amplitude constraint, the size of which can be adjusted according to requirements, roughly in the range of [0, 16 / 255], with a commonly used perturbation value of 8 / 255. The larger the perturbation value, the better the protection of semantic information of image data, but the greater the impact on image quality.
[0095] The mutual information between the victim model's input and output reflects the degree of interdependence between them. Estimating this mutual information is difficult for high-dimensional inputs; therefore, a simplified formula (Formula 2) is derived for computation. During perturbation optimization, the perturbation can be updated using gradient descent. The actual calculation process is as follows:
[0096] Forward propagation: Calculates the probability of the perturbed image passing through the victim model's output for each category. ;
[0097] Calculate entropy loss: ;
[0098] Backpropagation: Using the chain rule, backpropagation is performed through the victim model and the differentiable JPEG module to calculate the entropy loss against the perturbation. The gradient;
[0099] Updated perturbation: ;in It is a sign function, step size
[0100] Disturbance clipping: ;
[0101] Class-level perturbation aggregation: Averaging the perturbations of all images within the same class. .
[0102] CLIP is the clipping or throttling function. Perturbation clipping involves clipping the perturbation vector... Perform element-wise clipping, limiting the perturbation value at each pixel position to a range of [ ]Inside.
[0103] The objective function of the victim model in the above embodiments drives the generated perturbations to no longer be limited to misleading the model's classification results, but to destroy the statistical correlation between training data and model parameter updates from an information theory perspective, thereby fundamentally depriving the unauthorized model of its ability to learn effective features from the data.
[0104] Optionally, during the generation of the perturbation dictionary, the protected image dataset is JPEG compressed before training the victim model using the protected image set.
[0105] Optionally, the watermarked image dataset is JPEG compressed before the first decoder extracts the copyright watermark information; and / or, the protected image dataset is JPEG compressed before the second decoder extracts the copyright watermark information.
[0106] Specifically, the Differentiable JPEG Lossy Compression (Joint Photographic Experts Group, JPEG) module is used during training to simulate image compression in real-world scenarios, enhancing the robustness of watermarks and unlearnable perturbations to JPEG compression. In the experiments, the JPEG Quality Factor (QF) was set to 75 to simulate common image compression intensities, and then transferred to other JPEG QFs.
[0107] By optimizing the perturbation and watermarking objectives (minimizing mutual information and minimizing extraction loss) in an environment containing JPEG noise, the learned perturbation patterns and embedded watermark information are forced to be robust to JPEG compression. This ensures that even if the protected image is lossily compressed during distribution, its protective effect and copyright information are not easily eliminated. Simultaneously, JPEG compression increases the diversity of training data, making the generated perturbation and watermark extractor applicable not only to original images but also to degraded images that have undergone common processing, greatly improving the generalization ability and reliability of the entire copyright protection system in complex real-world scenarios.
[0108] Optionally, the first watermark decoder and the second watermark decoder have the same structure; and the second watermark decoder is trained using the parameters of the first watermark decoder as the initial state.
[0109] Specifically, in order to extract from protected images The watermark is extracted from the first watermark decoder, and a second watermark decoder with the same structure as the first watermark decoder can be trained. Its loss function is:
[0110] (3)
[0111] in Ultimately, the total loss of the watermarking network was... .
[0112] The second watermark decoder and the first watermark decoder are trained in stages: after the first watermark decoder is trained and the optimized perturbation dictionary is obtained, the watermark encoder and the first watermark decoder are fixed, and only the parameters of the second watermark decoder are trained. At this time, the parameters of the first watermark decoder are used to initialize the second watermark decoder. Finally, fine-tuning is performed on the perturbation data. The parameters of the first watermark decoder and the second watermark decoder are independent and are not shared.
[0113] For example, when watermark information It is a length of binary sequences (e.g.) When (bits), the output of the first watermark decoder or the second watermark decoder It is a length of A vector, where each element is a probability value after passing through the Sigmoid activation function. , which represents the probability that the bit is 1.
[0114] Therefore, the BCE loss of the entire watermarked message is the average of the losses of all bits:
[0115] (4)
[0116] in: It is a real watermark. The value of the bit (0 or 1); It is the first output of the decoder A probability value.
[0117] Using the above method, the second watermark decoder (D_d) inherits the prior knowledge of watermark feature extraction learned by the first watermark decoder through parameter initialization. It has a very fast convergence speed and a significant reduction in the required training data and computing resources. The final second watermark decoder is more robust and has a higher accuracy in the task of extracting watermarks from protected images.
[0118] The following is through Figure 2 The method of this invention will be described by way of example. For example... Figure 2 As shown, the overall process of copyright protection for image datasets consists of three core stages: First, in stage one, an invisible watermark is embedded in the original image through an encoder-decoder network, and a decoder is trained for extraction. Next, in stage two, an alternating optimization algorithm based on minimizing mutual information is used to generate class-level non-learnable perturbations with strong transferability for the watermarked image, thus obtaining a "protected image" that combines copyright identification with anti-learning capabilities. Finally, in stage three, a dual-path watermark extractor is designed, capable of reliably extracting the watermark regardless of the presence or absence of the perturbations to complete ownership verification. The entire process is connected by a prominent yellow path, clearly presenting the complete closed loop of data from its original state to a publicly protected state, and finally to a verifiable state, demonstrating the innovative design of watermarks and perturbations working collaboratively without interference.
[0119] Specifically, in Phase 1, the watermark embedding and extraction module adopts an encoder-decoder architecture. The encoder E consists of multiple convolutional layers, batch normalization layers, ReLU activation functions, and a Squeeze-and-Excitation attention module. It preserves image details through skip connections and fuses the original image and binary watermark information to output a visually high-quality watermarked image. The decoder has a symmetrical structure and includes transposed convolutional layers for feature upsampling. Its input first passes through a differentiable JPEG compression module to simulate real perturbations and finally outputs a watermark probability vector. In addition, a discriminator consisting of five convolutional blocks is introduced during training for adversarial optimization to ensure the imperceptibility of the watermark. The entire network is trained end-to-end by jointly minimizing the image reconstruction error, adversarial loss, and binary cross-entropy loss for watermark extraction.
[0120] The encoder E (i.e., the watermark encoder) is a deep convolutional neural network, with the following structure: Figure 2 As shown in (Stage 1), this network receives a three-channel original image and a copyright watermark information of length L as input, and outputs a watermarked image. The network includes a Squeeze-and-Excitation (SE) attention module, which adaptively adjusts the weights of the feature channels to optimize the position and intensity of the watermark embedding. The network receives the original RGB image and processed watermark message features as input. First, it extracts basic features through a 3×3 convolutional layer, followed by four consecutive Squeeze-and-Excitation (SE) blocks for advanced feature learning and channel attention recalibration. Next, the network fuses the enhanced image features with the message features and integrates them through another 3×3 convolutional layer. Finally, the encoder E uses a residual connection structure to concatenate the integrated features with the original image and generates a visually high-quality, watermarked encoded image through a 1×1 convolutional layer, with the output size consistent with the original input.
[0121] The first watermark decoder D receives the watermarked image as input. It first performs preliminary feature extraction through a 3×3 convolutional layer, followed by four SE blocks to enhance feature representation and focus on key information. Then, the network downsamples the feature map through a series of transposed convolutional layers, gradually reducing the spatial size to a dimension that matches the length of the watermark message. Finally, the decoder uses a 3×3 convolutional layer to map the multi-channel features to a single-channel output and reshapes it into a binary sequence, thereby accurately recovering the embedded original watermark message.
[0122] Discriminator A receives the watermarked image as input and first performs preliminary feature extraction through a 3×3 convolutional layer. Then, it sequentially passes through four downsampling convolutional blocks, each containing a 3×3 convolutional layer (with a stride of 2 to reduce spatial resolution), a batch normalization layer (except for the first block), and a LeakyReLU activation function. The number of output channels increases progressively with each block: 64, 128, 256, and 512, respectively. Next, the network uses a 1×1 convolutional layer to compress the feature channels to 1 and aggregates spatial information through a global average pooling layer. Finally, a fully connected layer maps the scalar features to the final discrimination probability, which is output via a Sigmoid activation function to distinguish whether the input image is the original image or a watermarked encoded image.
[0123] In Phase Two, the training and generation performed by the data owner constitute the preparation phase of the method in this embodiment of the invention, generating a class-level perturbation dictionary. With the perturbations fixed, the victim model is optimized by backpropagating the cross-entropy (CE Loss) (see Equations 3 and 4) and updating its parameters. With the model fixed, the perturbations are optimized by minimizing the mutual information loss (MI Loss) between the victim model's input and output (see Equation 2). This is equivalent to maximizing the entropy predicted by the model.
[0124] In Phase 3, the dual-watermark extractor operates using a parallel dual-decoder architecture: the image to be verified is simultaneously input into two structurally identical but functionally specialized decoders, where the first decoder D is responsible for extracting information from the assumed perturbation-free watermarked image, and the second decoder... The system is specifically designed to learn and overcome interference from unlearnable perturbations to extract information from protected images. After the two decoders output their extracted watermark bitstreams, they are compared with the original embedded watermark. As long as the output of either decoder matches the original watermark, ownership verification is successfully completed, thus ensuring that copyright information can be reliably traced regardless of whether the infringer holds perturbed data or clean data that has been restored.
[0125] To further understand the solutions of the embodiments of the present invention, the following example is provided. Figure 4 The application scenario shown is used as an example for illustrative explanation.
[0126] Phase 1: Training and Generation (performed by the data owner). This is the system preparation phase, with the goal of generating a class-level perturbation dictionary.
[0127] Input: The original labeled dataset {(image, category label)}.
[0128] process:
[0129] Watermark embedding: First, all images are processed into 256... Size 256. Randomly generated 64-bit watermark information is embedded with a unified copyright watermark through watermark encoder E to obtain an intermediate dataset.
[0130] Perturbation generation: An alternating optimization algorithm that minimizes mutual information is run. For each class c, the algorithm optimizes an optimal perturbation vector for all images in that class. .
[0131] Aggregate storage: This stores each category c along with its corresponding perturbation vector. Stored as a "perturbation dictionary": .
[0132] Output:
[0133] Protected public dataset: Each original image x_i is first watermarked, then sorted according to its label. Add the corresponding perturbation η_{ }, generate the final protected image , for public distribution.
[0134] The secret perturbation dictionary: This is the core "key" that is distributed to authorized users through secure channels.
[0135] Phase Two: Distribution, Protection, and Authorized Use. This phase of system operation involves data distribution and different processing flows for the two types of users.
[0136] Data distribution:
[0137] The data owner will have the dataset protected. , Publicly uploaded data (such as cloud storage or websites). This data already includes watermarks and category perturbations.
[0138] Simultaneously, a decryption key package, containing a perturbation dictionary {c:}, is sent to authorized users via secure means (such as encrypted email or a dedicated client). }
[0139] Unauthorized User Process:
[0140] Download protected datasets directly from public sources { , }
[0141] Using it directly to train the model, due to perturbation The features were destroyed, resulting in extremely poor model performance, thus achieving the purpose of protection.
[0142] Authorization User Process:
[0143] Simultaneously obtain publicly available data {} and key package (perturbation dictionary + tag file).
[0144] Data recovery (descrambling): For each image Its category can be determined from the label file. Then retrieve the corresponding perturbation from the perturbation dictionary. Perform subtraction: The result is usable data containing only watermarks and no perturbations, which can be used for normal model training.
[0145] Ownership verification: When it is necessary to verify an image (whether publicly available or private) Or after recovery When claiming copyright, two watermark extractors are used for watermark extraction and verification. As long as one of the extracted watermarks is correct, ownership can be proven.
[0146] Phase 3: Key Design Considerations and Advantages
[0147] High efficiency:
[0148] Storage: Only C perturbation vectors need to be stored (C is the number of categories), instead of the same order of magnitude as the number of images N.
[0149] Processing: Scrambling / descrambling is a simple element-by-element addition and subtraction operation with extremely low computational overhead.
[0150] Reversibility and Key Distribution:
[0151] Class-level design is key to achieving high efficiency and reversibility. Authorized users only need to obtain class-level keys (perturbation dictionaries) instead of processing each image individually.
[0152] The safe distribution is of this lightweight perturbation dictionary, rather than a large, clean dataset.
[0153] System robustness:
[0154] The watermark and perturbation work together. Even if an attacker removes the perturbation through some means (such as illegally obtaining the key), the watermark is still embedded in the image, allowing the source of the leak to be traced.
[0155] The dual extractor ensures that copyright verification can be performed regardless of the state (with or without interference).
[0156] This is understandable; it refers to adversarial training against unauthorized users: unauthorized users using protected images. During model training, unlearnable perturbations can cause a sharp drop in model performance.
[0157] Authorized Use and Traceability: Authorized users obtain category-specific perturbations from the data owner via a secure channel. ,pass A clean, watermarked image is recovered for training. If an authorized user leaks this image, the data owner can still use the first decoder D to extract the watermark and trace its origin.
[0158] The above is a detailed description of the embodiments of the present invention. The following is an experimental verification of the technical effects of the method of the embodiments of the present invention.
[0159] The experimental setup is as follows:
[0160] Datasets: ImageNet-100, Oxford-IIIT Pets
[0161] Evaluation indicators:
[0162] Semantic information protection: Test accuracy and ownership verification after training on protected image datasets using multiple models including Residual Network 18 (ResNet18), the Visual Geometry Group (VGG19) at Oxford University, MobileNet V2, GoogLeNet, and Densely Connected Convolutional Networks 121-layer (Dense121):
[0163] Bit Error Rate (BER): The error rate in watermark extraction, evaluated separately for images with perturbation (BER w / ) and without perturbation (BER w / o). A lower BER is better.
[0164] Peak Signal-to-Noise Ratio (PSNR): Evaluates the difference in visual quality between the protected image and the original image. A higher PSNR indicates better visual quality.
[0165] Table 1. Effects of Semantic Information Protection and Ownership Verification
[0166]
[0167] The values in parentheses represent the training accuracy after removing perturbations for authorized users, used to evaluate reversibility.
[0168] Comparison methods: Generative Adversarial Embedding Attack (GEAA), Adversarial Watermark (Adv-Watermark).
[0169] Embodiment of the present invention: Different disturbance amplitudes are marked ( Constraints Values), including .
[0170] The key experimental results are as follows:
[0171] 1. Semantic information protection performance (overwhelming advantage)
[0172] On ImageNet-100:
[0173] Embodiments of the present invention ( =16): The test accuracy of five different models was reduced from around 70% to around 1% (specific values ranged from 1.16% to 2.03%), and the model predictions exhibited the results of random guessing. This demonstrates that the generated perturbation has extremely strong cross-model transferability and can effectively prevent any unauthorized model learning.
[0174] Reversibility: After authorized users removed the perturbation, the model accuracy recovered to levels close to those trained on the original clean data (values in parentheses, e.g., ResNet18 recovered from 2.03% to 69.42%). This demonstrates the perfect reversibility of the class-level perturbation design, and that the watermark itself has minimal impact on model performance.
[0175] Perturbation amplitude robustness: when the perturbation amplitude is reduced to and Even then, the protective effect remains significant. The average test accuracy of the five models was 6.96% (the specific value ranged from 2.06% to 8.96%).
[0176] Comparison results with the comparative method:
[0177] GEAA: The average accuracy dropped to only 42.35%, indicating limited protection. Its accuracy after authorized recovery also decreased by approximately 6%.
[0178] Adv-Watermark: The average accuracy drops to only 58.18%, and because its watermark is visible and irreversible, it cannot support authorized recovery (no corresponding values in parentheses in the table).
[0179] A consistent trend was observed on Pets: the embodiments of the present invention reduced the accuracy to near-random levels (Pets: ~2.7%), while the accuracy loss after authorization recovery was minimal.
[0180] 2. Ownership verification performance (high robustness and fidelity): the watermarked image is compressed using JPEG75 before the watermark information is extracted.
[0181] On ImageNet-100:
[0182] Method of this invention embodiment (ε=4):
[0183] BER w / (with perturbation): 0.16%, PSNR: 41.25 dB
[0184] BER w / o (undisturbed): 0.22%, PSNR: 48.36dB
[0185] These data demonstrate that the dual watermark extractor can extract watermarks with extremely high accuracy (>99.7%) regardless of whether the image contains highly intrusive, unlearnable perturbations. Simultaneously, the protected image maintains very high visual quality (PSNR >41dB, the difference is imperceptible to the human eye).
[0186] Comparison results with the comparative method:
[0187] GEAA: The watermark extraction BER on undisturbed images is as high as 26.46%, indicating that watermark extraction is essentially a failure. Its PSNR is also low (34.77 dB), suggesting that it sacrifices image quality and watermark robustness for some protection.
[0188] Adv-Watermark: As a visible watermarking method, it has a very low PSNR (28.44dB), obvious visual interference, and does not involve the BER index for extracting invisible watermarks.
[0189] The effect of disturbance amplitude: when When the BER of watermark extraction in this embodiment of the invention is increased from 4 to 16, the image quality decreases.
[0190] Table 2. Robustness of perturbations to JPEF compression
[0191]
[0192] The results in Table 2 show that our method is significantly more robust to JPEG compression when using differentiable JPEG. The accuracy decreased from 63% to 39% (last column, rows 6-7). Furthermore, no significant changes were observed with different quantization factors or without compression (rows 8-9).
[0193] Table 3 Performance Improvements of the Dual Watermark Extractor
[0194]
[0195] Using a dual watermark extractor significantly improves watermark performance. "Single" represents a single watermark extractor, while "dual" and "simultaneous" represent two separate watermark extractors. This represents a dual watermark extractor. "Dual" indicates that a second extractor is randomly initialized. This indicates a fine-tuning of the second extractor. Indicates the magnitude of the perturbation; single indicates the magnitude of the perturbation. When the BER is 16, the extraction accuracy is 2.9%, showing a significant performance degradation. The extraction BER under perturbation interference is 0.81%, significantly improving the watermark extraction accuracy.
[0196] The image dataset copyright protection device provided by the present invention is described below. The image dataset copyright protection device described below and the image dataset copyright protection method described above can be referred to in correspondence.
[0197] like Figure 5 As shown, the present invention provides an image dataset copyright protection device, comprising the following modules:
[0198] The watermark embedding module 500 is used to embed copyright watermark information into the original image dataset to be protected through the watermark encoder, thereby generating a watermarked image dataset.
[0199] The first generation module 510 is used to generate a reversible, non-learnable perturbation dataset based on the watermarked image dataset by minimizing the mutual information between the input and output of the victim model.
[0200] The second generation module 520 is used to add the non-learnable perturbation dataset to the watermarked image dataset to generate a protected image dataset.
[0201] Extraction module 530 is used to extract the copyright watermark information from the watermarked image dataset using a first watermark decoder, and to extract the copyright watermark information from the protected image dataset using a second watermark decoder.
[0202] The first watermark decoder is trained based on the watermarked image dataset; the second watermark decoder is trained based on the protected image dataset.
[0203] According to the image dataset copyright protection device provided by the present invention, the non-learnable perturbation dataset is a perturbation dictionary generated according to image categories, and all images under the same image category share the same perturbation template.
[0204] According to the image dataset copyright protection device provided by the present invention, the watermark encoder and the first watermark decoder are obtained through joint training, the joint training including:
[0205] The watermark encoder and the first watermark decoder are trained by using a joint loss function and a discriminator.
[0206] The joint loss function includes: image fidelity loss between the watermarked image dataset and the original image dataset; extraction loss of the first watermark decoder for extracting the copyright watermark information; and discrimination loss of the discriminator for the watermarked image dataset.
[0207] The discriminator and the watermark encoder form a generative adversarial network, and the discriminator is used to distinguish the watermarked image dataset generated by the watermark encoder from the original image dataset.
[0208] According to the image dataset copyright protection device provided by the present invention, the process of generating the perturbation dictionary adopts an alternating training strategy and repeatedly executes the following steps until a preset stopping condition is met:
[0209] With the current perturbation dictionary fixed, the victim model is trained using the protected image dataset;
[0210] The victim model, after being trained, generates a corresponding personalized perturbation for each image in the watermarked image dataset by minimizing the mutual information between the input and output of the victim model.
[0211] The personalized perturbations of all images under the same image category are averaged to obtain the perturbation template of the image category, and the perturbation dictionary is updated.
[0212] According to the image dataset copyright protection device provided by the present invention, the protected image dataset is JPEG compressed before training the victim model using the protected image dataset during the generation of the perturbation dictionary.
[0213] According to the image dataset copyright protection device provided by the present invention, the watermarked image dataset is JPEG compressed before the first decoder extracts the copyright watermark information; and / or, the protected image dataset is JPEG compressed before the second decoder extracts the copyright watermark information.
[0214] According to the image dataset copyright protection device provided by the present invention, minimizing the mutual information between the input and output of the victim model specifically includes:
[0215] Achieve this through the following objective function:
[0216]
[0217] in, Represents a watermarked image dataset The i-th watermarked image in the image; express Corresponding personalized perturbations; For the victim model; ε represents the differentiable JPEG compression module; ε represents the perturbation amplitude constraint.
[0218] According to the image dataset copyright protection device provided by the present invention, the first watermark decoder and the second watermark decoder have the same structure; and the second watermark decoder is trained using the parameters of the first watermark decoder as the initial state.
[0219] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the image dataset copyright protection method provided by the methods described above.
[0220] In another aspect, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the image dataset copyright protection method provided by the above methods.
[0221] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0222] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for protecting the copyright of an image dataset, characterized in that, include: The copyright watermark information is embedded into the original image dataset to be protected by the watermark encoder to generate a watermarked image dataset. Based on the watermarked image dataset, a reversible, non-learnable perturbation dataset is generated by minimizing the mutual information between the input and output of the victim model. The non-learnable perturbation dataset is added to the watermarked image dataset to generate a protected image dataset. The copyright watermark information is extracted from the watermarked image dataset using a first watermark decoder, and the copyright watermark information is extracted from the protected image dataset using a second watermark decoder. The first watermark decoder is trained based on the watermarked image dataset. The second watermark decoder is trained based on the protected image dataset.
2. The image dataset copyright protection method according to claim 1, characterized in that, The non-learnable perturbation dataset is a perturbation dictionary generated according to image categories, and all images under the same image category share the same perturbation template.
3. The image dataset copyright protection method according to claim 1, characterized in that, The watermark encoder and the first watermark decoder are obtained through joint training, and the joint training includes: The watermark encoder and the first watermark decoder are trained by using a joint loss function and a discriminator. The joint loss function includes: image fidelity loss between the watermarked image dataset and the original image dataset; extraction loss of the first watermark decoder for extracting the copyright watermark information; and discrimination loss of the discriminator for the watermarked image dataset. The discriminator and the watermark encoder form a generative adversarial network, and the discriminator is used to distinguish the watermarked image dataset generated by the watermark encoder from the original image dataset.
4. The image dataset copyright protection method according to claim 2, characterized in that, The process of generating the perturbation dictionary employs an alternating training strategy and repeatedly executes the following steps until a preset stopping condition is met: With the current perturbation dictionary fixed, the victim model is trained using the protected image dataset; The victim model, after being trained, generates a corresponding personalized perturbation for each image in the watermarked image dataset by minimizing the mutual information between the input and output of the victim model. The personalized perturbations of all images under the same image category are averaged to obtain the perturbation template of the image category, and the perturbation dictionary is updated.
5. The image dataset copyright protection method according to claim 4, characterized in that, During the generation of the perturbation dictionary, the protected image dataset is JPEG compressed before training the victim model using the protected image set.
6. The image dataset copyright protection method according to claim 1, characterized in that, Before the first decoder extracts the copyright watermark information, the watermarked image dataset is JPEG compressed; and / or, before the second decoder extracts the copyright watermark information, the protected image dataset is JPEG compressed.
7. The image dataset copyright protection method according to claim 5, characterized in that, Minimizing the mutual information between the input and output of the victim model specifically includes: Achieve this through the following objective function: ； in, Represents a watermarked image dataset The i-th watermarked image in the image; express Corresponding personalized perturbations; For the victim model; ε represents the differentiable JPEG compression module; ε represents the perturbation amplitude constraint.
8. The image dataset copyright protection method according to claim 1, characterized in that, The first watermark decoder and the second watermark decoder have the same structure; and the second watermark decoder is trained using the parameters of the first watermark decoder as the initial state.
9. A copyright protection device for image datasets, characterized in that, include: The watermark embedding module is used to embed copyright watermark information into the original image dataset to be protected through the watermark encoder, generating a watermarked image dataset. The first generation module is used to generate a reversible, non-learnable perturbation dataset based on the watermarked image dataset by minimizing the mutual information between the input and output of the victim model. The second generation module is used to add the non-learnable perturbation dataset to the watermarked image dataset to generate a protected image dataset. The extraction module is used to extract the copyright watermark information from the watermarked image dataset using a first watermark decoder, and to extract the copyright watermark information from the protected image dataset using a second watermark decoder. The first watermark decoder is trained based on the watermarked image dataset. The second watermark decoder is trained based on the protected image dataset.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image dataset copyright protection method as described in any one of claims 1 to 8.