Generation method of right-confirmable confrontation sample with image copyright protection function

Abstract

The invention discloses a method for generating a certifiable confrontation sample with an image copyright protection function, and belongs to the technical field of machine learning. Comprises: acquiring an image sample set; an adversarial sample generation model is built, the adversarial sample generation model comprises an encoder, a decoder, a noise layer, a discriminator and a target classifier, and channel attention mechanism modules are introduced into the encoder and the decoder; and inputting the image sample set into an adversarial sample generation model for training, and when the adversarial sample generation model is trained, the adversarial sample containing copyright information output by the coding module is a right-confirmable adversarial sample with an image copyright protection function. According to the method, the copyright information is embedded into the image, and the adversarial training is utilized to generate the right-confirmable adversarial sample with the image copyright protection function, so that the copyright protection of the image is realized. By adding the noise layer in the training process, the robustness of the adversarial sample is improved, so that the adversarial sample can effectively cope with noise interference in different network environments.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a method for generating authentic adversarial samples with an image copyright protection function. Background Art

[0002] Deep learning, as an advanced artificial intelligence technology, has not only solved major challenges in scientific research and industrial production, but has also achieved tremendous success in social networking and computer vision. With the rapid development of deep learning, image content security has become a critical issue, particularly in areas such as image privacy and copyright protection. Among these issues, adversarial example technology has garnered widespread attention in the field of image privacy protection. However, current deep learning-based image privacy protection systems exhibit significant vulnerabilities to adversarial examples. While some adversarial attack methods that are effective in natural images can also be applied to image privacy protection, the diversity and feature differences in image privacy protection data make it difficult to directly transfer adversarial attack methods from natural images to privacy-preserving images. Existing adversarial example privacy protection methods still have several challenges: first, adversarial perturbations can potentially degrade the visual quality of images; second, existing methods focus primarily on privacy protection and struggle to simultaneously address image authenticity, integrity, and copyright protection. Therefore, current adversarial example technology can only address a single privacy issue and cannot fully meet image security requirements. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this paper leverages the global pixel properties of the underlying image and designs a verifiable adversarial example model with image copyright protection capabilities based on CGAN and a channel attention mechanism. This model not only effectively embeds copyright information and ensures its extraction accuracy, but also exhibits strong robustness against interference in diverse noise environments, thereby providing a more comprehensive approach to image privacy and copyright protection in online environments.

[0004] To achieve the above objectives, the present invention provides a method for generating authentic adversarial samples with image copyright protection, comprising the following steps: (1) Obtain image sample set; (2) Building an adversarial sample generation model, which includes an encoder, a decoder, a noise layer, a discriminator, and a target classifier. The encoder and decoder are integrated with a channel attention mechanism module. The encoder receives an image sample, a threshold image of the image sample, and binary copyright information of the image sample, and outputs an adversarial sample containing the copyright information; the noise layer distorts the adversarial sample to generate a noisy image; the target classifier receives the noisy image and outputs a corresponding classification label; the decoder extracts the copyright information from the noisy adversarial sample; the discriminator receives the adversarial sample containing the copyright information and returns a detection result; (3) The image sample set is input into the adversarial sample generation model for training. When the adversarial sample generation model is trained, the adversarial sample containing copyright information output by the encoding module is a verifiable adversarial sample with image copyright protection function.

[0005] Furthermore, the image samples are processed using the Otsu-thresholding method, where the smooth areas in the image are assigned to 0 and the texture areas are assigned to 1, thereby obtaining a threshold image of the image samples.

[0006] Furthermore, the encoder is based on the Dense-Net network architecture, which consists of three DensBlocks and a first convolution block connected in sequence; a channel attention module is added after each DensBlock, wherein each DensBlock consists of a second convolution block, a batch normalization module and an activation function; The image sample and the threshold image of the image sample are spliced ​​and input into the first DensBlock. The binary copyright information of the image sample is expanded and reshaped, and then connected with the output of each channel attention module and the output of all previous DensBlocks after the channel attention module, and then input into the next DensBlock or the first convolution block. Finally, the output of the first convolution block is added to the image sample pixel by pixel as the output of the encoder.

[0007] Furthermore, the activation function in the encoder adopts the LeakyReLU activation function; the convolution kernels of the first convolution block and the second convolution block are 3×3, and the image pixel padding and step size are both 1.

[0008] Furthermore, the decoder structure consists of four DensBlocks connected in sequence, and a channel attention module is added after the first three DensBlocks; the first three DensBlocks of the decoder are the same as the DensBlock of the encoder, and an adaptive average pooling layer is added at the end of the fourth DensBlock.

[0009] Furthermore, the discriminator includes four DensBlocks and a linear layer connected in sequence. The first three DensBlocks of the discriminator are the same as the DensBlock of the encoder. The batch normalization module and activation function are removed from the last DensBlock, and an adaptive average pooling layer is added. After the input image passes through the four DensBlocks, it is processed by the linear layer and the judgment result is output.

[0010] Furthermore, the target classifier uses four classification models: ResNet50, Inception-v3, EfficientNet, and DenseNet121 as attack models.

[0011] Furthermore, the noise layer adopts cropping, cropping removal, dropout, JPEG compression and Gaussian blur.

[0012] Furthermore, the total loss function of the adversarial sample generation model is: in: and Represents the weight of each loss, Represents the carrier image and represents adversarial examples, Represents copyright information, represents the extracted copyright information, A represents the judge, Represented as an image The true label, Represents the predicted image The label is The probability of , C represents the channel, H represents the height of the carrier image, and W represents the width of the carrier image.

[0013] The present invention also provides a system for generating authentic adversarial samples with image copyright protection, comprising: A data acquisition module, used to acquire an image sample set; A model building module is used to build an adversarial sample generation model. The adversarial sample generation model includes an encoder, a decoder, a noise layer, a discriminator, and a target classifier. The encoder and decoder introduce a channel attention mechanism module. The encoder receives an image sample, a threshold image of the image sample, and binary copyright information of the image sample, and outputs an adversarial sample containing the copyright information; the noise layer distorts the adversarial sample to generate a noisy image; the target classifier receives the noisy image and outputs a corresponding classification label; the decoder extracts the copyright information from the noisy adversarial sample; the discriminator receives the adversarial sample containing the copyright information and returns a detection result; A model training module, configured to input an image sample set into the adversarial sample generation model for training; The sample generation module is used to ensure that after the training of the adversarial sample generation model is completed, the adversarial sample containing copyright information output by the encoding module is a verifiable adversarial sample with image copyright protection function.

[0014] Beneficial effects of the present invention: This paper proposes a model that combines CGAN with a channel-attention mechanism. This model embeds copyright information into images and uses adversarial training to generate verifiable adversarial examples with copyright protection capabilities, thereby achieving copyright protection for images. By incorporating a noise layer into the training process, the robustness of the adversarial examples is improved, making them more resilient to noise interference in various network environments. Furthermore, by integrating digital watermarking technology, the risk of malicious collection and removal of copyright information is effectively prevented, further enhancing image privacy protection capabilities and providing a new solution for image protection in open network environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the flow of a method for generating authentic adversarial samples with image copyright protection function according to an embodiment of the present invention.

[0016] Figure 2 Schematic diagram of the adversarial sample generation model structure according to an embodiment of the present invention.

[0017] Figure 3 Schematic diagram of the encoder structure according to an embodiment of the present invention.

[0018] Figure 4 Schematic diagram of the decoder structure of an embodiment of the present invention.

[0019] Figure 5 Schematic diagram of the structure of the decision device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further explained and illustrated below with reference to the accompanying drawings and embodiments.

[0021] like Figure 1 As shown in FIG, a method for generating an authentic adversarial sample with image copyright protection function includes the following steps: S101: Obtain an image sample set.

[0022] The present invention uses the Caltech256 dataset for model training and testing. 24,000 images are randomly selected from the dataset as the training set, and 3,000 images are selected as the test set.

[0023] S102. Build an adversarial sample generation model, which includes an encoder, a decoder, a noise layer, a discriminator, and a target classifier. A channel attention mechanism module is introduced into the encoder and decoder.

[0024] like Figure 2 As shown in the figure, the model framework mainly consists of five parts: encoder E, decoder D, noise layer N, discriminator A and target classifier T. Its working principle is: (1) Encoder E receives the carrier image , threshold image and binary copyright information , and output an aggressive adversarial sample .

[0025] (2) Noise layer N receives adversarial samples , randomly select a noise attack to distort the adversarial sample to generate a noise image .

[0026] (3) Decoder D from the adversarial sample containing noise Extract copyright information .

[0027] (4) Discriminator A receives the image , detect the input image yes or , and finally returns the test results.

[0028] (5) The target classifier T receives the image after noise processing It takes as input and outputs the corresponding classification label.

[0029] The following is a detailed introduction to each module: The encoder E adopts a network architecture similar to Dense-Net, such as Figure 3As shown, the network consists of three sequentially connected DensBlocks and the first convolutional block; each DensBlock is followed by a channel attention module, which consists of a second convolutional block, a batch normalization module, and an activation function. The image sample and its thresholded image are concatenated and fed into the first DensBlock. The binary copyright information of the image sample is expanded and reshaped, then concatenated with the output of each channel attention module and the output of all previous DensBlocks after the channel attention module. This information is then fed into the next DensBlock or the first convolutional block. Finally, the output of the first convolutional block is added pixel by pixel to the image sample to form the encoder output.

[0030] The Dense-Net architecture ensures feature reuse and efficient information transfer by connecting each layer with the feature data extracted by all previous layers, while reducing the gradient vanishing problem and expanding the number of channels in each layer, which helps the network capture a wider variety of features in more images, such as edges, colors, textures, etc. At the same time, a channel attention module is added to each layer to adaptively assign different weights to each channel of the image, thereby enhancing the weights of important feature channels and improving the model's ability to select image features, thereby generating suitable weighted adversarial samples.

[0031] The activation function in the encoder uses the LeakyReLU activation function with a negative slope of 0.01, which can give a non-zero output when the input data is negative, avoiding the death of neurons and improving the learning ability of the model.

[0032] The convolution kernel size of the first and second convolution blocks is 3×3, and the image pixel padding and stride size are both 1.

[0033] In the encoder E framework, a color carrier image with C channels and image size is , threshold image and a binary copyright message of length As input, the copyright information is embedded into the carrier image to generate adversarial samples containing copyright information. , which can be expressed as: in: Represents the parameters of the encoder; the threshold image is obtained by processing the color carrier image using the Otsu-thresholding method, where the smooth area in the image is assigned to 0 and the texture area is assigned to 1.

[0034] The structure of decoder D is similar to that of encoder E, such as Figure 4As shown in the figure, it consists of four DensBlocks connected in sequence. The first three DensBlocks of the decoder are the same as those of the encoder. An adaptive average pooling layer is added to the fourth DensBlock at the end to reduce the number of model parameters. At the same time, the image feature map is reduced to 1×1 and its shape is compressed to restore the original shape of the copyright information. Like the encoder, a channel attention module is added after the first three DensBlocks to enable the model to learn the embedded location of the copyright information.

[0035] Noise layer N: The embodiment of the present invention introduces five types of noise for training, namely cropping, cropping removal, dropout, JPEG compression and Gaussian blur, and modifies the parameters of the noise to observe the interference on the model.

[0036] Decoder D will contain noise interference adversarial samples As input, extract the copyright information hidden in the image , which can be expressed as: in: Denote the network parameters of the decoder, The copyright of an image is confirmed by extracting the copyright information. Therefore, the goal of optimizing the decoder is to maximize the accuracy of extracting copyright information.

[0037] Discriminator A, such as Figure 5 As shown in the figure, it includes four DensBlocks and linear layers connected in sequence. Among them, the first three DensBlocks are the same as the DensBlock of the encoder. The batch normalization module and activation function are removed from the last DensBlock, and an adaptive average pooling layer is added to reduce the number of feature parameters and modify the size of the features. Finally, the judgment result is output through the linear layer processing (0 represents the carrier image and 1 represents the weighted adversarial sample).

[0038] During the training process, the discriminator A will take the carrier image or images containing copyright information As input, it accurately determines whether the input image contains copyright information. During the training process of embedding copyright information, through adversarial training with the discriminator A, the encoder E is forced to continuously improve the strategy and location selection of information embedding, and improve the discriminator's ability to detect whether copyright information is contained. Ultimately, the encoder can generate more covert adversarial examples that are difficult for the human eye to detect.

[0039] In this embodiment of the present invention, the target classifier uses four classification models: ResNet50, Inception-v3, EfficientNet, and DenseNet121 as attack models. To ensure that the generated adversarial samples can mislead the classification model's predictions, we introduce a pre-trained classification model into the framework. The image is input into the pre-trained classification model, and the model then outputs a predicted classification label. The classification label is then observed to see if it is the same as the original label. If they are different, it means that the image attack is successful, causing the model to predict the classification incorrectly. Otherwise, it means that the attack has failed. The parameters of different target attack networks are shown in Table 1.

[0040] Table 1 The adversarial sample generation model designed in the embodiment of the present invention uses a total of four loss functions during the training process.

[0041] 1) During adversarial training, copyright information is embedded into the carrier image, making the generated adversarial sample difficult to discern in appearance. Therefore, it is necessary to ensure that the quality of the generated image does not suffer from severe distortion. To address this issue, we use mean square error loss to measure the difference between the carrier image and the adversarial sample pixels, thereby improving the image visual quality, as shown in the following formula: 2) In the process of image copyright verification, the decoder D is needed to extract the copyright information from the noisy image, and the difference between the copyright information and the extracted copyright information needs to be minimized. Therefore, the mean square error loss is used to calculate the error between them, as shown in the following formula: 3) To generate adversarial examples, the encoder E and the discriminator A engage in a "game" to ensure that the copyright information in the generated adversarial examples is invisible. In this paper, we train the discriminator A not only to determine whether an image contains copyright information, but also to improve the information embedding strategy and image quality. The training goal is to make the generated adversarial examples mislead the discriminator's classification, causing the binary discriminator to output incorrect labels. To calculate this, we use a binary cross-entropy adversarial loss function, as shown in the following formula: 4) To ensure the security of user image information from the source of the network, we need to effectively counter the powerful retrieval and classification capabilities of DNNs to prevent images from being exploited and tampered. Therefore, we design a verifiable adversarial example with image copyright protection. The goal is to maximize the difference between the original label and the predicted label, thereby misleading the output of the target classifier T, causing the classifier to output the wrong label, as shown in the following formula: The overall goal of training the entire network framework is to minimize the total loss function so that the image embedded with binary copyright information can meet the requirements of four aspects: image quality, copyright information invisibility, information extraction and adversarial attack. Therefore, the total loss function can be expressed as: in: and Represents the weight of each loss, Represents the carrier image and represents adversarial examples, Represents copyright information, represents the extracted copyright information, A represents the judge, Represented as an image The true label, Represents the predicted image The label is The probability of , C represents the channel, H represents the height of the carrier image, and W represents the width of the carrier image.

[0042] S103. Input the image sample set into the adversarial sample generation model for training. When the adversarial sample generation model training is completed, the adversarial sample containing copyright information output by the encoding module is a verifiable adversarial sample with image copyright protection function.

[0043] During the training process, the image sample set was first preprocessed by scaling, rotating, and normalizing the images to increase data diversity and improve the robustness of the model. The embedded binary copyright information was sampled from a Bernoulli distribution with a probability of 0.5. The model was trained for 40 epochs with a learning rate of 2×10 -4 The Adam optimization algorithm and learning rate decay strategy improve model training efficiency. For the weight parameters of the loss function for model training, this embodiment sets θ = 1000, θ = 10, θ = 100, and θ = 1, and sets the intensity factor to 1. Testing and verification were performed using the PyTorch platform on an NVIDIA GeForce RTX 3090 GPU.

[0044] Due to the limited research on adversarial examples with copyright protection capabilities, and to enhance experimental comparison, this paper conducts phased experiments comparing popular deep learning-based watermark image generation methods (HiDDeN, TSDL, and MBRS) with adversarial attack methods (AdvGAN, C&W, FGSM, and PGD). Ultimately, adversarial examples containing copyright information are generated and compared with the model proposed in this embodiment. The success rate of adversarial examples against the attack model, peak signal-to-noise ratio extraction accuracy, and hidden data extraction accuracy are used as measurement metrics.

[0045] The attack success rate (ASR) is calculated by dividing the number of times an adversarial example successfully deceives the model by the total number of adversarial examples tested, as follows: Where: the numerator represents the number of samples with successful attacks, and the denominator represents the total number of image samples. Represents a logical AND operation. The value range of ASR is [0, 1]. A higher value means a better attack effect on the image.

[0046] Peak signal-to-noise ratio is used as a measure of the similarity between the carrier image and the generated adversarial sample, calculated by mean square error (MSE) Size of carrier image and adversarial examples The difference between each pixel value is obtained: Where: MSE is the maximum possible range of image pixel values. In lossy image compression, PSNR typically ranges from 30 to 50 dB, with larger values ​​indicating a closer resemblance between the lossy image and the carrier image, and smaller values ​​indicating a more severe lossy image distortion.

[0047] The accuracy of hidden data extraction is With the original ciphertext The sum of the equal digits divided by the total number of digits in the ciphertext: in: Is a logical indicator function, when the ciphertext Equal to the extracted ciphertext When , the output is 1. A higher accuracy means that the extracted ciphertext is closer to the original ciphertext. Conversely, the error rate (BER) is the probability of extracting incorrect information: In experiments involving the generation of adversarial examples in stages, if the adversarial attack experiment is first performed and then information embedding is performed to generate adversarial examples, the second stage of information embedding may destroy the key adversarial information in the first stage of the adversarial example, causing the network's misjudgment of the target class to weaken or even disappear, thereby reducing the effectiveness of the attack and affecting the success rate of the final generated adversarial example. Therefore, the method of first performing information embedding and then conducting the adversarial attack is adopted. The experimental results are shown in Table 2.

[0048] Table 2 As can be seen from this, in the experiments combining TSDL and FGSM, the BER of the experiments conducted on the four classifiers was greater than 40, making it impossible to accurately extract information. Compared with these experiments, the ASR of the method proposed in the present invention on different classifiers exceeded 98%, the BER was very close to 0, and the quality of the generated adversarial sample images was also the best. Therefore, the adversarial samples generated by the network structure proposed in the embodiment of the present invention have very outstanding performance in information extraction and adversarial attacks.

[0049] To verify the effectiveness of the channel attention module on the adversarial sample generation model, we conducted an ablation experiment. While keeping other network models and parameters unchanged, we eliminated the attention mechanism modules in the encoder and decoder and set the watermark length to 60. We then compared the PSNR, ASR, and ACC results of the ablation experiment with the original model. The experimental results are shown in Table 3.

[0050] Table 3 index Method of the present invention Ablation experiments PSNR 37.5 30.8 ASR 98.2% 85.6% ACC 99.82% 97.15% It can be found that the experimental comparison index results of the model with the channel attention mechanism are better than those of the model without the attention mechanism, which verifies that the channel attention module can enhance the attack capability of adversarial samples and use the correlation between image channels to embed data into the appropriate channel, thereby adjusting the effect of the embedded data on the image.

[0051] The above specific description further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating authentic adversarial samples with image copyright protection, characterized by: The steps include: (1) Obtain image sample set; (2) Building an adversarial sample generation model, which includes an encoder, a decoder, a noise layer, a discriminator, and a target classifier. The encoder and decoder are integrated with a channel attention mechanism module. The encoder receives an image sample, a threshold image of the image sample, and binary copyright information of the image sample, and outputs an adversarial sample containing the copyright information; The noise layer distorts the adversarial sample to generate a noisy image; The target classifier receives the noise-processed image and outputs the corresponding classification label; The decoder extracts copyright information from noisy adversarial examples; The discriminator receives adversarial samples containing copyright information and returns detection results; (3) The image sample set is input into the adversarial sample generation model for training. When the adversarial sample generation model is trained, the adversarial sample containing copyright information output by the encoding module is a verifiable adversarial sample with image copyright protection function.

2. The method for generating authentic adversarial examples with image copyright protection according to claim 1, characterized in that: The image samples are processed using the Otsu-thresholding method, whereby smooth regions in the image are assigned a value of 0 and texture regions are assigned a value of 1, thereby obtaining a threshold image of the image samples.

3. The method for generating authentic adversarial examples with image copyright protection according to claim 1, characterized in that: The encoder is based on the Dense-Net network architecture, consisting of three DensBlocks and a first convolutional block connected in sequence; each DensBlock is followed by a channel attention module, wherein each DensBlock consists of a second convolutional block, a batch normalization module, and an activation function; The image sample and the threshold image of the image sample are spliced ​​and input into the first DensBlock. The binary copyright information of the image sample is expanded and reshaped, and then connected with the output of each channel attention module and the output of all previous DensBlocks after the channel attention module, and then input into the next DensBlock or the first convolution block. Finally, the output of the first convolution block is added to the image sample pixel by pixel as the output of the encoder.

4. The method for generating authentic adversarial examples with image copyright protection according to claim 3, characterized in that: The activation function in the encoder adopts the LeakyReLU activation function; the convolution kernels of the first convolution block and the second convolution block are 3×3, and the image pixel padding and step size are both 1.

5. The method for generating authentic adversarial examples with image copyright protection according to claim 1, characterized in that: The decoder structure consists of four DensBlocks connected in sequence, and a channel attention module is added after the first three DensBlocks; the first three DensBlocks of the decoder are the same as the DensBlock of the encoder, and an adaptive average pooling layer is added at the end of the fourth DensBlock.

6. The method for generating authentic adversarial examples with image copyright protection according to claim 1, characterized in that: The discriminator includes four DensBlocks and a linear layer connected in sequence. The first three DensBlocks of the discriminator are the same as the DensBlock of the encoder. The batch normalization module and activation function are removed from the last DensBlock, and an adaptive average pooling layer is added. After the input image passes through the four DensBlocks, it is processed by the linear layer and the judgment result is output.

7. The method for generating authentic adversarial examples with image copyright protection according to claim 1, characterized in that: The target classifier uses four classification models: ResNet50, Inception-v3, EfficientNet, and DenseNet121 as attack models.

8. The method for generating authentic adversarial examples with image copyright protection according to claim 1, characterized in that: The noise layer adopts cropping, cropping removal, dropout, JPEG compression and Gaussian blur.

9. The method for generating authentic adversarial samples with image copyright protection function according to claim 1, characterized in that: The total loss function of the adversarial sample generation model is: in: and Represents the weight of each loss, Represents the carrier image and represents adversarial examples, Represents copyright information, represents the extracted copyright information, A represents the judge, Represented as an image The true label, Represents the predicted image The label is The probability of , C represents the channel, H represents the height of the carrier image, and W represents the width of the carrier image.

10. A system for generating authentic adversarial samples with image copyright protection, characterized in that: include: A data acquisition module, used to acquire an image sample set; A model building module is used to build an adversarial sample generation model. The adversarial sample generation model includes an encoder, a decoder, a noise layer, a discriminator, and a target classifier. The encoder and decoder introduce a channel attention mechanism module. The encoder receives an image sample, a threshold image of the image sample, and binary copyright information of the image sample, and outputs an adversarial sample containing the copyright information; The noise layer distorts the adversarial sample to generate a noisy image; The target classifier receives the noise-processed image and outputs the corresponding classification label; The decoder extracts copyright information from noisy adversarial examples; The discriminator receives adversarial samples containing copyright information and returns detection results; A model training module, configured to input an image sample set into the adversarial sample generation model for training; The sample generation module is used to ensure that after the training of the adversarial sample generation model is completed, the adversarial sample containing copyright information output by the encoding module is a verifiable adversarial sample with image copyright protection function.

Images