A method for generating image adversarial watermark based on particle swarm algorithm

Abstract

The present invention discloses a method for generating an image adversarial watermark based on a particle swarm algorithm, comprising: combining an original image with a watermark through alpha blending to generate a watermark image according to the position and transparency information of the watermark; inputting the generated watermark image into a target neural network to obtain information on each classification confidence output by the target neural network; and obtaining an adversarial watermark image through a particle swarm algorithm based on each classification confidence of the watermark image. The present invention models the adversarial watermark generation problem as an optimization problem, and generates an adversarial watermark through the result of the optimization problem. The particle swarm algorithm is used to efficiently solve the optimization problem. The confidence difference is used as the optimization target to avoid missing adversarial watermark generation to a certain extent.

Description

Technical Field

[0001] The invention belongs to the technical field of network security and machine learning, and specifically relates to an image adversarial watermark generation method based on a particle swarm algorithm. Background Art

[0002] Adversarial samples are images that add noise that is difficult for the human eye to detect, causing the machine to produce incorrect classification results. Image adversarial watermarks use adversarial sample technology to protect images and prevent them from being stolen by malicious programs. Compared with traditional image watermarks, image adversarial watermarks are adversarial and can cause image recognition models to produce incorrect classification results.

[0003] Image adversarial watermarks have two uses in actual scenarios. First, as a visible watermark added to an image, it can declare the copyright information of the image to prevent it from being stolen. Second, as an adversarial image, when a malicious program collects images, the adversarial nature of the adversarial watermark causes the malicious program to produce incorrect classification results, interfering with the malicious program's collection of image information. The image adversarial watermark generation problem can be transformed into a constrained optimization problem. However, the search space of this optimization problem is relatively large. If it is directly generated by enumeration, it will cause a huge waste of performance.

[0004] In view of the above problems, how to design a reasonable optimization goal and use the intelligent optimization algorithm to optimize the problem and generate image adversarial watermarks is an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide an image adversarial watermark generation method based on particle swarm algorithm to improve the efficiency and adversarial property of adversarial watermark generation.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for generating an image adversarial watermark based on a particle swarm algorithm, the method comprising:

[0008] Step 1: According to the position and transparency information of the watermark, the original image and the watermark are combined through alpha blending to generate a watermark image;

[0009] Step 2: Input the generated watermark image into the target neural network to obtain the information of each classification confidence output by the target neural network;

[0010] Step 3: Based on the confidence of each classification of the watermark image, the adversarial watermark image is obtained by using the particle swarm algorithm, including:

[0011] Step 31: Convert the adversarial watermark generation problem into an optimization problem of finding (p, q, α, scale) values ​​so that the confidence of the generated adversarial watermark image is non-maximum, and establish the optimization objective as:

[0012] minimize fit(p,q,α,scale)=f t (g(p,q,α,scale))-f s (g(p,q,α,scale))

[0013] In the formula, minimizing the fitness function fit is used as the optimization goal, f t (g(p,q,α,scale)) represents the confidence of the correct classification label t obtained for the watermark image classification, f s (g(p,q,α,scale)) represents the maximum confidence value of other classification labels except the correct classification label t obtained for the classification of the watermarked image. The g(p,q,α,scale) function is the watermark generation function, (p,q) is the position coordinate of the watermark in the original image, α is the transparency value of the watermark, and scale is the ratio of the length and width of the watermark to the length and width of the original image;

[0014] Step 32, Particle Swarm Algorithm Initialization: During initialization, each particle in the particle swarm is randomly distributed in the D-dimensional space with random speed and position. The position of particle i at the kth iteration is recorded as The velocity of particle i at the kth iteration is recorded as The position of each particle is composed of (p, q, α, scale), representing different adversarial watermark images;

[0015] Step 33, iteratively update the particle swarm until the maximum number of iterations is reached and then stop, and output the position and transparency information of the watermark in each iteration;

[0016] Step 34: After stopping the iteration, the particle swarm algorithm outputs the optimal (p, q, α, scale) value, which is input into the watermark generation function g(p, q, α, scale) to generate an adversarial watermark image with an image adversarial watermark.

[0017] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0018] Preferably, the step of combining the original image with the watermark to generate the watermarked image by alpha blending comprises:

[0019] The pixels where the watermark overlaps with the original image are calculated as follows:

[0020] v(G)=(v(W)*α+v(X)*255) / 255

[0021] Where v(G) is the pixel value of the overlapping pixel, v(W) is the pixel value of the watermark pixel, and v(X) is the pixel value of the original image pixel.

[0022] Preferably, the target neural network is a classification model f, the correct classification label of the image is t, and the confidence of label t is f t There are n categories in total, and the confidence of each category output is f 1…n .

[0023] Preferably, the iterative updating of the particle swarm until the maximum number of iterations is reached and then stops, and the position and transparency information of the watermark are outputted in each iteration, including:

[0024] Step 331, calculate the fitness value of each particle, and the fitness function fit expression is as follows:

[0025] fit(p,q,α,scale)=f t (g(p,q,α,scale))-f s (g(p,q,α,scale))

[0026] The historical optimal position experienced by particle i in the previous k generations of iterations is recorded as The global optimal position experienced by the entire population during the first k generations of iterations is recorded as

[0027] Step 332: Compare the calculated fitness values. If the current fitness value of the particle is better than the fitness value of the historical optimal position of the particle, update the historical optimal position to the current position of the particle.

[0028] Step 333: compare the fitness value of the particle with the fitness value of the current global optimal position. If the fitness value of the particle is better than the fitness value of the global optimal position, update the global optimal position to the position of the particle.

[0029] Step 334: Update the velocity and position of the particle. The update formula is as follows:

[0030]

[0031]

[0032] Where k is the current iteration number of the particle, d is the current dimension of the particle, is the velocity of particle i in dimension d at the kth iteration, c1 and c2 are acceleration coefficients, indicating that the particle flies to the individual optimal and the population optimal g k The acceleration coefficient when , r1 and r2 are two random functions used to increase the randomness during the search, and w is the inertia weight;

[0033] Step 335, the number of iterations k is increased by 1. If the number of iterations is greater than the maximum number of iterations T, the iteration is stopped; otherwise, the process returns to step 331 to continue execution.

[0034] As a preferred method, in the iterative update of the particle swarm, before executing step 332, it is first determined whether f t <f s The size of f t <f s , that is, when the value of the function fit is less than 0, the iteration is terminated early; otherwise, step 332 is executed.

[0035] The image adversarial watermark generation method based on particle swarm algorithm provided by the present invention models the adversarial watermark generation problem as an optimization problem, and generates the adversarial watermark through the result of the optimization problem. The particle swarm algorithm is used to efficiently solve the optimization problem. The confidence difference is used as the optimization target to avoid missing the adversarial watermark generation to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the image adversarial watermark generation method based on particle swarm algorithm of the present invention;

[0037] Figure 2 It is an iterative flow chart of the particle swarm algorithm of the present invention;

[0038] Figure 3 This is a diagram showing the anti-watermark effect of images in the experiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0041] In order to overcome the shortcomings of adversarial watermark generation in the prior art, this embodiment provides an image adversarial watermark generation method based on a particle swarm algorithm. The adversarial watermark generation problem is converted into an optimization problem through modeling. The particle swarm algorithm is used for optimization iteration, and the confidence difference is used as the optimization target to generate an image adversarial watermark.

[0042] like Figure 1 As shown, the image adversarial watermark generation method based on particle swarm algorithm of this embodiment includes:

[0043] Step 1: Image watermark generation based on alpha blending: According to the position and transparency information of the watermark, the original image and the watermark are combined through alpha blending to generate a watermark image.

[0044] First, this embodiment defines the watermark parameters. The following parameters control the watermark generation: First, the coordinate parameters (p, q) that determine the position of the watermark in the original image represent the position of the upper left corner of the watermark in the original image. Second, the transparency value α controls the alpha channel of the watermark image to make the watermark transparent. Finally, the watermark size scale represents the ratio of the length and width of the watermark to the length and width of the original image.

[0045] Alpha blending to generate watermark: When generating a watermark image, according to the given transparency value α, alpha blending is used to generate a watermark image with a given transparency value α. Formula (1) represents the pixel situation of the overlapping part of the watermark and the original image in alpha blending, v(X) represents the pixel of the original image X, v(W) is the watermark pixel to be added, and v(G) is the watermark pixel of the final generated image.

[0046] v(G)=(v(W)*α+v(X)*255) / 255 #(1)

[0047] Step 2: Neural network identifies watermarked image: Input the generated watermarked image into the target neural network to obtain the confidence information of each classification output by the target neural network.

[0048] The target neural network used in this embodiment is the classification model f, the correct classification label of the image is set to t, and the confidence of label t is f t There are n categories in total, and the confidence of each category output is f 1…n .

[0049] Step 3, optimization iteration based on particle swarm algorithm: Based on the confidence of each classification of the watermark image, the adversarial watermark image is obtained through the particle swarm algorithm. Figure 2 As shown, the iterative steps are as follows:

[0050] Step 31: Convert the adversarial watermark generation problem into an optimization problem of finding (p, q, α, scale) values ​​so that the confidence of the generated adversarial watermark image is non-maximum, and establish the optimization objective as:

[0051] minimize fit(p,q,α,scale)=f t (g(p,q,α,scale))-f s (g(p,q,α,scale)) #(2)

[0052] In the formula, minimizing the fitness function fit is used as the optimization goal, f t (g(p,q,α,scale)) represents the confidence of the correct classification label t obtained for the watermark image classification, f s (g(p,q,α,scale)) represents the maximum confidence value of other classification labels except the correct classification label t obtained for the classification of the watermarked image. The g(p,q,α,scale) function is the watermark generation function, that is, the aforementioned alpha blending algorithm. (p,q) is the position coordinate of the watermark in the original image, α is the transparency value of the watermark, and scale is the ratio of the length and width of the watermark to the length and width of the original image.

[0053] The existing adversarial watermarking algorithm adopts the method of minimizing f t (f t That is f t The value of (g(p,q,α,scale))) is used as the optimization target, and the optimization iteration will only move towards f t If the optimization goal proposed by the present invention takes into account f s (f s That is f s (g(p,q,α,scale))) changes, f t Reduce and f s Increasing can reduce f t -f s The value of , while not missing the existing optimization target results, also adds additional possible results, thereby achieving a higher attack success rate.

[0054] Therefore, before using the particle swarm algorithm to iteratively obtain the watermark position, this embodiment first determines the fitness function fit, which can represent f t With f s The relationship between t <f s f tThe confidence of is no longer the highest confidence among all classifications, and the classification model will not classify the watermark image as t at this time. The particle at this time is the solution that can be successfully attacked. Therefore, if f t <f s That is, when the value of the function fit is less than 0, the iteration is terminated in advance, that is, the (p, q, α, scale) at this time is taken to generate an image counter-watermark, and the process is terminated directly without executing subsequent steps, otherwise, the subsequent steps are continued.

[0055] Step 32, Particle Swarm Algorithm Initialization: During initialization, each particle in the particle swarm is randomly distributed in the D-dimensional space with random speed and position. The position of particle i at the kth iteration is recorded as The velocity of particle i at the kth iteration is recorded as The position of each particle is composed of (p, q, α, scale), representing different adversarial watermark images. The maximum number of iterations of the particle swarm is T, the current number of iterations is k, and k is initialized to 1.

[0056] Step 33, iteratively update the particle swarm until the maximum number of iterations is reached and then stop, and output the position and transparency information of the watermark in each iteration.

[0057] Step 331, calculate the fitness value of each particle, and the fitness function fit expression is as follows:

[0058] fit(p,q,α,scale)=f t (g(p,q,α,scale))-f s (g(p,q,α,scale)) #(3)

[0059] The lower the fitness function value, the better the particle position. The historical optimal position experienced by particle i in the previous k generations of iterations is recorded as The global optimal position experienced by the entire population during the first k generations of iterations is recorded as

[0060] Step 332: Compare the calculated fitness values. If the current fitness value of the particle is better than the fitness value of the historical optimal position of the particle, update the historical optimal position to the current position of the particle.

[0061] Step 333: compare the fitness value of the particle with the fitness value of the current global optimal position. If the fitness value of the particle is better than the fitness value of the global optimal position, update the global optimal position to the position of the particle.

[0062] Step 334: Update the particle's velocity and position. The update formula is as follows.

[0063]

[0064]

[0065] Where k is the current iteration number of the particle, d is the current dimension of the particle, is the velocity of particle i in dimension d at the kth iteration, c1 and c2 are acceleration coefficients, indicating that the particle flies to the individual optimal and the population optimal g k The acceleration coefficient is used to adjust its step size, r1 and r2 are two random functions used to increase the randomness during the search, and w is the inertia weight. The particle speed and position are iteratively updated through formula (4) and formula (5) to make the particle approach the optimization target.

[0066] Step 335, the number of iterations k is increased by 1. If the number of iterations is greater than the maximum number of iterations T, the iteration is stopped; otherwise, the process returns to step 331 to continue execution.

[0067] Step 34: After stopping the iteration, the particle swarm algorithm outputs the optimal (p, q, α, scale) value, which is input into the watermark generation function g(p, q, α, scale) to generate an adversarial watermark image with an image adversarial watermark.

[0068] In order to verify the effectiveness of this solution, the following is a further explanation based on experimental data:

[0069] This example uses 1000 random images from ImageNet and images from CIFAR-10 as the original host images for the experiment. Four university logo images are used as watermarks to be added, and their attack conditions are tested when the size of the watermark host image is 1 / 2, 1 / 3, and 1 / 4.

[0070] Figure 3 Some images generated by the algorithm in this paper are shown with adversarial watermarks. The first row is the original image, and the second row is an image with a watermark of a specific position and transparency added. The corresponding text below the image shows the result of classification by the model. The generated adversarial watermark image can mislead the classification model, but does not affect human eye recognition. It can prevent the machine from correctly identifying the image content when expressing the copyright information of the copyright owner.

[0071] This paper uses four well-trained network models as the target models of the experiment. These four models are ResNet101, AlexNet, VGG19 and InceptionV3. The parameters of the particle swarm algorithm in this paper are as follows: the number of particles is 50, the number of iterations is 100, the inertia weight w is 0.8, and c1 and c2 are 1.1. The algorithm compared in this paper is the existing image adversarial watermark generation algorithm BHE (Basin Hopping Evolution), and its parameters all use the default parameters of the algorithm. Table 1 shows the experimental results in the ImageNet dataset. Table 2 shows the results in the CIFAR-10 dataset.

[0072] Table 1 Attack success rates of different algorithms in ImageNet dataset

[0073]

[0074]

[0075] Table 2 Attack success rates of different algorithms in the CIFAR-10 dataset

[0076] Watermark size algorithm ResNet101 AlexNet VGG19 InceptionV3 average 1 / 4 BHE 52.88％ 60.34％ 48.06％ 48.42％ 52.43％ 1 / 4 Particle Swarm 64.20％ 74.38％ 61.64％ 59.50％ 64.93％ 1 / 3 BHE 61.93％ 71.32％ 58.52％ 60.81％ 63.15％ 1 / 3 Particle Swarm 73.06％ 83.75％ 71.15％ 72.18％ 75.04％ 1 / 2 BHE 79.15％ 84.44％ 75.36％ 78.05％ 79.25％ 1 / 2 Particle Swarm 86.33％ 92.10％ 84.72％ 86.84％ 87.50％

[0077] By comparing the results of the particle swarm algorithm and the BHE algorithm, it can be found that under different network models, the attack effect of the particle swarm algorithm in this paper is better than that of the BHE algorithm. On the ImageNet dataset, the attack success rate of the particle swarm algorithm is 3.47% higher than that of the BHE algorithm on average, and on the CIFAR-10 dataset, the attack success rate of the particle swarm algorithm is 10.88% higher than that of the BHE algorithm on average.

[0078] The particle swarm algorithm in this paper takes the confidence difference as the optimization target. During the operation of the algorithm, when the confidence difference is less than 0, it means that the attack has been successful, and the iteration can be terminated in advance to increase the generation speed. Therefore, for the image anti-watermark attack, this paper counts the running time of the algorithm in different network models in the experiments in Table 1 and Table 2, and the results are shown in Table 3.

[0079] Table 3 Time statistics of image watermarking experiment

[0080] algorithm ResNet101 AlexNet VGG19 InceptionV3 average BHE 13342s 2520s 7511s 11020s 8598.25s Particle Swarm 5528s 294s 4307s 4712s 3710.25s

[0081] According to the experimental results in Table 3, the image adversarial watermark generation method based on particle swarm algorithm proposed in this paper takes less time to generate adversarial watermarks in multiple network model experiments. On average, it only takes 43.15% of the time of the BHE algorithm to complete the adversarial watermark experiment, which is more efficient.

[0082] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0083] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A method for generating image adversarial watermark based on particle swarm optimization algorithm, characterized in that: The image adversarial watermark generation method based on particle swarm algorithm includes: Step 1: Based on the position and transparency information of the watermark, the original image and the watermark are combined through alpha blending to generate a watermark image; Step 2: Input the generated watermark image into the target neural network to obtain the confidence information of each classification output by the target neural network; Step 3: Based on the confidence of each classification of the watermark image, the adversarial watermark image is obtained through the particle swarm algorithm, including: Step 31: Convert the adversarial watermark generation problem into an optimization problem of finding the (p, q, α, scale) value so that the confidence of the generated adversarial watermark image is non-maximum. The optimization objective is established as: minimize fit(p,q,α,scale)=f t (g(p,q,α,scale))-f s (g(p,q,α,scale)) In the formula, minimizing the fitness function fit is used as the optimization goal, f t (g(p,q,α,scale)) represents the confidence of the correct classification label t obtained for the watermark image classification, f s (g(p,q,α,scale)) represents the maximum confidence value of the classification labels other than the correct classification label t obtained for the watermark image classification. The g(p,q,α,scale) function is the watermark generation function, (p,q) is the position coordinate of the watermark in the original image, α is the transparency value of the watermark, and scale is the ratio of the length and width of the watermark to the length and width of the original image; Step 32, Particle Swarm Algorithm Initialization: During initialization, each particle in the particle swarm is randomly distributed in the D-dimensional space with random speed and position. The position of particle i at the kth iteration is recorded as The velocity of particle i at the kth iteration is recorded as The position of each particle is composed of (p, q, α, scale), representing a different adversarial watermark image; Step 33: Iterate and update the particle swarm until the maximum number of iterations is reached, and output the position and transparency information of the watermark in each iteration; Step 34: After stopping the iteration, the particle swarm algorithm outputs the optimal (p, q, α, scale) value, which is input into the watermark generation function g(p, q, α, scale) to generate an adversarial watermark image with the image adversarial watermark.

2. The method for generating an image adversarial watermark based on a particle swarm algorithm according to claim 1, wherein: The step of combining the original image with the watermark through alpha blending to generate a watermarked image includes: The pixels where the watermark overlaps with the original image are calculated as follows: v(G)=(v(W)*α+v(X)*255) / 255 Where v(G) is the pixel value of the overlapping pixel, v(W) is the pixel value of the watermark pixel, and v(X) is the pixel value of the original image pixel.

3. The method for generating an image adversarial watermark based on a particle swarm algorithm according to claim 1, wherein: The target neural network is the classification model f, the correct classification label of the image is t, and the confidence of label t is f t , there are n categories in total, and the confidence of each category output is f 1…n .

4. The method for generating an image adversarial watermark based on a particle swarm algorithm according to claim 1, wherein: The particle swarm is iteratively updated until the maximum number of iterations is reached and then stopped. Each iteration outputs the position and transparency information of the watermark, including: Step 331: Calculate the fitness value of each particle. The fitness function fit is expressed as follows: fit(p,q,α,scale)=f t (g(p,q,α,scale))-f s (g(p,q,α,scale)) The historical optimal position experienced by particle i during the previous k generations of iterations is recorded as The global optimal position experienced by the entire population during the first k generations of iterations is recorded as Step 332: Compare the calculated fitness values. If the current fitness value of the particle is better than the fitness value of the particle's historical optimal position, update the historical optimal position to the particle's current position. Step 333: Compare the fitness value of the particle with the fitness value of the current global optimal position. If the fitness value of the particle is better than the fitness value of the global optimal position, update the global optimal position to the position of the particle. Step 334: Update the particle's velocity and position. The update formula is as follows: Where k is the current iteration number of the particle, d is the current dimension of the particle, is the velocity of particle i in dimension d at the kth iteration, c1 and c2 are acceleration coefficients, indicating that the particle flies to the individual optimal and the population optimal g k The acceleration coefficient when , r1 and r2 are two random functions used to increase the randomness of the search, and w is the inertia weight; Step 335: The number of iterations k is increased by 1. If the number of iterations is greater than the maximum number of iterations T, the iteration is stopped; otherwise, the process returns to step 331 to continue.

5. The method for generating an image adversarial watermark based on a particle swarm algorithm according to claim 1, wherein: In the iterative update of the particle swarm, before executing step 332, first determine f t <f s The size of f t <f s , that is, when the value of the function fit is less than 0, the iteration is terminated early; otherwise, step 332 is executed.

Images