A method and system for generating physical world adversarial samples based on robust features

Abstract

The application discloses a physical world adversarial sample generation method based on robust features, comprising the following steps: obtaining a pre-trained autoencoder, three pre-trained image classification task models, and a data set for an image classification task, setting attack parameters, and using the autoencoder and the three image classification task models to iteratively optimize an image with a class label number t in the data set for the image classification task to obtain robust feature encoding F of the image with the class label number t tar An image xc with a class label number c is randomly selected from the data set for the image classification task, weight matrix m and alpha are initialized, and the selected image with a class label number v and the robust feature encoding F of the image with the class label number t are utilized tar The weight matrix m and alpha are respectively optimized to obtain optimized weight matrix m' and alpha', and finally, an adversarial sample is obtained according to the optimized weight matrix m' and alpha'.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and more particularly, relates to a method and system for generating physical-world adversarial examples (PAE) based on robust features. BACKGROUND

[0002] Adversarial examples refer to samples that cause a deep learning model to make incorrect predictions by applying a small and carefully designed perturbation to the input data. With the wide application of deep neural networks in real (physical) world such as autonomous driving systems, facial recognition systems, medical imaging systems, etc., researching adversarial examples that can be applied in the physical world is of great significance to improve the security of artificial intelligence systems and applications. Existing physical-world adversarial examples can be divided into the following three categories according to their manifestations:

[0003] 1. Adversarial examples based on pixel perturbation: attackers add carefully designed noise perturbations to the pixel points of the original image to generate perturbed image samples, and print them to the physical world; this kind of sample realizes the misdirection of the deep learning model through pixel-level modification of the image.

[0004] 2. Adversarial examples based on adversarial patches: attackers design a small and delicate pattern (i.e. patch) and print it to the physical world, which is attached to the surface of the target object through sticking, etc.; when the model makes predictions on the object, the adversarial patch will cause the model to output incorrect results.

[0005] 3. Adversarial examples based on optical methods: attackers use laser, light beam, shadow, etc. optical conditions in the physical world to apply perturbation effects, thereby interfering with the model's image perception process and causing the model to make incorrect judgments.

[0006] However, the above-mentioned several existing physical-world adversarial examples still have some non-negligible defects: first, the above-mentioned three kinds of adversarial examples have poor transferability, i.e. the adversarial examples generated for a specific model and task are difficult to produce effective attack effects in other models and tasks, and have poor transferability; second, the above-mentioned adversarial examples based on optical methods have poor robustness, and their attack effects are easily affected by the physical world environment conditions and sampling conditions, such as sampling distance and angle, environmental lighting, reflection and refraction, etc., resulting in a significant decline in attack performance; third, the above-mentioned adversarial examples based on pixel perturbation and adversarial patches have difficulty in balancing attack performance and concealment, and improving the attack performance of adversarial examples often sacrifices their concealment, making them more easily perceived by the human eye or detected. SUMMARY

[0007] In view of the above defects or improvement needs of the prior art, the present application provides a method and system for generating physical world adversarial samples based on robust features, which aims to solve the technical problems of poor transferability of the existing three kinds of adversarial samples, poor transferability, poor robustness of the existing adversarial samples based on optical methods, and the attack effect being easily affected by the physical world environment conditions and sampling conditions, resulting in a significant decline in attack performance, and the attack performance and concealment of the existing adversarial samples based on pixel perturbation and adversarial patches being difficult to balance, and the improvement of the attack performance of the adversarial samples often sacrificing their concealment, resulting in their being more easily perceived by the human eye or detected.c

[0008] To achieve the above-mentioned purpose, according to one aspect of the present application, a method for generating physical world adversarial samples based on robust features is provided, comprising the following steps:

[0009] (1) obtaining a pre-trained autoencoder, three pre-trained image classification task models, and a data set for an image classification task, wherein the autoencoder comprises an encoder E and a decoder D;

[0010] (2) setting attack parameters, and using the autoencoder and the three image classification task models obtained in step (1) to iteratively optimize the images with class label number t in the data set for the image classification task, to obtain the robust feature encoding F tar of the images with class label number t, wherein t∈[1, the total number of classes of the images in the data set for the image classification task];

[0011] (3) randomly selecting an image x c with class label number c from the data set for the image classification task obtained in step (1) (c is freely selected by the attacker, its value range is 0 to 999, and is different from t set in step (2)), initializing the weight matrix m and α, and using the selected image with class label number c and the robust feature encoding F tar obtained in step (2) to optimize the weight matrix m and α respectively, to obtain the optimized weight matrix m' and α', wherein m is a matrix with the same dimension as the selected x c , each element in the weight matrix m is randomly initialized in the interval [0, 1], the weight matrix α is a matrix with the same dimension as the robust feature encoding F tar obtained in step (2), and each element in the weight matrix α is randomly initialized in the interval [0, 1];

[0012] (4) obtaining the final adversarial sample xadv .

[0013] Preferably, the autoencoder selected in step (1) is the backbone network of the VGG-16 model;

[0014] The three pre-trained image task classification models obtained are ResNet-50, VGG-16, and DenseNet-121, and the pre-training rounds of the four are all 100 rounds;

[0015] The data set for image classification task obtained is the ImageNet ILSVRC 2012 data set, which includes 1000 types of images.

[0016] Preferably, step (2) includes the following sub-steps:

[0017] (2-1) Initialize the attack parameters, and load the autoencoder and the three image classification task models obtained in step (1);

[0018] (2-2) Use the three image classification task models loaded in step (1) to respectively predict the images with class label number t in the data set for image classification task obtained in step (1), to respectively obtain the prediction confidence of the image corresponding to each image task classification model, and obtain all images with prediction confidence greater than θ corresponding to all three image task classification models from all images with class label number t in the data set, and randomly obtain N images from them to form an image set S;

[0019] (2-3) Use the encoder E in the autoencoder obtained in step (1) to sequentially encode all images in the set S obtained in step (2-2), and take the average of all encoding results to obtain the robust feature encoding F0 of the target class;

[0020] (2-4) Set the counter cnt = 0, and initialize the optimization variable f initialization equal to the robust feature encoding F0 of the target class obtained in step (2-3);

[0021] (2-5) Use the decoder D in the autoencoder obtained in step (1) to decode the optimization variable f to obtain the decoding result D(f);

[0022] (2-6) Generate a random matrix δ with the same dimension as the decoding result D(f) obtained in step (2-5), and each element in the random matrix δ is independently sampled from a standard normal distribution ;

[0023] (2-7) Add the random matrix obtained in step (2-6) as random noise disturbance to the decoding result D(f) obtained in step (2-5) to obtain a disturbed result D(f)+mδ, and input the disturbed result into the three pre-trained classification models obtained in step (1) respectively for prediction, and calculate the loss function L according to the prediction result, wherein m represents the noise intensity coefficient added, and the value range is 0.5 to 4, preferably 2;

[0024] (2-8) Obtain the gradient g of the optimization variable f according to the loss function L obtained in step (2-7);

[0025] (2-9) Update the optimization variable f according to the gradient g obtained in step (2-8);

[0026] (2-10) Update the counter cnt1=cnt1+1, and judge whether n1 is greater than the optimization iteration number n in the attack parameter obtained in step (2-1); if yes, use the current optimization variable f as the robust feature encoding F of the image with the class label number t tar , the process is ended, otherwise return to step (2-5).

[0027] Preferably, the attack parameters in step (2-1) include the optimization iteration number n, the number N of target class images selected for extracting robust features, and the prediction confidence threshold θ of the target class images for extracting robust features;

[0028] The number N of target class images selected for extracting robust features in step (2-1) has a value range of 10 to 20, preferably 15;

[0029] The prediction confidence threshold θ of the target class images for extracting robust features in step (2-1) has a value range of 0.5 to 1, preferably 0.7;

[0030] The optimization iteration number n in step (2-1) has a value range of 1000 to 3000, preferably 2500.

[0031] Preferably, the robust feature encoding of the target class in step (2-3) is calculated by the following formula:

[0032]

[0033] Where x i is the i-th image in the image set S, E(x i ) represents the encoding result of the i-th image in the image set S, and i∈[1,N],N represents the total number of images in the image set S;

[0034] The calculation formula of the loss function L in step (2-7) is:

[0035]

[0036] wherein M j is the jth image classification task model of the three pre-trained image classification task models obtained in step (1), and j∈[1, 3], L CE is a cross-entropy function.

[0037] The calculation formula in step (2-8) is

[0038] The formula in step (2-9) is f = f - lr·g.

[0039] wherein lr represents the update rate of the optimization variable f, and its value range is 0.001 to 0.01, and preferably 0.005.

[0040] Preferably, step (3) comprises the following sub-steps:

[0041] (3-1) randomly selecting an image x with a class label number c from the data set obtained in step (1) c , initializing the weight matrix m and a;

[0042] (3-2) initializing the parameters for optimizing fusion, and setting the counter cnt2 = 0;

[0043] (3-3) obtaining an intermediate result x' according to the current weight matrix m and a, and the image x with a class label number c selected in step (3-1) c

[0044] (3-4) obtaining the adversarial loss L adv and the perceptual loss L cog according to the intermediate result x' obtained in step (3-3);

[0045] (3-5) obtaining the gradient g1 of the weight matrix m and the gradient g2 of the weight matrix a according to the adversarial loss and the perceptual loss obtained in step (3-4), respectively;

[0046] (3-6) updating the weight matrix m and a according to the gradient g1 of the weight matrix m and the gradient g2 of the weight matrix a obtained in step (3-5), and the parameters for optimizing fusion initialized in step (3-1);

[0047] ​(3-7) updating the counter cnt2 = cnt2 + 1, and determining whether cnt2 is greater than the iteration number n2 in the parameter for optimized fusion after initialization in step (3-1); if yes, using the weight matrix m obtained in step (3-6) as the optimized weight matrix m', using the weight matrix a obtained in step (3-6) as the optimized weight matrix a', and ending the process, otherwise returning to step (3-3).

[0048] Preferably, the parameter for optimized fusion in step (3-2) specifically comprises an optimized iteration number n2, an updating rate s1 of the weight matrix m, and an updating rate s2 of the weight matrix a.

[0049] The optimized iteration number n2 ranges from 100 to 1000, preferably 500;

[0050] The updating rate s1 of the weight matrix m ranges from 0.001 to 0.01, preferably 0.002;

[0051] The updating rate s2 of the weight matrix a ranges from 0.001 to 0.05, preferably 0.01.

[0052] Preferably, the calculation formula of step (3-3) is:

[0053]

[0054] wherein represents element-wise multiplication of matrices;

[0055] The calculation formula of the adversarial loss in step (3-4) is:

[0056]

[0057] The calculation formula of the perceptual loss in step (3-4) is:

[0058] L cog = ||m||1+ TV(m) + SSIM(x', x c )

[0059] wherein ||·|| is L1 norm, TV(·) is total variation function, and SSIM(·) is structural similarity index;

[0060] The calculation formula of step (3-5) is:

[0061]

[0062] The updating formula of step (3-6) is:

[0063] m = m - s1*g1

[0064] a = a - s2*g2

[0065] Preferably, the formula for calculating the adversarial sample in step (4) is:

[0066]

[0067] According to another aspect of the present application, a robust feature-based physical world adversarial sample generation system is provided, comprising:

[0068] A first module for obtaining a pre-trained autoencoder, three pre-trained image classification task models, and a data set for an image classification task, wherein the autoencoder comprises an encoder E and a decoder D;

[0069] A second module for setting attack parameters and using the autoencoder and the three image classification task models obtained by the first module to iteratively optimize the images with class label number t in the data set for the image classification task to obtain the robust feature encoding F of the images with class label number t tar , wherein t e [1, the total number of classes of images in the data set for the image classification task];

[0070] A third module for randomly selecting an image x with class label number c from the data set for the image classification task obtained by the second module c (c is freely selected by the attacker, its value range is 0 to 999, and is different from t set by the second module), initializing the weight matrix m and a, and using the selected image with class label number c and the robust feature encoding F of the image with class label number t obtained by the second module tar to optimize the weight matrix m and a respectively to obtain the optimized weight matrix m' and a', wherein m is a matrix with the same dimension as the selected x c , each element in the weight matrix m is randomly initialized in the interval [0, 1], the weight matrix a is a matrix with the same dimension as the robust feature encoding F of the image with class label number t obtained by the second module, and each element in the weight matrix a is randomly initialized in the interval [0, 1]; tar .

[0071] A fourth module for obtaining the final adversarial sample x adv from the optimized weight matrix m' and a' obtained by the third module.

[0072] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0073] 1. Because this invention employs steps (2-1) to (2-10), it utilizes an algorithm for extracting robust features from target category images. The extracted robust features are robust to disturbances in the physical world environment, effectively improving the robustness of adversarial examples.

[0074] 2. Since the present invention employs steps (3-1) to (3-5), it uses a robust feature fusion method to inject the robust features of the extracted target category image into the clean image, so that the injected robust features can be recognized and perceived by models with different architectures, thereby effectively improving the transferability of adversarial examples.

[0075] 3. Since the present invention employs steps (3-3) to (3-5), it uses a minimized cognitive pattern algorithm to erase unnecessary disturbances in the image, which can maximize the concealment of adversarial examples while ensuring the performance of adversarial example attacks. Attached Figure Description

[0076] Figure 1 This is a flowchart of the method for generating adversarial examples in the physical world based on robust features according to the present invention;

[0077] Figure 2 This is a detailed flowchart of step (2) in the method of the present invention;

[0078] Figure 3 This is a flowchart of the present invention that fuses the target robust features onto a clean image to generate adversarial examples. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0080] The basic approach of this invention employs two main measures to enhance the transferability, robustness, and stealth of adversarial examples in the physical world. Firstly, a process is designed to extract robust features of the target category and then fuse them into a clean image, comprehensively improving the transferability and robustness of the adversarial examples. Secondly, an algorithm that minimizes cognitive patterns is used to erase unnecessary perturbations, enhancing stealth without affecting the attack performance of the adversarial examples.

[0081] The purpose of the present application is to design physical world adversarial samples with good attack performance, to construct a physical world adversarial sample test dataset, and to lay a foundation for evaluating and improving the robustness and security of deep learning models. The advantage of the present application is that the designed adversarial samples can adapt to various complex physical world environments and achieve attack effects on models of various architectures, and have good concealment and are not easy to be detected by the human eye.

[0082] In order to make the purpose, technical method and advantages of the present application more clear and explicit, the present application will be further described in detail below in combination with the drawings.

[0083] As shown in Figure 1 , the present application provides a physical world adversarial sample generation method based on robust features, comprising the following steps:

[0084] (1) obtaining a pre-trained autoencoder, three pre-trained image classification task models, and a dataset for image classification tasks, wherein the autoencoder comprises an encoder E and a decoder D;

[0085] Specifically, the autoencoder selected in this step is the backbone network of the VGG-16 model, the three pre-trained image classification task models obtained are ResNet-50, VGG-16, and DenseNet-121, and the pre-training rounds of the four are all 100 rounds. The dataset for image classification tasks obtained is the ImageNet ILSVRC 2012 dataset;

[0086] It should be noted that each type of image in the dataset for image classification tasks is assigned a class label number; in the dataset selected in the present application, there are 1000 types of images, so the class label number is between 0 and 999.

[0087] (2) setting attack parameters, and using the autoencoder and the three image classification task models obtained in step (1) to iteratively optimize the images in the dataset for image classification tasks with each class label number t (freely selected by the attacker, ranging from 0 to 999) to obtain the robust feature encoding F tar of the image with class label number t;

[0088] As shown in Figure 2 , this step comprises the following substeps:

[0089] (2-1) initializing attack parameters, and loading the autoencoder and the three image classification task models obtained in step (1);

[0090] Specifically, the attack parameters in this step include the number of optimization iterations n, the number of target category images selected for extracting robust features N, and the prediction confidence threshold θ of the target category images for extracting robust features;

[0091] More specifically, the attack parameters initialized in this step need to be flexibly selected according to actual conditions and requirements;

[0092] The number of target category images selected for extracting robust features N in this step ranges from 10 to 20, preferably 15;

[0093] The prediction confidence threshold θ of the target category images for extracting robust features in this step ranges from 0.5 to 1, preferably 0.7;

[0094] The number of optimization iterations n in this step ranges from 1000 to 3000, preferably 2500;

[0095] (2-2) Using the three image classification task models loaded in step (1) respectively, predict the images with class label number t in the data set for image classification task obtained in step (1) to obtain the prediction confidence of each image classification task model respectively, and obtain all images with prediction confidence greater than θ corresponding to all three image classification task models from all images with class label number t in the data set, and randomly obtain N images from them to form an image set S;

[0096] (2-3) Using the encoder E in the autoencoder obtained in step (1), encode all images in the set S obtained in step (2-2) in turn, and take the average of all encoding results to obtain the robust feature encoding F0 of the target category;

[0097] Specifically, the robust feature encoding of the target category calculated in this step is calculated using the following formula:

[0098]

[0099] Where x i is the i-th image in the image set S, E(x i ) represents the encoding result of the i-th image in the image set S, and i∈[1, N], N represents the total number of images in the image set S;

[0100] (2-4) Set the counter cnt = 0, and initialize the optimization variable f initialization equal to the robust feature encoding F0 of the target category obtained in step (2-3);

[0101] (2-5) decoding the optimization variable f using the decoder D in the autoencoder obtained in step (1) to obtain a decoding result D(f);

[0102] (2-6) generating a random matrix δ with the same dimension as the decoding result D(f) obtained in step (2-5), each element in the random matrix δ being independently sampled from a standard normal distribution N(0, 1);

[0103] (2-7) adding the random matrix obtained in step (2-6) as random noise disturbance to the decoding result D(f) obtained in step (2-5) to obtain a disturbed result D(f) + mδ, and inputting the disturbed result into the three pre-trained classification models obtained in step (1) respectively for prediction, and calculating a loss function L according to the prediction results, wherein m represents a noise intensity coefficient added, and the value range of m is 0.5 to 4, and the preferred value is 2;

[0104] Specifically, the calculation formula of the loss function L in this step is:

[0105]

[0106] wherein M j is the jth image classification task model in the three pre-trained image classification task models obtained in step (1), and j ∈ [1, 3], and L CE is a cross-entropy function;

[0107] The advantage of this sub-step (2-7) is that when extracting the robust feature encoding of the image with the class label number t, diversified noise disturbance is introduced, so that the extracted feature encoding is robust to the physical world environment, which helps to improve the robustness of the adversarial samples generated in the subsequent steps.

[0108] (2-8) obtaining the gradient g of the optimization variable f according to the loss function L obtained in step (2-7); specifically, the calculation formula in this step is

[0109] (2-9) updating the optimization variable f according to the gradient g obtained in step (2-8);

[0110] Specifically, the formula in this step is f = f - lr·g;

[0111] wherein lr represents the update rate of the optimization variable f, and the value range of lr is 0.001 to 0.01, and the preferred value is 0.005;

[0112] (2-10) Update the counter cnt1 = cnt1 + 1, and determine whether cnt1 is greater than the optimization iteration number n in the attack parameter obtained in step (2-1); if yes, use the current optimization variable f as the robust feature code F of the image with the class label number t tar , the process ends, otherwise return to step (2-5);

[0113] The advantages of the above sub-steps (2-7) to (2-10) are that multiple image classification task models are used in the optimization process, so that the robust feature code F of the image with the class label number t tar extracted can be recognized by models of multiple different architectures, which helps to improve the migration of the adversarial samples generated in the subsequent steps.

[0114] (3) Randomly select an image x with a class label number c from the data set for the image classification task obtained in step (1) c (c is freely selected by the attacker, the value range is 0 to 999, and is different from t set in step (2)), initialize the weight matrix m and a, and use the selected image with the class label number c and the robust feature code F of the image with the class label number t obtained in step (2) tar Optimize the weight matrix m and a respectively to obtain the optimized weight matrix m' and a' respectively, wherein m is a matrix with the same dimension as the selected x c , the value of each element in the weight matrix m is randomly initialized in the interval [0, 1], and the weight matrix a is a matrix with the same dimension as the robust feature code F of the image with the class label number t obtained in step (2) tar , the value of each element in the weight matrix a is randomly initialized in the interval [0, 1]; the overall process of this step is shown in Figure 3 ;

[0115] The advantage of this step (3) is that by optimizing two weight matrices m and a, instead of directly using the image as the optimization variable, the space of the optimization variable is larger, which is beneficial to improve the attack performance of the adversarial samples.

[0116] Specifically, this step includes the following sub-steps:

[0117] (3-1) Randomly select an image x with a class label number c from the data set obtained in step (1) c , initialize the weight matrix m and a;;

[0118] (3-2) Initialize the parameters for optimizing fusion, and set the counter cnt2 = 0;

[0119] The parameters for optimizing fusion specifically include optimizing the number of iterations n2, the update rate s1 of the weight matrix m, and the update rate s2 of the weight matrix a.

[0120] In this step, the value range of the number of optimization iterations n2 is set to 100 to 1000, preferably 500;

[0121] In this step, the value range of the update rate s1 of the weight matrix m is set to 0.001 to 0.01, preferably 0.002;

[0122] In this step, the value range of the update rate s2 of the weight matrix a is set to 0.001 to 0.05, preferably 0.01.

[0123] (3-3) According to the current weight matrix m and a, and the image x with the category label number c selected in step (3-1) c , obtain the intermediate result x';

[0124] Specifically, the calculation formula of this step is:

[0125]

[0126] wherein represents element-wise multiplication of matrices;

[0127] (3-4) Obtain the adversarial loss L adv and the perceptual loss L cog from the intermediate result x' obtained in step (3-3).

[0128] Specifically, the calculation formula of the adversarial loss in this step is:

[0129]

[0130] The calculation formula of the perceptual loss in this step is:

[0131] L cog = ||m||1+TV(m)+SSIM(x′,x c )

[0132] wherein ||·|| is the L1 norm, TV(·) is the total variation function, and SSIM(·) is the structural similarity index;

[0133] The advantage of this sub-step (3-4) is that the designed loss function not only includes the optimization of the attack target, but also considers enhancing the concealment of the adversarial sample, making it less likely to be detected and detected, which helps to achieve a balance between the attack performance and the concealment of the adversarial sample.

[0134] (3-5) According to the adversarial loss and the perceptual loss obtained in step (3-4), the gradient g1 of the weight matrix m and the gradient g2 of the weight matrix a are obtained respectively;

[0135] Specifically, the calculation formula of this step is:

[0136]

[0137] (3-6) According to the gradient g1 of the weight matrix m and the gradient g2 of the weight matrix a obtained in step (3-5), and the parameters used for optimizing fusion after initialization in step (3-1), the weight matrix m and a are updated;

[0138] Specifically, the update formula in this step is:

[0139] m = m - s1*g1

[0140] a = a - s2*g2

[0141] (3-7) Update the counter cnt2 = cnt2 + 1, and determine whether cnt2 is greater than the iteration number n2 in the parameters used for optimizing fusion after initialization in step (3-1); if yes, the weight matrix m obtained in step (3-6) is used as the optimized weight matrix m', the weight matrix a obtained in step (3-6) is used as the optimized weight matrix a', and the process ends, otherwise, return to step (3-3);

[0142] (4) According to the optimized weight matrix m' and a' obtained in step (3), the final adversarial sample x is obtained adv ;

[0143] Specifically, the formula for calculating the adversarial sample in this step is:

[0144]

[0145] Those skilled in the art will readily understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for generating adversarial examples in the physical world based on robust features, characterized in that, Includes the following steps: (1) Obtain the pre-trained autoencoder, three pre-trained image classification task models, and a dataset for image classification tasks, wherein the autoencoder includes an encoder E and a decoder D. (2) Set the attack parameters, and use the autoencoder obtained in step (1) and the three image classification task models to iteratively optimize the images with category label number t in the dataset used for image classification tasks, so as to obtain the robust feature encoding F of the images with category label number t. tar , where t∈[1, the total number of image categories in the dataset used for image classification tasks]; (3) Randomly select an image x with category label number c from the dataset obtained in step (1) for the image classification task. c c is freely chosen by the attacker, with a value ranging from 0 to 999, and is different from t set in step (2). The weight matrix m and α are initialized, and the robust feature encoding F is obtained using the image with the selected category label number c and the image with the category label number t obtained in step (2). tar The weight matrices m and α are optimized separately to obtain the optimized weight matrices m′ and α′, respectively, where m is the dimension and the selected x. c The same matrix, where each element in the weight matrix m is randomly initialized within the interval [0,1], and the weight matrix α is the image robust feature code F with category label number t obtained in step (2). tar The same matrix, with each element in the weight matrix α randomly initialized within the interval [0,1]; step (3) includes the following sub-steps: (3-1) Randomly select an image x with category label number c from the dataset obtained in step (1). c Initialize the weight matrices m and α; (3-2) Initialize the parameters used for fusion optimization and set the counter cnt2 = 0; (3-3) Based on the current weight matrix m and α, and the image x with category label number c selected in step (3-1) c Obtain the intermediate result x′; the calculation formula for step (3-3) is: in Represents element-wise matrix multiplication; (3-4) Obtain the adversarial loss L based on the intermediate result x′ obtained in step (3-3). adv With perceived loss L cog ; (3-5) Based on the adversarial loss and perceptual loss obtained in step (3-4), obtain the gradient g1 of the weight matrix m and the gradient g2 of the weight matrix α, respectively. (3-6) Update the weight matrix m and α based on the gradient g1 of the weight matrix m and the gradient g2 of the weight matrix α obtained in step (3-5), as well as the parameters used to optimize the fusion after initialization in step (3-1). (4) Obtain the final adversarial sample x based on the optimized weight matrix m′ and α′ obtained in step (3). adv .

2. The method for generating adversarial examples in the physical world based on robust features according to claim 1, characterized in that, The autoencoder selected in step (1) is the backbone network of the VGG-16 model; The three pre-trained image task classification models obtained are ResNet-50, VGG-16, and DenseNet-121, all of which have been pre-trained for 100 rounds. The dataset used for the image classification task was the ImageNet ILSVRC 2012 dataset, which includes 1000 types of images.

3. The method for generating adversarial examples in the physical world based on robust features according to claim 1 or 2, characterized in that, Step (2) includes the following sub-steps: (2-1) Initialize the attack parameters and load the autoencoder and three image classification task models obtained in step (1); (2-2) Use the three image classification task models loaded in step (1) to predict the image with category label number t in the dataset for image classification task obtained in step (1) respectively, so as to obtain the prediction confidence of the image corresponding to each image task classification model. From all images with category label number t in the dataset, obtain all images whose prediction confidence of all three image task classification models is greater than θ, and randomly select N images from them to form an image set S. (2-3) Use the encoder E in the autoencoder obtained in step (1) to encode all the images in the set S obtained in step (2-2) in sequence, and take the average of all the encoding results to obtain the robust feature encoding F0 of the target category; (2-4) Set the counter cnt1 = 0 and initialize the optimization variable f to be equal to the robust feature code F0 of the target category obtained in step (2-3); (2-5) Use the decoder D in the autoencoder obtained in step (1) to decode the optimization variable f to obtain the decoding result D(f); (2-6) Generate a random matrix δ with the same dimension as the decoding result D(f) obtained in step (2-5), where each element of the random matrix δ is derived from a standard normal distribution. Obtained by independent sampling; (2-7) Add the random matrix obtained in step (2-6) as random noise perturbation to the decoding result D(f) obtained in step (2-5) to obtain the perturbation result D(f)+mδ. Input the perturbation result into the three pre-trained classification models obtained in step (1) for prediction. Calculate the loss function L based on the prediction result, where m represents the added noise intensity coefficient, and its value ranges from 0.5 to 4. (2-8) Obtain the gradient g of the optimization variable f based on the loss function L obtained in step (2-7); (2-9) Update the optimization variable f based on the gradient g obtained in step (2-8); (2-10) Update the counter cnt1 = cnt1 + 1, and determine whether cnt1 is greater than the number of optimization iterations n in the attack parameters obtained in step (2-1); if so, use the current optimization variable f as the robust feature encoding F of the image with class label number t. tar If the process ends, return to step (2-5).

4. The method for generating adversarial examples in the physical world based on robust features according to claim 3, characterized in that, The attack parameters in step (2-1) include the number of optimization iterations n, the number of target class images N selected for robust feature extraction, and the prediction confidence threshold θ of the target class images used for robust feature extraction; In step (2-1), the number N of target category images selected for robust feature extraction ranges from 10 to 20; In step (2-1), the prediction confidence threshold θ for the target category image used to extract robust features ranges from 0.5 to 1; In step (2-1), the number of optimization iterations n ranges from 1000 to 3000.

5. The method for generating adversarial examples in the physical world based on robust features according to claim 4, characterized in that, Step (2-3) calculates the robust feature code of the target category using the following formula: Where x i Let E(x) be the i-th image in the image set S. i Let represent the encoding result of the i-th image in the image set S, where i ∈ [1, N] and N represents the total number of images in the image set S; The formula for calculating the loss function L in step (2-7) is as follows: Where M j Let j be the j-th image classification task model among the three pre-trained image classification task models obtained in step (1), and j∈[1,3], L CE It is the cross-entropy function; The calculation formula in step (2-8) is: The formula in step (2-9) is f = f - lr·g; Where lr represents the update rate of the optimization variable f, and its value ranges from 0.001 to 0.

01.

6. The method for generating adversarial examples in the physical world based on robust features according to claim 5, characterized in that, Step (3) also includes the following sub-steps following steps (3-6): (3-7) Update the counter cnt2 = cnt2 + 1, and determine whether cnt2 is greater than the iteration number n2 in the parameters used for optimization fusion after initialization in step (3-1); if so, use the weight matrix m obtained in step (3-6) as the optimized weight matrix m′, and use the weight matrix α obtained in step (3-6) as the optimized weight matrix α′, and the process ends; otherwise, return to step (3-3).

7. The method for generating adversarial examples in the physical world based on robust features according to claim 6, characterized in that, The parameters used for optimization fusion in step (3-2) specifically include the number of optimization iterations n2, the update rate s1 of the weight matrix m, and the update rate s2 of the weight matrix α; The number of optimization iterations n2 can range from 100 to 1000; The update rate s1 of the weight matrix m ranges from 0.001 to 0.01; The update rate s2 of the weight matrix α ranges from 0.001 to 0.

05.

8. The method for generating adversarial examples in the physical world based on robust features according to claim 7, characterized in that, The formula for calculating the adversarial loss in step (3-4) is as follows: The formula for calculating the perceived loss in step (3-4) is as follows: L cog =||m||1+TV(m)+SSIM(x′,x c ) Where ||·|| is the L1 norm, TV(·) is the total variation function, and SSIM(·) is the structured similarity index; The calculation formula for step (3-5) is as follows: The update formula for step (3-6) is: m = m - s1 * g1 α = α - s² * g².

9. The method for generating adversarial examples in the physical world based on robust features according to claim 8, characterized in that, The formula for calculating adversarial examples in step (4) is: 。 10. A physical-world adversarial example generation system based on robust features, characterized in that, include: The first module is used to obtain a pre-trained autoencoder, three pre-trained image classification task models, and a dataset for image classification tasks, wherein the autoencoder includes an encoder E and a decoder D. The second module is used to set the attack parameters and iteratively optimize the images with class label number t in the dataset used for image classification tasks using the autoencoder obtained in the first module and three image classification task models, in order to obtain the robust feature encoding F of the images with class label number t. tar , where t∈[1, the total number of image categories in the dataset used for image classification tasks]; The third module is used to randomly select an image x with category label number c from the dataset obtained from the first module for the image classification task. c , c is freely chosen by the attacker, its value ranges from 0 to 999, and it is different from t set by the second module. The weight matrix m and α are initialized, and the robust feature encoding F is obtained from the image with the selected class label number c and the image with the class label number t obtained by the second module. tar The weight matrices m and α are optimized separately to obtain the optimized weight matrices m′ and α′, respectively, where m is the dimension and the selected x. c The same matrix, where each element in the weight matrix m is randomly initialized within the interval [0,1], and the weight matrix α is the image robust feature code F with category label number t obtained from the second module. tar The same matrix, with each element in the weight matrix α randomly initialized within the interval [0,1]; the third module includes the following sub-modules: The first submodule is used to randomly select an image x with category label number c from the dataset obtained from the first module. c Initialize the weight matrices m and α; The second submodule is used to initialize the parameters used for fusion optimization and set the counter cnt2 = 0; The third submodule is used to determine the image x with category label number c selected by the first submodule based on the current weight matrix m and α. c Obtain the intermediate result x′; the calculation formula for the third submodule is: in Represents element-wise matrix multiplication; The fourth submodule is used to obtain the adversarial loss L based on the intermediate result x′ obtained from the third submodule. adv With perceived loss L cog ; The fifth submodule is used to obtain the gradient g1 of the weight matrix m and the gradient g2 of the weight matrix α based on the adversarial loss and perceptual loss obtained from the fourth submodule. The sixth submodule is used to update the weight matrix m and α based on the gradient g1 of the weight matrix m and the gradient g2 of the weight matrix α obtained from the fifth submodule, as well as the parameters used for optimization and fusion after initialization by the first submodule. The fourth module is used to obtain the final adversarial example x based on the optimized weight matrices m′ and α′ obtained in the third module. adv .

Images