A Highly Transferable Black-Box Attack Method Based on Adversarial Example Augmentation

Abstract

The present invention provides a highly transferable black box attack method based on adversarial example augmentation, comprising: step S1, obtaining the original image and the pseudo label of the original image; step S2, building a deep feature extractor; step S3, generating adversarial examples; step S4, inputting the adversarial examples into the proxy model to obtain the prediction result; step S5, calculating the structural similarity between the prediction result and the pseudo label and screening the adversarial examples; step S6, inputting the adversarial examples into the deep feature extractor to obtain two deep features; step S7, calculating the KL divergence between the two deep features, expanding the difference between the prediction result and the pseudo label and screening the adversarial examples; step S8, integrating the adversarial example set to query the target model to obtain the optimal adversarial example. The beneficial effect is that the present invention can generate an adversarial example set with high transferability and improve the attack effect.

Description

Technical Field

[0001] The present invention relates to the technical field of black box attacks, and in particular to a highly transferable black box attack method based on adversarial example augmentation. Background Art

[0002] Deep neural networks are the most widely used tools in the field of computer vision. As people continue to explore the field of deep learning, many adversarial attack methods have been studied to prove the vulnerability of deep neural networks. Adversarial attacks aim to introduce carefully designed perturbations into images to manipulate the prediction results of deep neural networks. However, white-box adversarial attack methods require knowledge of the internal information of the model, which is inconsistent with the actual attack situation. In black-box adversarial attack methods, query-based attack methods usually require a large number of queries. Attackers can easily defend against such attacks by limiting the number of API calls. In black-box attacks, transfer-based attack methods generate adversarial examples for proxy models to attack target models. Compared with the former, such attack methods are more covert.

[0003] The existing transfer-based attack methods all hope that the structure of the proxy model can be similar to the target model, and use the gradient information of the trained proxy model to generate adversarial examples. In addition, these methods usually only generate a single adversarial example, which makes the transferability of the adversarial example more dependent on the structure of the proxy model. The difference between the proxy model and the target model leads to the low transferability of the adversarial example. Summary of the invention

[0004] The technical problem to be solved by the present invention is to generate a set of adversarial examples with high transferability and improve the attack effect. In order to overcome the defects of the above-mentioned prior art (or related technology), the present invention provides a highly transferable black-box attack method based on adversarial example augmentation.

[0005] The present invention provides a highly transferable black box attack method based on adversarial example augmentation, comprising the following steps:

[0006] Step S1, obtain any picture as the original image , and call an image two-class segmentation model as a proxy model to convert the original image Input into the proxy model to obtain the original image Pseudo-labels ;

[0007] Step S2, building a deep feature extractor;

[0008] Step S3, based on the original image Creating additive perturbations Sum multiplicative disturbance , and according to the original image , the additive perturbation and the multiplicative perturbation Get adversarial examples ;

[0009] Step S4: Input into the proxy model to obtain the prediction result , and use the binary cross entropy loss function to calculate the prediction result and the pseudo-label The loss between , and using the infinite norm to calculate the original image With the adversarial example The difference between ;

[0010] Step S5, back propagation optimization objective function , update the additive perturbation With the multiplicative perturbation , and calculate the prediction results and the pseudo-label The structural similarity between them, and when the structural similarity is below a preset threshold The corresponding adversarial example Store as adversarial examples ,in , Indicates preset parameters;

[0011] Step S6, repeatedly executing step S3, step S4, and step S5, and performing the adversarial example obtained after each repetition. As new adversarial examples , the adversarial example Input into the deep feature extractor to obtain deep features , and the generated adversarial examples Input into the deep feature extractor to obtain deep features ,in ;

[0012] Step S7, calculating the deep features With the deep features The KL divergence between them is used to construct the objective function based on the KL divergence. , and optimize the overall objective function To expand the adversarial examples The corresponding prediction results and the pseudo-label When the predicted results and the pseudo-label The structural similarity between them is lower than the preset threshold and Less than the preset threshold , retain the corresponding adversarial example ,in Indicates the preset distance value. Indicates preset parameters;

[0013] Step S8, repeatedly executing step S6 and step S7, and retaining the adversarial examples after each repetition Integrate into an adversarial example set, using each of the adversarial examples in the adversarial example set The target model is queried to obtain the corresponding target prediction result and target pseudo label, and the adversarial example with the largest difference between the target prediction result and the target pseudo label is selected. As an optimal adversarial example.

[0014] Compared with the prior art, the highly transferable black-box attack method based on adversarial example augmentation has the following advantages:

[0015] In the present invention, the original image is acquired and the pseudo label is calculated in step S1, the deep feature extractor is constructed in step S2, and the adversarial example is performed in step S3. The prediction result is calculated by step S4, and the structural similarity between the prediction result and the pseudo label is calculated by step S5, as well as the adversarial example The deep features are extracted through step S6, and the difference between the prediction result and the pseudo label is enlarged through step S7, as well as the adversarial example The present invention screens and generates, integrates the adversarial example set and generates the optimal adversarial example through step S8; from the perspective of adversarial example augmentation, the present invention focuses on generating an adversarial example set. Compared with the single adversarial example generation in the existing method, the adversarial example set has higher transferability, and the present invention utilizes the difference in deep features of adversarial examples for augmentation, optimizes different deep features for different adversarial examples, and has the ability to attack more target models in terms of features. The present invention is based on the transmission and query method, and uses the adversarial example set to query and attack the target model to obtain the optimal adversarial example, which has higher transferability and can improve the attack effect.

[0016] In a possible implementation, in step S2, the deep feature extractor is constructed using the ResNet50 backbone network and the Res2Net50 backbone network to obtain the deep feature With the deep features .

[0017] In a possible implementation manner, in step S3, a The same size and all channel pixel values ​​are all 0 as the additive perturbation , and create a The image with the same size and a single channel pixel value of all 1 and the other channel pixel values ​​of all 0 is used as the multiplicative perturbation .

[0018] In a possible implementation, in step S5 and step S7, the structural similarity is obtained by SSIM calculation.

[0019] In a possible implementation, in step S2, the last average pooling layer and the fully connected layer of the ResNet50 backbone network and the Res2Net50 backbone network are removed, and the fifth layer of the ResNet50 backbone network and the Res2Net50 backbone network are used as output ends to obtain the deep features. With the deep features .

[0020] In a possible implementation, in step S6, the adversarial example And each of the adversarial examples The adversarial example is also included before being input to the deep feature extractor And each of the adversarial examples Perform normalization.

[0021] In a possible implementation, the normalization process includes:

[0022] The adversarial example And each of the adversarial examples The size is scaled to A scaled image is obtained, and then the pixel values ​​of all pixels in the R channel of the scaled image are normalized to a mean of 0.485 and a variance of 0.229, the pixel values ​​of all pixels in the G channel are normalized to a mean of 0.456 and a variance of 0.224, and the pixel values ​​of all pixels in the B channel are normalized to a mean of 0.406 and a variance of 0.225.

[0023] In a possible implementation manner, in step S3, the original image Convert from RGB color space to YUV color space and perturb with the multiplicative Multiply them together to make the multiplicative perturbation The interference image is generated in the YUV color space, and then the interference image is converted to the RGB color space and combined with the additive perturbation Add together to make the additive disturbance Generate interference in the RGB color space to obtain the adversarial example .

[0024] In a possible implementation manner, in step S3, after the original image is After converting from RGB color space to YUV color space, the multiplicative perturbation The channel with all pixel values ​​1 and the original image The Y channel of the original image is multiplied pixel by pixel, and the remaining two channels with all pixel values ​​​​are multiplied by the original image The U channel and the V channel are multiplied pixel by pixel to obtain the interference image. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of the steps of the present invention;

[0026] Figure 2 A schematic diagram of the adversarial example augmentation framework of the present invention;

[0027] Figure 3 The following is a graph showing some detection effects of the method of the present invention and existing attack methods on different data sets. DETAILED DESCRIPTION

[0028] First, those skilled in the art should understand that these implementations are only used to explain the technical principles of the embodiments of the present invention, and are not intended to limit the protection scope of the embodiments of the present invention. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0029] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] See also Figure 1 and Figure 2 The embodiment of the present invention discloses a highly transferable black box attack method based on adversarial example augmentation, comprising the following steps:

[0031] Step S1: Select an image to generate its adversarial example , the original image is , original image The size is , select an existing two-category image segmentation model as a proxy model, and be able to obtain the gradient information of this proxy model, use an existing two-category image segmentation model to predict the selected image and generate a prediction result, and use the result as the pseudo label of the image, denoted as ;

[0032] Step S2, using the ResNet50 backbone network and the Res2Net50 backbone network to build a deep feature extractor, the fifth layer output of the two backbone networks is used as the output end to obtain the deep features of the image, the ResNet50 backbone network and the Res2Net50 backbone network both remove the last average pooling layer and the fully connected layer, and directly use the fifth layer as the output. Before all images enter the ResNet50 backbone network and the Res2Net50 backbone network, the images need to be preprocessed: First, the size of the original image is scaled to ; Secondly, the scaled image is normalized, and the pixel values ​​of all pixels in the R channel of the scaled image are normalized to a mean of 0.485 and a variance of 0.229, the pixel values ​​of all pixels in the G channel are normalized to a mean of 0.456 and a variance of 0.224, and the pixel values ​​of all pixels in the B channel are normalized to a mean of 0.406 and a variance of 0.225; the feature map of the fifth layer of the ResNet50 backbone network is recorded as , The size is , the feature map of the fifth layer of Res2Net50 is recorded as , The size is ;

[0033] Step S3, create a The same image with all pixel values ​​0 is used to optimize the additive perturbation, denoted as ; Create a new image with the same size as the original image The same image with all pixel values ​​1 in a single channel and all pixel values ​​0 in other channels is used to optimize the multiplicative perturbation, denoted as ; Initialize adversarial examples , the original image Convert from RGB color space to YUV color space and perturb with multiplicative Generate interference in the brightness space of the image, then convert the resulting interference image into RGB color space and add the perturbation Additive disturbance The overall interference is generated in the image space, and the process is recorded as ;

[0034] Step S4: Initialize the adversarial example Input into the proxy model to get the prediction result recorded as , using the binary cross entropy loss function to calculate the prediction results With pseudo labels The loss between ; Use the infinite norm to calculate the original image With adversarial examples The difference between ;

[0035] Step S5, back propagation optimization objective function , update the additive perturbation With multiplicative perturbation , when adversarial examples Prediction results on the surrogate model With pseudo labels The structural similarity between Keep this picture as , , where N represents the number of images in the adversarial example set;

[0036] Step S6, repeat steps S3, S4, and S5 to generate new adversarial examples , the adversarial example Input into the deep feature extractor to obtain its deepest features. This process is recorded as , the generated adversarial examples Input into a deep feature extractor to obtain adversarial examples The deepest feature of ;

[0037] Step S7, calculate deep features With deep features The KL divergence between and construct the second objective function , is the preset distance value;

[0038] Step S8: Repeat steps S6 and S7 to continuously generate new adversarial examples. And save it to achieve the effect of adversarial example augmentation, obtain the adversarial example set, use the images in the adversarial example set to query the target model, and the target model’s prediction result with the largest difference from the pseudo-label is the optimal adversarial example. The difference between the two is calculated using the binary cross entropy loss function.

[0039] Continue to see Figure 1In step S1, the proxy model is selected as the camouflaged target detection model SINet, and the pseudo label is generated by the camouflaged target detection model FSPnet. Both SINet and FSPnet are existing structural frameworks, and their network structures have been made public, such as: the SINet network model has been recorded in the reference Fan, Deng-Ping, et al. "Camouflaged object detection." Proceedingsof the IEEE / CVF conference on computer vision and pattern recognition. 2020. ("Camouflaged target detection"), and the FSPnet network model has been recorded in Huang, Zhou, et al. "Featureshrinkage pyramid for camouflaged object detection with transformers." Proceedings of the IEEE / CVF conference on computer vision and patternrecognition. 2023.

[0040] Continue to see Figure 1 In step S2, the image preprocessing process is: using existing technology to scale the image size to , Since the input of deep neural networks usually has a fixed form, the original image needs to be preprocessed. Here, the normalization process is only performed on the original image, while the pseudo-label corresponding to the original image is not normalized. The ResNet50 backbone network and the Res2Net50 backbone network are both existing structural frameworks, and their network structures have been made public, such as: Reference K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. ("Image Recognition Based on Deep Residual Learning") have recorded the ResNet50 backbone network, Gao, Shang-Hua, et al. "Res2net: A new multi-scale backbone architecture." IEEE transactions on pattern analysis and machine intelligence 43.2 (2019): The Res2Net50 backbone network is documented in 652-662. (“Res2Net: A New Multi-Scale Network Architecture”).

[0041] Continue to see Figure 1 , the initial additive perturbation With multiplicative perturbation The size of the image is the same as the original image. , is an all-zero image, multiplicative perturbation In a single channel, it is all 1, and the other two channels are all 0. After the original image is converted to the YUV channel, the multiplicative perturbation The all-1 channel of the original image is multiplied pixel by pixel with the Y channel, and the other two all-0 channels are multiplied pixel by pixel with the UV channel of the original image, so that the multiplicative perturbation It only affects the brightness of the image.

[0042] Continue to see Figure 1 In step S5, set is 0.15, is 4 and the maximum number of adversarial examples N is 20.

[0043] Continue to see Figure 1 In step S7, the calculation of KL divergence is to calculate the deep features With deep features Processing, extracting the deepest feature layer at equal intervals and The features of step S2 are constructed into a new feature layer for KL calculation to reduce the amount of calculation. Extraction The size is ,right Extraction The size is , overall, the generated adversarial examples Input into two feature extraction networks, and then extract at equal intervals to obtain and , where the KL divergence is calculated as and The divergence between and The sum of the divergences between The distance is set to 500; optimize the overall objective function , enlarging the difference between the prediction results of the new adversarial examples on the proxy model and the pseudo-labels, while making the minimum KL divergence between the generated adversarial examples and the new adversarial examples close to , so as to achieve the purpose of maintaining the difference in features between the new and old adversarial examples. When the structural similarity between the prediction results of the adversarial examples on the proxy model and the pseudo-label is lower than and Less than the preset value Keep this adversarial example when Set to 0.001, Set to 250, and stop generating adversarial examples if the above conditions are not met.

[0044] Continue to see Figure 1 In order to further verify the feasibility and effectiveness of the method of the present invention, the method of the present invention was tested:

[0045] The experiment was conducted on the salient object detection task and the camouflaged object detection task, and the existing ECSSD dataset, COD10K dataset, and CAMO dataset were selected for testing.

[0046] Four widely used evaluation indicators are used, including mean absolute error , average precision , structural similarity index , weighted regional similarity metrics , specifically, the mean absolute error The indicator is used to evaluate the pixel-level accuracy between the predicted results and the true labels; the average precision Evaluate the position accuracy between the predicted results and the true labels; structural similarity index It is used to measure the structural similarity between the predicted result and the true label. In the index test of adversarial attack, the worse the index result is, the better the attack effect is.

[0047] Table 1 Quantitative comparison of the evaluation indicators of the proposed method and common object segmentation methods on the ECSSD dataset under the SOD task

[0048] ;

[0049] Table 2 Quantitative comparison of the evaluation indicators of the proposed method and common target segmentation methods on the CAMO dataset under the COD task

[0050] ;

[0051] Table 3 Quantitative comparison of the evaluation indicators of the proposed method and common target segmentation methods on the COD10K dataset under the COD task

[0052] ;

[0053] Table 4 HDR-VDP quality scores of adversarial images generated by the method of the present invention and common attack methods

[0054] ;

[0055] Tables 1, 2 and 3 above give quantitative comparisons of the evaluation indicators of the method of the present invention and common attack methods using the ECSSD dataset, COD10K and CAMO datasets under SOD and COD tasks; Table 4 above gives the HDR-VDP quality scores of the adversarial images generated by the method of the present invention and common attack methods. Figure 3 The following are some graphs showing the detection effects of the method of the present invention and existing attack methods on different data sets. It can be found that the method of the present invention has a lower evaluation index, a higher quality score, and a better attack effect.

[0056] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" etc. means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0057] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A highly transferable black-box attack method based on adversarial example augmentation, characterized in that The following steps are involved: Step S1, obtain any picture as the original image , and call an image two-class segmentation model as a proxy model to convert the original image Input into the proxy model to obtain the original image Pseudo-labels ; Step S2, building a deep feature extractor; Step S3, based on the original image Creating additive perturbations Sum multiplicative disturbance , and according to the original image , the additive perturbation and the multiplicative perturbation Get adversarial examples ; Step S4: Input into the proxy model to obtain the prediction result , and use the binary cross entropy loss function to calculate the prediction result and the pseudo-label The loss between , and using the infinite norm to calculate the original image With the adversarial example The difference between ; Step S5, back propagation optimization objective function , update the additive perturbation With the multiplicative perturbation , and calculate the prediction results and the pseudo-label The structural similarity between them, and when the structural similarity is below a preset threshold The corresponding adversarial example Store as adversarial examples ,in , Indicates preset parameters; Step S6, repeatedly executing step S3, step S4, and step S5, and performing the adversarial example obtained after each repetition. As new adversarial examples , the adversarial example Input into the deep feature extractor to obtain deep features , and the generated adversarial examples Input into the deep feature extractor to obtain deep features ,in ; Step S7, calculating the deep features With the deep features The KL divergence between them is used to construct the objective function based on the KL divergence. , and optimize the overall objective function To expand the adversarial examples The corresponding prediction results and the pseudo-label When the predicted results and the pseudo-label The structural similarity between them is lower than the preset threshold and Less than the preset threshold , retain the corresponding adversarial example ,in Indicates the preset distance value. Indicates preset parameters; Step S8, repeatedly executing step S6 and step S7, and retaining the adversarial examples after each repetition Integrate into an adversarial example set, using each of the adversarial examples in the adversarial example set The target model is queried to obtain the corresponding target prediction result and target pseudo label, and the adversarial example with the largest difference between the target prediction result and the target pseudo label is selected. As an optimal adversarial example.

2. The highly transferable black-box attack method based on adversarial example augmentation according to claim 1, characterized in that: In step S2, the deep feature extractor is constructed by using the ResNet50 backbone network and the Res2Net50 backbone network to obtain the deep feature With the deep features .

3. The highly transferable black-box attack method based on adversarial example augmentation according to claim 1, characterized in that: In step S3, a The same size and all channel pixel values ​​are all 0 as the additive perturbation , and create a The image with the same size and a single channel pixel value of all 1 and the other channel pixel values ​​of all 0 is used as the multiplicative perturbation .

4. The highly transferable black-box attack method based on adversarial example augmentation according to claim 1, characterized in that: In step S5 and step S7, the structural similarity is obtained by SSIM calculation.

5. The highly transferable black-box attack method based on adversarial example augmentation according to claim 2, characterized in that: In step S2, the last average pooling layer and the fully connected layer of the ResNet50 backbone network and the Res2Net50 backbone network are removed, and the fifth layer of the ResNet50 backbone network and the Res2Net50 backbone network are used as output ends to obtain the deep features. With the deep features .

6. The highly transferable black-box attack method based on adversarial example augmentation according to claim 2, characterized in that: In step S6, the adversarial example And each of the adversarial examples The adversarial example is also included before being input to the deep feature extractor And each of the adversarial examples Perform normalization.

7. The highly transferable black-box attack method based on adversarial example augmentation according to claim 6, characterized in that: The normalization process includes: The adversarial example And each of the adversarial examples The size is scaled to A scaled image is obtained, and then the pixel values ​​of all pixels in the R channel of the scaled image are normalized to a mean of 0.485 and a variance of 0.229, the pixel values ​​of all pixels in the G channel are normalized to a mean of 0.456 and a variance of 0.224, and the pixel values ​​of all pixels in the B channel are normalized to a mean of 0.406 and a variance of 0.

225.

8. The highly transferable black-box attack method based on adversarial example augmentation according to claim 7, characterized in that: In step S3, the original image Convert from RGB color space to YUV color space and perturb with the multiplicative Multiply them together to make the multiplicative perturbation The interference image is generated in the YUV color space, and then the interference image is converted to the RGB color space and combined with the additive perturbation Add together to make the additive disturbance Generate interference in the RGB color space to obtain the adversarial example .

9. The highly transferable black-box attack method based on adversarial example augmentation according to claim 8, characterized in that: In step S3, the original image After converting from RGB color space to YUV color space, the multiplicative perturbation The channel with all pixel values ​​1 and the original image The Y channel of the original image is multiplied pixel by pixel, and the remaining two channels with all pixel values ​​​​are multiplied by the original image The U channel and the V channel are multiplied pixel by pixel to obtain the interference image.

Images