Confrontation sample generation method and system, electronic equipment and storage medium

Abstract

The invention belongs to a sample generation method, and provides an adversarial sample generation method and system, an electronic device and a storage medium for solving the technical problem that an existing adversarial attack method for target detection is poor in universality, a target image is input into a target detection model, and multiple prediction results of the target detection model are obtained. Obtaining a feature map output by a backbone network of the target detection model, generating a corresponding attention map according to a plurality of prediction results, obtaining a feature map after attention map enhancement according to the attention map and the feature map, adjusting the shape of the feature map after attention map enhancement, and calculating the standard deviation of each channel of the adjusted feature map; as an index for evaluating the dispersion degree of the feature map, the generated confrontation sample can effectively attack the target detection model by reducing the dispersion degree of the feature map. And then, by taking the minimum sum of the standard deviations of the channels of the adjusted feature map as an optimization target, generating an adversarial sample in combination with an algorithm for generating the adversarial sample.

Description

Technical Field

[0001] The present application relates to a sample generation method, and specifically to an adversarial sample generation method, system, electronic device, and storage medium. Background Art

[0002] Adversarial attacks are an important direction in the research of deep learning model security. Traditional adversarial attacks mainly focus on image classification tasks, and their goal is to mislead the model into making incorrect classifications by adding carefully designed small perturbations to the image. In recent years, the emergence of attention maps has provided attackers with a new means to enhance the attack capabilities of adversarial samples. Through the attention maps generated by the model, attackers can observe the important image regions that the model relies on when making predictions. Using this information, attackers can design targeted perturbations that act on the most sensitive features of the model. However, existing adversarial attack methods for target detection are often less versatile and need to be modified according to different target detection model architectures. Summary of the invention

[0003] In order to solve the technical problem that existing adversarial attack methods for target detection have poor versatility, the present application provides an adversarial sample generation method, system, electronic device, and storage medium.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions: In the first aspect, the present application proposes a method for generating adversarial samples, including: Input the target image into the target detection model, obtain multiple prediction results of the target detection model, and obtain the feature map output by the backbone network of the target detection model; the target detection model is a deep learning model that performs image classification tasks; the multiple prediction results are results with different probabilities predicted by the target detection model; Generate corresponding attention maps based on multiple prediction results; According to the attention map and the feature map, obtaining a feature map enhanced by the attention map; Adjusting the shape of the feature map after the attention map enhancement to obtain an adjusted feature map; Calculate the standard deviation of each channel of the adjusted feature map; The optimization goal is to minimize the sum of the standard deviations of each channel of the adjusted feature map, and the adversarial sample is generated in combination with the algorithm used to generate adversarial samples.

[0005] Furthermore, generating a corresponding attention map according to the multiple prediction results includes: Use an interpretable algorithm to generate corresponding attention maps using multiple prediction results.

[0006] Furthermore, obtaining the feature map enhanced by the attention map according to the attention map and the feature map includes: The attention map is multiplied by the feature map point by point to obtain a feature map enhanced by the attention map.

[0007] Further, the adjusting the shape of the feature map after the attention map enhancement includes: The height and width of the feature map enhanced by the attention map are merged, and the number of channels is kept unchanged.

[0008] Furthermore, the generating of the adversarial sample in combination with the algorithm for generating the adversarial sample includes: Iteratively generate adversarial samples through the momentum iterative fast gradient sign method:

[0009]

[0010]

[0011]

[0012] in, Indicates t +The accumulated gradient at 1 iteration, Indicates t The accumulated gradient at iterations, represents the cumulative coefficient, Represents the loss function relative to the input image The gradient of express The one-norm of Indicates t The adversarial example at the iteration, represents the attack step length, Represents the input image The corresponding adversarial example, represents the clipping function, Represents the activation map The number of pixels in The output of the last convolutional layer of the backbone network of the target detection model An activation map; The loss function includes:

[0013] in, represents the loss function, Represents the adjusted feature map The standard deviation calculated for each channel, Represents the total number of channels in the resized feature map.

[0014] In the second aspect, the present application proposes an adversarial sample generation system, comprising: A prediction module is used to input a target image into a target detection model, obtain multiple prediction results of the target detection model, and obtain a feature map output by a backbone network of the target detection model; the target detection model is a deep learning model that performs image classification tasks; the multiple prediction results are results of different probabilities predicted by the target detection model; Attention module, used to generate corresponding attention maps based on multiple prediction results; An enhancement module, used for obtaining a feature map enhanced by the attention map according to the attention map and the feature map; An adjustment module, used for adjusting the shape of the feature map after the attention map enhancement to obtain an adjusted feature map; A calculation module, used to calculate the standard deviation of each channel of the adjusted feature map; The sample module is used to generate adversarial samples by taking the minimum sum of the standard deviations of each channel of the adjusted feature map as the optimization goal and combining it with the algorithm for generating adversarial samples.

[0015] Furthermore, generating a corresponding attention map according to the multiple prediction results includes: Use an interpretable algorithm to generate corresponding attention maps using multiple prediction results.

[0016] Furthermore, obtaining the feature map enhanced by the attention map according to the attention map and the feature map includes: The attention map is multiplied by the feature map point by point to obtain a feature map enhanced by the attention map.

[0017] Further, the adjusting the shape of the feature map after the attention map enhancement includes: The height and width of the feature map enhanced by the attention map are merged, and the number of channels is kept unchanged.

[0018] Furthermore, the generating of the adversarial sample in combination with the algorithm for generating the adversarial sample includes: Iteratively generate adversarial samples through the momentum iterative fast gradient sign method:

[0019]

[0020]

[0021]

[0022] in, Indicates t +The accumulated gradient at 1 iteration, Indicates t The accumulated gradient at iterations, represents the cumulative coefficient, Represents the loss function relative to the input image The gradient of express The one-norm of Indicates t The adversarial example at the iteration, represents the attack step length, Represents the input image The corresponding adversarial example, represents the clipping function, Represents the activation map The number of pixels in The output of the last convolutional layer of the backbone network of the target detection model An activation map; The loss function includes:

[0023] in, represents the loss function, Represents the adjusted feature map The standard deviation calculated for each channel, Represents the total number of channels in the resized feature map.

[0024] In a third aspect, the present application provides an electronic device, comprising: Memory for storing computer programs; A processor is used to implement the steps of the above-mentioned adversarial sample generation method when executing the computer program.

[0025] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned adversarial sample generation method are implemented.

[0026] Compared with the prior art, this application has the following beneficial effects: The present application proposes a method for generating adversarial samples, inputting a target image into a target detection model, obtaining multiple prediction results of the target detection model, and obtaining a feature map output by the backbone network of the target detection model, generating a corresponding attention map according to the multiple prediction results, and obtaining a feature map enhanced by the attention map according to the attention map and the feature map, which can more accurately locate important areas that have a significant impact on the prediction results. Then adjust the shape of the feature map enhanced by the attention map to obtain the adjusted feature map, calculate the standard deviation of each channel of the adjusted feature map as an indicator for evaluating the degree of dispersion of the feature map, and reduce the dispersion of the feature map so that the generated adversarial sample can achieve an effective attack on the target detection model. Then, the sum of the standard deviations of each channel of the adjusted feature map is minimized as the optimization goal, and the adversarial sample is generated in combination with the algorithm for generating adversarial samples. Therefore, the present application only needs to focus on the feature map of the target detection model, without considering the difference in loss functions of different target detection models, and the generated adversarial samples have strong versatility and transferability.

[0027] This application also proposes an adversarial sample generation system, an electronic device and a computer storage medium, which have all the advantages of the above-mentioned adversarial sample generation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 A first flowchart of a method for generating adversarial samples in this application; Figure 2 A second flow chart of a method for generating adversarial samples in this application; Figure 3 Comparison of the detection results of the adversarial samples generated for this application on the target detection model Faster RCNN; Figure 4 Comparison of the detection results of the adversarial samples generated for this application on the target detection model Retina Net; Figure 5 Comparison of the detection results of the adversarial samples generated for this application on the target detection model Mask ECNN; Figure 6 Comparison of the detection results of the adversarial samples generated for this application on the target detection model YOLOv5; Figure 7A schematic diagram of an adversarial sample generation system for this application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0031] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0032] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0033] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0034] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0035] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0036] With the rapid development of artificial intelligence technology, deep learning has made remarkable achievements in the fields of image processing and object detection. Object detection models, such as Faster R-CNN and YOLO, have been widely used in various scenarios, including autonomous driving, video surveillance, and medical image analysis. At the same time, the security issues of these object detection models have gradually become the focus of research.

[0037] Adversarial attacks are an important direction in the research of deep learning model security. By actively generating adversarial samples and training deep learning models, the impact of adversarial attacks on deep learning models can be reduced, thereby improving the detection accuracy and anti-attack capabilities of deep learning models.

[0038] Based on the above situation, the Chinese invention patent with publication number CN114969728A proposes an attack method for generating adversarial samples based on the attention map attack on important areas. The method obtains the attention map of the image for the category and obtains a binary matrix according to the set threshold. The value of 1 in the binary matrix represents the important area, and the value of 0 represents the ordinary area. In the subsequent attack, the method only chooses to add perturbations to the important area, thereby alleviating the shortcomings of excessive perturbations in the current attack and easy recognition by the human eye. Although this technology has achieved certain results in image classification tasks, it still faces new challenges when applied to target detection tasks. Target detection not only requires the model to identify multiple targets in the image, but also requires accurate prediction of the location of the target. The application of adversarial attacks in the field of target detection usually requires the target to be missed or misdetected. The Chinese invention patent application with publication number CN116935186A proposes a target detection attack method using an attention map. The method uses the prediction box generated by the target detection model and the category attention map, and takes the intersection of the generated prediction box and the salient area of ​​the attention map as the initial perturbation area. In subsequent attacks, this method only adds disturbances in the initial disturbance area to generate the final adversarial sample. This method can not only attack the target detection model, but also ensure good visual invisibility. However, this method has poor versatility and needs to be modified according to the characteristics of different target detection model architectures. In addition, the method has weak migration attack capabilities.

[0039] In summary, the transferability of adversarial examples refers to the ability of adversarial examples generated on one model to maintain their adversarial effects on another model. Existing adversarial attack methods often lack transferability, which means that adversarial examples are effective on the original model, but their attack effects will drop rapidly after being transferred to other models. This limitation seriously restricts the practicality and widespread use of adversarial attack methods. This is because when generating adversarial examples, the importance of internal features of the model is often ignored, the model's attention mechanism is not fully utilized, and the model's sensitivity to key feature areas is ignored, which affects the ability of adversarial examples to transfer between different models.

[0040] Therefore, the existing adversarial attack methods have poor transferability between different target detection models, resulting in adversarial samples being effective only on specific models and not effective on other models, which limits the breadth and effectiveness of adversarial attacks in practical applications. In addition, existing adversarial attack methods usually need to rely on the parameters and loss functions of specific models, lack universality, and are difficult to transfer and apply between different types of target detection models.

[0041] In response to the above problems, this application proposes a gray-box adversarial attack method that uses attention maps to assist in improving the transferability of adversarial samples. A gray-box attack is an adversarial sample generated on a substitute model that can produce a migration attack effect on an unknown target model. It not only improves the transferability of adversarial samples, but also has strong versatility and is easy to use between different target detection models. The application is described in detail below in conjunction with the embodiments and drawings.

[0042] like Figure 1 As shown, it is a first flow chart of a method for generating adversarial samples in the present application, which may include: S101, input the target image into the target detection model, obtain multiple prediction results of the target detection model, and obtain the feature map output by the backbone network of the target detection model; the target detection model is a deep learning model that performs image classification tasks; the multiple prediction results are results with different probabilities predicted by the target detection model.

[0043] The target detection model can solve two problems in the image: one is where the multiple targets in the image are, that is, the location; the other is what these targets are, that is, the category. Therefore, the target detection model not only needs to classify the objects in the image, but also needs to accurately locate the bounding box of each object. The backbone network of the target detection model is usually a convolutional neural network, which is mainly responsible for extracting features in the image. These features will be used by subsequent modules for target detection and positioning. The output of the target detection model usually includes category prediction, location prediction and confidence. The target detection model gives the probability of each detected object belonging to each category, and gives the bounding box coordinates of each detected object. It also gives a confidence score for each detected object to indicate the confidence of the target detection model in the prediction result.

[0044] S102, generating a corresponding attention map according to multiple prediction results.

[0045] In object detection models, attention maps are often used to reveal the image regions that the object detection model focuses on when making predictions.

[0046] S103, obtaining a feature map enhanced by the attention map according to the attention map and the feature map.

[0047] The feature map enhanced by the attention map combines the original feature map with the weights of the attention map to highlight the areas in the feature map that the object detection model pays attention to. This process involves applying the weight values ​​of the attention map to each pixel or feature vector of the feature map so that these areas can be given more attention in subsequent processing.

[0048] S104, adjusting the shape of the feature map after the attention map enhancement to obtain an adjusted feature map.

[0049] The purpose of adjusting the shape of the feature map in this application is to enhance the representation capability of the features while also being able to adapt to the needs of subsequent steps.

[0050] S105, calculating the standard deviation of each channel of the adjusted feature map.

[0051] The standard deviation of a channel is a statistic that measures the degree of dispersion of pixel values ​​in a channel in a feature map. It is used to represent the average distance or dispersion between all pixel values ​​in the channel and the mean. The larger the standard deviation, the wider the distribution of pixel values ​​in the channel; the smaller the standard deviation, the more concentrated the distribution of pixel values.

[0052] S106, taking the minimum sum of the standard deviations of each channel of the adjusted feature graph as the optimization goal, and combining the algorithm for generating adversarial samples to generate adversarial samples.

[0053] It should be noted that minimizing the sum of the standard deviations of each channel of the adjusted feature map means that we hope that the generated adversarial samples are as close to the distribution of the original samples as possible in the feature space, but misleading in some key aspects. In practical applications, the choice of adversarial sample generation algorithm can be determined as needed, such as the fast gradient sign method and the basic iterative method. These algorithms usually generate adversarial samples by calculating the gradient of the model to the input. In the iterative process of generating adversarial samples, the standard deviation of the adjusted feature map is calculated at each step and used as part of the optimization target. By continuously adjusting the input samples, the sum of the standard deviations is minimized, while ensuring that the adversarial samples can still effectively mislead the model.

[0054] By combining the minimization of the sum of the standard deviations of each channel of the adjusted feature map with the adversarial sample generation algorithm, adversarial samples that are both misleading and as close to the original sample distribution as possible in the feature space can be generated. This method has the potential to improve the concealment and robustness of adversarial samples while maintaining their ability to mislead the model.

[0055] In this application, the adversarial attack method based on attention map enhancement and feature map standard deviation optimization can more accurately locate and attack important areas that have a significant impact on the prediction results by using attention maps to enhance feature maps. By reducing the dispersion of the enhanced feature map, the generated adversarial samples can effectively attack the target detection model. Since this method only needs to focus on the feature map of the model (the backbone network part) without considering the difference in loss functions of different models, it has strong versatility. The adversarial samples generated by the proposed method have stronger transferability and show excellent attack effects in different target detection models.

[0056] like Figure 2 As shown, it is a second flow chart of a method for generating adversarial samples in the present application, which may include: S201, using the original image data to train the target detection model, so that the target detection model performs well on the normal data set.

[0057] The target detection model can be trained using commonly used training methods to ensure that the target detection model can perform predictions normally.

[0058] S202, the target image Input the trained target detection model, and the model outputs multiple prediction results At the same time, obtain the feature map output by the backbone network of the target detection model .

[0059] Among them, The prediction results are expressed as , represents the bounding box, represents the target score, Represents the predicted probability of each category.

[0060] S203, using interpretable algorithms, using prediction results Generating Attention Map H , which is used to show the important areas in the input image that have a positive impact on the prediction results.

[0061] Use interpretable methods to analyze the decision-making process of the model, calculate the contribution or importance of each part of the input data to the prediction result, and use these contribution levels to generate a visual attention map. In practical applications, interpretable methods are applied to analyze the decision-making process of the model on the input data, which can be gradient-based, local linear approximation-based, feature importance-based, or model distillation-based methods. Under the framework of interpretable methods, calculate the contribution or importance of each part of the input data to the prediction result. Finally, use the calculated attention weights to generate an attention map, which is usually a two-dimensional matrix with the same dimension as the input data. By visualizing these contribution levels, an intuitive attention map can be obtained, which reveals which parts of the input data the model pays attention to when making predictions.

[0062] The attacker aims to exploit the attention map of the target image To highlight the areas that have a significant impact on the prediction results, it provides guidance for enhancing the feature map. Attention map , the following methods can be used:

[0063]

[0064]

[0065]

[0066] in, Indicates Bounding box categories The predicted probability of represents the maximum value of the predicted category probability among all bounding boxes, Indicates The weights of the activation maps, The output of the last convolutional layer of the backbone network of the target detection model An activation map, express Middle i Channel pixels j The value of Represents the activation map The number of pixels in Represents the activation function.

[0067] S204, the generated attention map With feature map Multiply by points to get the feature map enhanced by the attention map .

[0068] This process can be regarded as a weighted operation on the original feature map, where the attention map Provides weights. In practical applications, it is necessary to ensure that the attention map With feature map The dimensions of are matched. If they have different dimensions, they need to be adjusted appropriately to have the same shape. After point-wise multiplication, the feature map enhanced by the attention map Each element in is a feature map Corresponding elements and intention diagrams The product of the corresponding weights in the attention graph The area with higher weight in the feature map after attention map enhancement The areas with higher weights will be enlarged accordingly, while areas with lower weights will be reduced.

[0069] The following methods can be used for point multiplication: Given a target image , using the target detection model Prediction can extract feature maps :

[0070]

[0071] in, represents the activation function, Represents element-wise multiplication.

[0072] S205, feature map after attention map enhancement After changing the shape, the standard deviation is calculated for each channel , as an indicator to evaluate the dispersion of feature maps.

[0073] For the enhanced feature map The shape is changed to meet the needs of subsequent processing steps. Feature map Reshape Shape feature map , this transformation process merges the two spatial dimensions (height and width) of the feature map while keeping the number of channels unchanged. By calculating the standard deviation of each channel, we can get .

[0074] S206, setting the optimization goal to minimize the standard deviation of each channel The sum of , combined with the momentum iterative fast gradient sign method, generates the final adversarial sample .

[0075] According to the optimization goal, the loss function can be expressed as:

[0076] in, Represents the feature map after shape change The The standard deviation calculated for each channel, Represents the loss function.

[0077] According to the loss function, the momentum iterative fast gradient sign method can iteratively generate adversarial samples:

[0078]

[0079]

[0080]

[0081] in, Indicates t +The accumulated gradient at 1 iteration, Indicates t The accumulated gradient at iterations, represents the cumulative coefficient, Represents the loss function relative to the input image The gradient of express The one-norm of Indicates t The adversarial example at the iteration, represents the attack step length, Represents the input image The corresponding adversarial example, Indicates t +1 adversarial example at iteration 1, represents the cropping function to ensure that the final generated adversarial sample is a legitimate image, Represents the activation map The number of pixels in The output of the last convolutional layer of the backbone network of the target detection model An activation map, is a symbolic function This application introduces an interpretable algorithm to generate an attention map based on the prediction results, and uses the dot product between the attention map and the feature map to obtain the feature map after the attention map is enhanced, which can more accurately locate the important areas that have a significant impact on the prediction results. This application performs shape change processing on the enhanced feature map, calculates the standard deviation of each channel as an indicator to evaluate the degree of dispersion of the feature map, and reduces the dispersion of the feature map, so that the generated adversarial samples can achieve an effective attack on the target detection model. In addition, this application only needs to focus on the feature map of the target detection model, without considering the differences in loss functions of different models, and has strong versatility and transferability.

[0082] In order to verify the adversarial sample generation method of this application, the adversarial samples generated by this application are used to perform transfer attack tests on four target detection models, Faster RCNN, RetinaNet, Mask RCNN and YOLOv5. The specific attack results are shown in the figure. Figures 3 to 6 As shown, Figure 3 Comparison of the detection results of the adversarial samples generated for this application on the target detection model Faster RCNN. Figure 4 Comparison of the detection results of the adversarial samples generated for this application on the target detection model RetinaNet. Figure 5 Comparison of the detection results of the adversarial samples generated for this application on the target detection model Mask ECNN, Figure 6 Comparison of the detection results of the adversarial samples generated for this application on the target detection model YOLOv5. Figures 3 to 6 In the figure, the left side are all original images, and the right side are all adversarial samples. Figures 3 to 6 It can be observed that for adversarial samples, the target detection model predicts a large number of irrelevant bounding boxes. The specific attack effect is shown in Table 1. It can be seen that the adversarial attack method of this application has been tested on multiple target detection models, including Faster R-CNN, RetinaNet, Mask R-CNN and YOLOv5. The mAP of the original image is 72.69%, 74.43%, 76.79% and 67.60%, respectively, which proves the efficient performance of these target detection models under standard test conditions. However, when predicting the adversarial samples generated by this application, the accuracy of these models dropped significantly to 3.37%, 8.43%, 6.74% and 14.92%, respectively. This clearly shows that the adversarial samples produced by the method proposed in this application have good migration capabilities.

[0083] Table 1 Comparison of mAP of target detection models before and after attack

[0084] The above experiments and related result analysis verify the effectiveness of the attack method corresponding to the adversarial samples obtained in this application.

[0085] Based on the above adversarial sample generation method, such as Figure 7 As shown, the present application also proposes an adversarial sample generation system, which may include: A prediction module is used to input a target image into a target detection model, obtain multiple prediction results of the target detection model, and obtain a feature map output by a backbone network of the target detection model; the target detection model is a deep learning model that performs image classification tasks; the multiple prediction results are results of different probabilities predicted by the target detection model; Attention module, used to generate corresponding attention maps based on multiple prediction results; An enhancement module, used for obtaining a feature map enhanced by the attention map according to the attention map and the feature map; An adjustment module, used for adjusting the shape of the feature map after the attention map enhancement to obtain an adjusted feature map; A calculation module, used to calculate the standard deviation of each channel of the adjusted feature map; The sample module is used to generate adversarial samples by taking the minimum sum of the standard deviations of each channel of the adjusted feature map as the optimization goal and combining it with the algorithm for generating adversarial samples.

[0086] In some embodiments of the adversarial sample generation system of the present application, when generating a corresponding attention map based on multiple prediction results, an interpretable algorithm can be used to generate the corresponding attention map using multiple prediction results.

[0087] In some embodiments of the adversarial sample generation system of the present application, when obtaining a feature map enhanced by the attention map based on the attention map and the feature map, the attention map and the feature map can be multiplied point by point to obtain the feature map enhanced by the attention map.

[0088] In some embodiments of the adversarial sample generation system of the present application, when adjusting the shape of the feature map after attention map enhancement, the height and width of the feature map after attention map enhancement can be merged, and the number of channels can be kept unchanged.

[0089] In some embodiments of the adversarial sample generation system of the present application, when generating adversarial samples in combination with an algorithm for generating adversarial samples, adversarial samples can be iteratively generated by using a momentum iterative fast gradient sign method:

[0090]

[0091]

[0092]

[0093] in, Indicates t +The accumulated gradient at 1 iteration, Indicates t The accumulated gradient at iterations, represents the cumulative coefficient, Represents the loss function relative to the input image The gradient of express The one-norm of Indicates t The adversarial example at the iteration, represents the attack step length, Represents the input image The corresponding adversarial example, represents the clipping function, Represents the activation map The number of pixels in The output of the last convolutional layer of the backbone network of the target detection model An activation map; Loss functions, including:

[0094] in, represents the loss function, Represents the adjusted feature map The standard deviation calculated for each channel, Represents the total number of channels in the resized feature map.

[0095] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0096] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0097] An electronic device provided in an embodiment of the present application includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the adversarial sample generation method described in any of the above embodiments are implemented.

[0098] Another electronic device provided in the embodiment of the present application may also include: an input port connected to the processor, used to transmit multimodal data collected by an external acquisition device to the processor; and a display unit connected to the processor, used to display the processing results of the processor to the outside world; a communication module connected to the processor, used to realize the communication between the electronic device and the outside world. The display unit can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module includes but is not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless connection (including wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s).

[0099] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the adversarial sample generation method described in any of the above embodiments are implemented.

[0100] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0101] For the description of the relevant parts of the adversarial sample generation system, electronic device, and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the adversarial sample generation method provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0102] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for generating adversarial samples, characterized in that: include: Input the target image into the target detection model, obtain multiple prediction results of the target detection model, and obtain the feature map output by the backbone network of the target detection model; The target detection model is a deep learning model for performing image classification tasks; the multiple prediction results are results of different probabilities predicted by the target detection model; Generate corresponding attention maps based on multiple prediction results; According to the attention map and the feature map, obtaining a feature map enhanced by the attention map; Adjusting the shape of the feature map after the attention map enhancement to obtain an adjusted feature map; Calculate the standard deviation of each channel of the adjusted feature map; The optimization goal is to minimize the sum of the standard deviations of each channel of the adjusted feature map, and the adversarial sample is generated in combination with the algorithm used to generate adversarial samples.

2. The adversarial sample generation method according to claim 1, characterized in that: The generating a corresponding attention map according to the multiple prediction results includes: Use an interpretable algorithm to generate corresponding attention maps using multiple prediction results.

3. The adversarial sample generation method according to claim 2, characterized in that: The step of obtaining a feature map enhanced by the attention map according to the attention map and the feature map includes: The attention map is multiplied by the feature map point by point to obtain a feature map enhanced by the attention map.

4. The adversarial sample generation method according to claim 3, characterized in that: The adjusting the shape of the feature map after the attention map enhancement includes: The height and width of the feature map enhanced by the attention map are merged, and the number of channels is kept unchanged.

5. The adversarial sample generation method according to claim 4, characterized in that: The method of generating an adversarial sample by combining an algorithm for generating an adversarial sample includes: Iteratively generate adversarial samples through the momentum iterative fast gradient sign method: in, Indicates t +The accumulated gradient at 1 iteration, Indicates t The accumulated gradient at iterations, represents the cumulative coefficient, Represents the loss function relative to the input image The gradient of express The one-norm of Indicates t The adversarial example at the iteration, represents the attack step length, Represents the input image The corresponding adversarial example, represents the clipping function, Represents the activation map The number of pixels in The output of the last convolutional layer of the backbone network of the target detection model An activation map; The loss function includes: in, represents the loss function, Represents the adjusted feature map The standard deviation calculated for each channel, Represents the total number of channels in the resized feature map.

6. A system for generating adversarial samples, characterized in that: include: The prediction module is used to input the target image into the target detection model, obtain multiple prediction results of the target detection model, and obtain the feature map output by the backbone network of the target detection model; The target detection model is a deep learning model for performing image classification tasks; the multiple prediction results are results of different probabilities predicted by the target detection model; Attention module, used to generate corresponding attention maps based on multiple prediction results; An enhancement module, used for obtaining a feature map enhanced by the attention map according to the attention map and the feature map; An adjustment module, used for adjusting the shape of the feature map after the attention map enhancement to obtain an adjusted feature map; A calculation module, used to calculate the standard deviation of each channel of the adjusted feature map; The sample module is used to generate adversarial samples by taking the minimum sum of the standard deviations of each channel of the adjusted feature map as the optimization goal and combining it with the algorithm for generating adversarial samples.

7. The adversarial sample generation system according to claim 6, characterized in that: The generating a corresponding attention map according to the multiple prediction results includes: Use an interpretable algorithm to generate corresponding attention maps using multiple prediction results.

8. The adversarial sample generation system according to claim 7, characterized in that: The step of obtaining a feature map enhanced by the attention map according to the attention map and the feature map includes: The attention map is multiplied by the feature map point by point to obtain a feature map enhanced by the attention map.

9. The adversarial sample generation system according to claim 8, characterized in that: The adjusting the shape of the feature map after the attention map enhancement includes: The height and width of the feature map enhanced by the attention map are merged, and the number of channels is kept unchanged.

10. The adversarial sample generation system according to claim 9, characterized in that: The method of generating an adversarial sample by combining an algorithm for generating an adversarial sample includes: Iteratively generate adversarial samples through the momentum iterative fast gradient sign method: in, Indicates t +The accumulated gradient at 1 iteration, Indicates t The accumulated gradient at iterations, represents the cumulative coefficient, Represents the loss function relative to the input image The gradient of express The one-norm of Indicates t The adversarial example at the iteration, represents the attack step length, Represents the input image The corresponding adversarial example, represents the clipping function, Represents the activation map The number of pixels in The output of the last convolutional layer of the backbone network of the target detection model An activation map; The loss function includes: in, represents the loss function, Represents the adjusted feature map The standard deviation calculated for each channel, Represents the total number of channels in the resized feature map.

11. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the adversarial sample generation method according to any one of claims 1 to 10 when executing the computer program.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the adversarial sample generation method according to any one of claims 1 to 10 are implemented.

Images