An attention reversal adversarial attack method and device for remote sensing target detection

Abstract

The application provides an attention reversal adversarial attack method and device for remote sensing target detection, creates a derivable tensor as an adversarial sample, and the initial value is a clean sample pixel value. By dimension reduction, linear activation and normalization processing of the feature maps of each layer of the target detector, a clean heat map and an adversarial heat map are generated, and an attention flip loss is constructed based on the clean heat map reversal map and the adversarial heat map deviation. The intersection over union filtering and classification score filtering are used to screen the candidate boxes to be attacked, an optimization target box is set, and a boundary box translation loss is constructed by combining the intersection over union of the predicted candidate box and the real boundary box. At the same time, the background score maximization loss is established by maximizing the classification probability of the background label, and the imperceptibility constraint loss is established based on the deviation of the pixel values between the adversarial sample and the clean sample. The above losses are combined to update the adversarial sample to obtain a target adversarial sample. The method effectively generates an adversarial sample with high attackability and concealment, improves the accuracy and effect of the adversarial attack, and enhances the robustness of the model in a complex environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and adversarial attack, and particularly relates to an attention reversal adversarial attack method and device for remote sensing target detection. BACKGROUND

[0002] Remote sensing image target detection has important application value in the fields of military reconnaissance, disaster monitoring, urban planning, etc. Its core task is to automatically identify and locate specified targets from high-resolution remote sensing images to provide key information support for decision-making. However, with the wide application of deep learning technology in this field, the vulnerability of the model has become increasingly prominent. Deep learning models are vulnerable to adversarial samples. Attackers can cause the model to output incorrect results, such as target misclassification, missed detection or false alarm, by adding carefully designed tiny perturbations to the input image, which seriously threatens the security and reliability of the remote sensing image target detection system.

[0003] Existing adversarial attack methods mainly fall into two categories: adversarial patch attack and pixel-level attack. Adversarial patch attack misleads the detection model by adding a significant perturbation area around or on the target object, but these features are easily identified by defense algorithms, limiting the feasibility and effectiveness of their practical application. Pixel-level attack produces adversarial perturbations by modifying each pixel value in the image, which has stronger concealment, but as the image resolution increases and the model complexity increases, the demand for computing resources and time also increases significantly, further increasing the complexity of the attack strategy. In addition, existing adversarial attack methods usually only interfere at the output end of the model, lacking perturbation of key links in the model decision-making process, resulting in limited attack accuracy and effect, and failing to fully exploit the attack potential of adversarial samples. Therefore, a new adversarial attack scheme is needed. SUMMARY

[0004] In view of this, the embodiments of the present application provide an attention reversal adversarial attack method and device for remote sensing target detection to eliminate or improve one or more defects in the prior art, and solve the problems of low attack accuracy, weak attack performance and poor generality of adversarial samples in the existing adversarial attack method for remote sensing target detection model.

[0005] One aspect of the present application provides an attention reversal adversarial attack method for remote sensing target detection, which comprises the following steps:

[0006] A differentiable tensor is created as an adversarial sample, the original remote sensing image in a preset data set is taken as a clean sample, and the pixel value of the clean sample is taken as the initial value of the tensor;

[0007] The clean samples and the adversarial samples are input into the target detector. The feature maps of each layer in the target detector are reduced in dimension, linearly activated, and normalized to obtain the clean heatmap corresponding to the clean samples and the adversarial heatmap corresponding to the adversarial samples. An attention flip loss is constructed based on the inverted map of the clean heatmap corresponding to each layer in the target detector and the bias of the adversarial heatmap.

[0008] By filtering with intersection-union ratio (IU) and classification score, candidate boxes to be attacked are selected. For each candidate box to be attacked, a corresponding optimized target box is set in the adversarial sample. The adversarial sample is selected by the target detector to predict the corresponding candidate box. The distance between the optimized target box and the predicted candidate box is combined with the IU of the predicted candidate box and the true bounding box in the clean sample to construct the bounding box translation loss.

[0009] For the adversarial sample predicted by the target detector, the background score maximization loss is established by maximizing the classification probability of the background label.

[0010] An imperceptible constraint loss is established based on the deviation of pixel values ​​between the adversarial sample and the clean sample;

[0011] The overall attack objective function is constructed by combining the attention flipping loss, the bounding box translation loss, the background score maximization loss, and the imperceptibility constraint loss to update and optimize the adversarial sample, so as to obtain the target adversarial sample.

[0012] In some embodiments, the feature maps of each layer in the target detector are subjected to dimensionality reduction, linear activation, and normalization to obtain the clean heatmap corresponding to the clean sample, and the calculation formula is:

[0013] ;

[0014] Where i represents the current iteration round, , Indicates the first Width and height of each feature map Indicates the number of channels in the feature map; Used to filter negative values. It is a min-max normalization operation used to adjust the distribution of a heatmap to... Within the range;

[0015] In some embodiments, the inverted graph of the clean heatmap is calculated as follows:

[0016] ;

[0017] in, Indicates when the clean heat map, denotes Z-score normalization, denotes the clean sample heat map denotes the mean value, denotes denotes the standard deviation.

[0018] In some embodiments, the candidate bounding boxes to be attacked are screened out through intersection over union filtering and classification score filtering, including:

[0019] calculating the intersection over union between each predicted candidate bounding box and the real bounding box , selecting the top preset number of first type candidate bounding boxes with the highest intersection over union;

[0020] ranking according to the classification score of each predicted candidate bounding box, and selecting the top preset number of second type candidate bounding boxes with the highest classification score;

[0021] selecting a preset number of candidate bounding boxes from the combination of the first type candidate bounding boxes and the second type candidate bounding boxes as the candidate bounding boxes to be attacked.

[0022] In some embodiments, the calculation formula of the attention flip loss is:

[0023] ;

[0024] wherein, denotes the heat map of the i-th round of iteration of the adversarial sample, denotes the inverse of the clean heat map; SL1 denotes a smooth L1 function, and the calculation formula is as follows:

[0025] .

[0026] In some embodiments, the calculation formula of the bounding box translation loss is:

[0027] ;

[0028] wherein, denotes the size of the predicted candidate bounding box, denotes the size of the optimization target box, denotes the center point coordinates of the predicted candidate bounding box, denotes the center point coordinates of the real bounding box, denotes the predicted candidate bounding box, denotes the real bounding box in the clean sample.

[0029] In some embodiments, the background score maximization loss has a calculation formula as follows:

[0030]

[0031] wherein, denotes the probability of belonging to the background label, denotes the probability of belonging to other labels;

[0032] The calculation formula of the imperceptibility constraint loss is:

[0033] ;

[0034] wherein, denotes the pixel value of the adversarial sample, denotes the pixel value of the clean sample;

[0035] The calculation formula of the overall attack objective function is:

[0036] ;

[0037] wherein, is a parameter for limiting the imperceptibility of the perturbation.

[0038] In some embodiments, the present application also provides an attention reversing adversarial attack device for remote sensing target detection, comprising a processor, a memory and a computer program or instructions stored on the memory, the processor being configured to execute the computer program / instructions, and the device realizing the steps of the above method when the computer program or instructions are executed.

[0039] In some embodiments, the present application also provides a computer readable storage medium having a computer program or instructions stored thereon, the computer program or instructions being executed by a processor to realize the steps of the above method.

[0040] In some embodiments, the present application is a computer program product, comprising a computer program or instructions, the computer program or instructions being executed by a processor to realize the steps of the above method.

[0041] The beneficial effects of the present application are:

[0042] The attention reverse adversarial attack method and device for remote sensing target detection provided by the application create a derivable tensor as an adversarial sample, and the initial value is the pixel value of a clean sample; the feature maps of each layer in the target detector are processed by dimension reduction, linear activation and normalization to generate a clean heat map and an adversarial heat map, and an attention flip loss is constructed based on the reverse map of the clean heat map and the deviation of the adversarial heat map; the intersection over union filtering and classification score filtering are used to screen the candidate boxes to be attacked, and an optimized target box is set, and a bounding box translation loss is constructed by combining the intersection over union of the predicted candidate box and the real bounding box; a background score maximization loss is established by maximizing the classification probability of the background label; an imperceptibility constraint loss is established based on the deviation of the pixel value between the adversarial sample and the clean sample; and the above losses are combined to construct a total attack objective function to update the adversarial sample, and finally the target adversarial sample is obtained. The method can effectively generate an adversarial sample with high attackability and concealability, improve the accuracy and effect of the adversarial attack, and enhance the robustness of the model in a complex environment.

[0043] Additional advantages, objects, and features of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0044] It will be understood by those skilled in the art that the objects and advantages of the present application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0046] Figure 1 The flowchart of the attention reverse adversarial attack method for remote sensing target detection according to an embodiment of the application.

[0047] Figure 2 The overall framework diagram of the attention reverse adversarial attack method for remote sensing target detection according to an embodiment of the application.

[0048] Figure 3 The comparison diagram of the multi-scale reverse map, the clean heat map and the adversarial heat map in an embodiment of the application. DETAILED DESCRIPTION

[0049] For the purposes of the present invention, the technical solutions and advantages thereof will be more clearly apparent from the following detailed description of embodiments and the accompanying drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0050] Here, it should also be noted that, in order to avoid obscuring the present invention with unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the drawings, while other details not closely related to the present invention are omitted.

[0051] It should be emphasized that the term "comprises / comprising" when used in this text refers to the presence of stated features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0052] Remote sensing image target detection is a key research direction in computer vision, widely applied in military reconnaissance, disaster monitoring and urban planning, etc. Its core is to automatically identify and locate targets such as buildings, vehicles and roads from high-resolution images, providing key information for decision-making. However, the application of deep learning models in this field exposes their vulnerability, easily attacked by adversarial samples. Attackers add tiny perturbations to images, causing the model to output errors such as target misclassification, missed detection or false alarm, threatening system security and reliability. Adversarial attack research has become a focus, by imposing tiny perturbations on input data, causing target detection models to make incorrect predictions. This research direction not only evaluates the robustness of models, but also reveals potential vulnerabilities of models in complex environments, providing a basis for improving model performance and laying a theoretical foundation for developing safer systems. In recent years, many adversarial attack methods have emerged, mainly divided into adversarial patch attacks and pixel-level attacks.

[0053] Adversarial patch attacks mislead detection models by adding significant disturbance areas around or on the target, but these features are easily identified by defense algorithms, limiting their practical application. Pixel-level attacks generate adversarial perturbations by modifying each pixel value, minimizing visual interference, making it difficult for models to correctly detect targets. Its advantage lies in limited attack amplitude and wide area coverage, strong concealment. However, with the increase of image resolution and model complexity, pixel-level attacks require a large amount of computing resources and time, and the complexity of attack strategies increases.

[0054] In the prior art, for the generation of adversarial samples of a remote sensing image target detection model, input images are mainly optimized by constructing a loss function to make the model make mistakes. Technically, it is divided into two parts of attack object selection and target function design. In the attack object selection, some methods select image sub-blocks to mislead the model by optimizing the pixel values of the sub-blocks, and other methods design a loss function to offset the bounding boxes predicted by the detector to affect target recognition and positioning. In the target function design, adversarial loss functions are designed for the two core tasks of bounding box positioning and target classification. For bounding box positioning, the model is made to recognize errors by maximizing the position deviation; for target classification, the target is misclassified by maximizing the classification loss. However, these methods have shortcomings, resulting in low attack precision and weak performance. The main problems are: (1) the existing adversarial attack methods usually only interfere at the output end of the target detection model, and lack of disturbance to the key links of the decision-making process of the detector, such as the feature extraction and feature mining stage. This makes the training of adversarial samples only rely on the output part of the information, and cannot go deep into the core decision-making process of the model. Therefore, the precision and effect of the attack are relatively limited, which affects the overall performance of the attack in the multi-stage decision-making process of the detector; (2) the existing adversarial sample generation methods usually lack effective feature guidance, and do not fully utilize the internal features of the target detection model in the training process, resulting in that the generated adversarial samples are difficult to focus on the key areas affecting the model decision, the attack disturbance is uneven, and the judgment of the detection model cannot be significantly affected, and the attack potential of the adversarial samples cannot be fully played.

[0055] To address the challenges, the present application combines multi-scale features, attention mechanisms, attack object selection strategies and adaptive optimization methods to reduce computational overhead and enhance the adaptability of pixel-level attacks in complex environments. Specifically, to address the problems of weak adversarial attack transferability and poor generality in the adversarial sample generation process, the present application uses the heat map obtained by the explainability of the neural network to obtain the model decision basis as the attack object, and combines the traditional attack based on interference decision results to form a new adversarial attack method for the whole process of neural network decision of a deep learning target detection model.

[0056] Specifically, the present application provides an attention reversal adversarial attack method for remote sensing target detection, as shown in Figure 1 The method comprises the following steps S101-S106:

[0057] Step S101: create a derivable tensor as an adversarial sample, use the original remote sensing image in the preset data set as a clean sample, and use the pixel value of the clean sample as the initial value of the tensor.

[0058] Step S102: input the clean sample and the adversarial sample into the target detector, reduce the dimension, linearly activate and normalize the feature map of each layer in the target detector to obtain a clean heat map corresponding to the clean sample and an adversarial heat map corresponding to the adversarial sample; and construct an attention flip loss based on the inverted map of the clean heat map corresponding to each layer in the target detector and the deviation of the adversarial heat map.

[0059] Step S103: screen out a candidate box to be attacked through intersection over union filtering and classification score filtering; for each candidate box to be attacked, set a corresponding optimization target box in the adversarial sample, select a corresponding prediction candidate box for the adversarial sample through the target detector, combine the distance between the optimization target box and the prediction candidate box with the intersection over union between the prediction candidate box and the real bounding box in the clean sample to construct a bounding box translation loss.

[0060] Step S104: maximize the classification probability of the background label for the prediction result of the adversarial sample through the target detector to establish a background score maximization loss.

[0061] Step S105: establish an imperceptibility constraint loss based on the deviation of the pixel value between the adversarial sample and the clean sample.

[0062] Step S106: combine the attention flip loss, the bounding box translation loss, the background score maximization loss and the imperceptibility constraint loss to construct a total attack objective function, and update and optimize the adversarial sample to obtain a target adversarial sample.

[0063] The purpose of step S101 is to provide an adjustable initial sample for the subsequent optimization process, so that the sample with adversarial property can be gradually generated through the optimization algorithm. The clean sample can be obtained from a remote sensing image dataset to meet the data characteristics in this scenario. The derivable tensor can be created using a deep learning framework (such as PyTorch or TensorFlow).

[0064] The purpose of step S102 is to generate a heat map that can represent the model attention distribution by analyzing the feature map of each layer in the target detector, and to interfere with the model attention by constructing a loss function. The target detection model is the object to be attacked, which will focus on the position of the target object to be recognized when the clean sample is input. The clean sample heat map is generated by dimension reduction, linear activation and normalization of the feature map of each layer, which is used to mark the position of the model attention. Further, the inverted map of the clean sample heat map is constructed to mark the position that the target detector should pay more attention to in the subsequent iteration process of the adversarial sample.

[0065] In some embodiments, the feature maps of each layer in the target detector are subjected to dimensionality reduction, linear activation, and normalization to obtain clean heatmaps corresponding to clean samples. The calculation formula is as follows:

[0066] ;

[0067] Where i represents the current iteration round, , Indicates the first Width and height of each feature map Indicates the number of channels in the feature map; Used to filter negative values. It is a min-max normalization operation used to adjust the distribution of a heatmap to... Within the range.

[0068] The heatmap corresponding to the adversarial example after processing by the target detector is also calculated in the same manner.

[0069] In some embodiments, the inverted version of a clean heatmap is calculated as follows:

[0070] ;

[0071] in, Indicates when A clean heat map at that time. This indicates that the Z-score is normalized. Heatmap of clean sample The mean, express The standard deviation.

[0072] In some embodiments, the attention flipping loss is calculated as follows:

[0073] ;

[0074] in, This represents the heatmap of the adversarial sample in the i-th iteration. This represents the inverted version of a clean heatmap; SL1 represents the smoothing L1 function, calculated as follows:

[0075] .

[0076] Step S103 filters out candidate boxes to be attacked through Intersection over Union (IoU) filtering and classification score filtering, and sets an optimized target box for each candidate box, constructing a bounding box translation loss. The purpose of this step is to ensure that the attack is focused on the candidate boxes most likely to affect the model's detection results, thereby improving the accuracy and efficiency of the attack.

[0077] In some embodiments, candidate boxes to be attacked are filtered out by intersection-union filtering and classification score filtering, including steps S1031~S1033:

[0078] Step S1031: Calculate each predicted candidate box With the true bounding box Based on the intersection-union ratio (IU / UU) between the candidate boxes, select the first set number of candidate boxes with the highest IU / UU ratio.

[0079] Step S1032: Sort the predicted candidate boxes according to their classification scores and select the top set number of second-class candidate boxes with the highest classification scores.

[0080] Step S1033: Select a set number of boxes from the combination of the first type of candidate boxes and the second type of candidate boxes as attack candidates. Specifically, this can be represented as: .

[0081] After obtaining the candidate bounding boxes to be attacked, set corresponding optimized target boxes for them. The optimized target boxes should not touch or overlap with the real bounding boxes. In principle, they should be as far away from the real bounding boxes as possible.

[0082] In some embodiments, the formula for calculating the bounding box translation loss is:

[0083] ;

[0084] in, Indicates the size of the predicted candidate box. This indicates that the size of the target bounding box should be optimized. This represents the coordinates of the center point of the predicted candidate box. This represents the coordinates of the center point of the actual bounding box. Indicates the predicted candidate box. This represents the true bounding box in a clean sample.

[0085] Step S104 realizes the missed detection of the target by optimizing the adversarial sample to make the target detector misclassify the target as a background category. This step aims to enhance the effect of adversarial attack, so that the model cannot correctly identify the target when facing the adversarial sample. In the target detection task, the model not only needs to accurately locate the target, but also needs to correctly classify the target. By maximizing the classification probability of the background label, the model can be more inclined to misclassify the target as the background, thereby realizing the missed detection of the target. Specifically, by optimizing the adversarial sample, the model maximizes the classification score of the background label when predicting, thereby reducing the classification score of the target label to achieve the effect of missed detection. The background score in the classification score is taken as the optimization object by taking the negative value, so that the category of all prediction candidate boxes is determined as the background, and the missed detection of the target is realized. If the detector does not predict the background category score in the classification score, all foreground scores are taken as the optimization object, so that the foreground score is reduced to realize the missed detection.

[0086] In some embodiments, the background score maximization loss is calculated as follows:

[0087]

[0088] wherein, represents the probability of belonging to the background label, represents the probability of belonging to other labels.

[0089] In step S105, the purpose of the imperceptibility constraint loss is to ensure that the generated adversarial sample is as similar as possible to the clean sample in vision, thereby reducing the risk of detection of the adversarial sample. This step aims to improve the concealment of the adversarial sample, so that it can interfere with the model without being detected. In the adversarial attack, the adversarial sample needs to maintain the model's misjudgment while minimizing the visual difference between the clean sample. By introducing the imperceptibility constraint loss, the pixel value difference between the adversarial sample and the clean sample can be limited, ensuring that the adversarial sample is consistent with the clean sample in human vision. This not only improves the concealment of the adversarial sample, but also increases the success rate of the attack.

[0090] The calculation formula of the imperceptibility constraint loss is as follows:

[0091] ;

[0092] wherein, represents the pixel value of the adversarial sample, represents the pixel value of the clean sample.

[0093] The calculation formula of the overall attack target function is as follows:

[0094] ;

[0095] wherein, a parameter for limiting the imperceptibility of the disturbance.

[0096] Step S106 constructs a total attack target function by weighted sum of the above four loss functions. The function integrates the disturbance of model attention, the damage to the bounding box, the misleading of the classification result and the guarantee of the concealment of the adversarial sample. By optimizing the target function, an adversarial sample with high attackability and concealment can be generated.

[0097] In some embodiments, the present application also provides an attention reversal adversarial attack device for remote sensing target detection, comprising a processor, a memory and computer programs / instructions stored on the memory, the processor being configured to execute the computer programs / instructions, and the device realizing the steps of the above method when the computer programs / instructions are executed.

[0098] In some embodiments, the present application also provides a computer readable storage medium having computer programs or instructions stored thereon, the computer programs or instructions being executed by a processor to realize the steps of the above method.

[0099] In some embodiments, the present application is a computer program product, comprising computer programs or instructions, which are executed by a processor to realize the steps of the above method.

[0100] The present application will be described below in conjunction with a specific embodiment:

[0101] In view of the demand for high-resolution remote sensing image target credible intelligent perception, in view of the problems of weak adversarial attack transferability and poor adversarial sample generality in the process of generating adversarial samples, the present application is based on space-based high-resolution visible light remote sensing images, combined with deep learning theory, aiming at space-based remote sensing image target detection network, according to the application demand of adversarial attack, according to the technical approach of attack object selection, attack target function design and adversarial sample iterative generation, an attention reversal adversarial attack method for remote sensing target detection is designed.

[0102] The embodiment proposes an attention reversal attack method, which is aimed at several types of rotating target detectors including single-stage, multi-stage, anchor box and no anchor box, and constructs an adversarial attack algorithm from two aspects of feature-based attack and candidate box-based attack. The network integrates multi-level features, attention mechanism and explainability, and designs algorithm modules including attack object selection, attack target function design and adversarial sample iterative generation. A dynamic attention reversal module is constructed based on the explainability mechanism of the neural network, the model attention area is indicated by a heat map, and the feature-based attack is realized. A candidate box filtering module is designed to select the attack object based on the association between the candidate box and the true value box, and the candidate box-based attack is realized. An attack target function is designed based on the above modules, which includes attention reversal loss, boundary box drift loss, background score maximization loss and imperceptibility constraint.

[0103] For the attack object selection, attack target function design and adversarial sample iterative generation process, the application designs a general attack object selection scheme for the model, extracts multi-level feature maps and predicted candidate boxes from the remote sensing image target detection network, constructs a dynamic attention reversal module and a candidate box filtering module respectively, selects the attack object, and correspondingly proposes an attack target function. The input clean sample pixel value is taken as the neural network training object, and the adversarial sample generation is realized through gradient descent. The method framework is as shown in Figure 2 .

[0104] The basic steps of the embodiment are as follows:

[0105] Step one: constructing a dynamic attention reversal module. The module extracts multi-level features output by the backbone network of the target detector, converts the feature map into a heat map, and converts the heat map into a reversal map.

[0106] Step two: constructing a candidate box filtering module. The module extracts a large number of candidate boxes predicted by the target detector, filters and classifies the candidate boxes through the intersection over union (IoU) to select the candidate boxes to be attacked.

[0107] Step three: designing an attack target function, constructing an attention reversal loss, a boundary box drift loss, a background score maximization loss and an imperceptibility constraint. Training and testing are performed to complete adversarial sample generation and model robustness evaluation.

[0108] The following will be explained in detail:

[0109] In the target detection task, it is assumed that there is a target detector which accepts an input image and outputs a prediction , where is the boundary box of the th target, indicating the spatial position of the target, is the score of the category to which the target belongs. In this framework, the goal of adversarial attack is to generate an adversarial image such that the prediction of the target detector on the input image is wrong, i.e. the output predicted label is different from the original label , i.e. . The optimization objective can be formulated as minimizing the difference between the input image and the adversarial image while ensuring that the target detector makes a wrong prediction on the adversarial image, which is as follows:

[0110] ;

[0111] Target detection generally consists of two steps: the first step is to extract separable target features from the input image, and the second step is to locate the potential target region and identify it. Correspondingly, the adversarial attack is also divided into two parts: feature-based attack and proposal-based attack, and two corresponding modules are proposed, namely dynamic attention reverse module and candidate box screening module, which together with a series of loss functions constitute the adversarial attack framework.

[0112] Step one: build a dynamic attention reverse module. This module extracts the multi-level features output by the backbone network of the target detector, converts the feature map into a heat map, and converts the heat map into a reverse map.

[0113] The features extracted from the backbone network of the target detector contain rich information of the input image, in which the low-level features describe the details, and the high-level features focus on semantics. In the training iteration process, multi-scale feature maps are extracted as attack targets, and through dimension reduction, activation function and normalization operation, a heat map containing multi-scale target information is obtained. In each iteration , the calculation formula of the heat map

[0114]

[0115] wherein, represents the width and height of the th feature map, represents the channel number of the feature map. is used to filter negative values, and is the min-max normalization operation, which is used to adjust the distribution of the heat map to the range.

[0116] The highlighted areas in the heatmap represent the model's regions of interest in the input image, which form the basis for further regression and classification. To reverse the model's regions of interest, an inverted graph of the clean input samples is introduced as an explicit constraint, and an attention reversal attack is implemented by designing an additional loss function. The formula for calculating the inverted graph is:

[0117]

[0118] in, Indicates when The heatmap of clean samples at this time, where the input image is a clean sample that has not undergone any perturbation. This represents Z-score normalization, where, Heatmap of clean sample The mean, express The standard deviation of the value. By taking its negative value, the region of interest can be reversed, making low values ​​that the model is not interested in become high values ​​in the heatmap, and vice versa. Finally, a min-max normalization operation is added to obtain the inverted map. ,like Figure 3 As shown in the figure Represents a multi-scale inverted graph. Representing a clean heatmap and This represents the heatmap of the confrontation during the training process. This represents a feature map derived from the backbone network.

[0119] Step 2: Construct a candidate box filtering module. This module extracts a large number of candidate boxes predicted by the object detector and filters them out to be attacked by using intersection-over-union (IoU) filtering and classification score filtering.

[0120] Before the object detector outputs its prediction, it generates thousands of candidate boxes. Each potential target is surrounded by numerous overlapping candidate boxes, leading to redundant and enormous computational overhead during adversarial example training. Therefore, selecting the candidate boxes to be attacked is extremely important. To address this, a candidate box selection mechanism is designed for two filtering strategies: intersection-union (IoU) filtering and classification score filtering.

[0121] For each iteration On the one hand, by calculating each predicted candidate box With the true bounding box The intersection-over-union ratio (IoU) between the two is used to select the first. There are several candidate bounding boxes. The intersection-union ratio (IU) is a metric that measures the degree of overlap between two bounding boxes. Two boxes can be viewed as sets of two pixels, and their IU is equal to the area of ​​the overlapping portion of the two boxes divided by the combined area. On the other hand, candidates are ranked according to their classification scores and the corresponding target category scores, and the top candidates are selected. a selected candidate box. The specific process can be represented as, .

[0122] Step three: design the attack objective function, construct attention reversal loss, bounding box shift loss, background score maximization loss and imperceptibility constraint. Train and test, complete the generation of adversarial samples and model robustness evaluation.

[0123] In order to realize the adversarial attack on different target detectors, the objective function is designed to contain multiple loss terms. Each loss term corresponds to an attack strategy, which aims to destroy the performance of the model. Specifically, the following loss functions are used to achieve different attack goals:

[0124] 1. Attention reversal loss: get the reverse map from clean samples , and get the heat map from adversarial samples . In order to force the detector to focus on the opposite area, we use the smooth L1 ( ) function to reduce the difference between the clean sample reverse map and the adversarial iterative heat map :

[0125]

[0126]

[0127] 2. Bounding box shift loss: in order to destroy the bounding box predicted by the model, an intuitive idea is to set an optimization target box for each selected candidate box , and then make the selected candidate box and the optimization target box closer:

[0128]

[0129]

[0130]

[0131] For the center point , a successfully attacked candidate box should be far away from the true value. Combined with the IoU distance between the selected candidate box and the real bounding box, the bounding box translation loss can be represented as

[0132]

[0133] Background score maximization loss: In order to achieve the vanishing attack, even if the target detector misclassifies the target as a background category, a background score maximization loss is introduced. By maximizing the classification probability of the background label, the model is forced to output the background label. Specifically, the negative value of the background score in the classification score is taken as the optimization object, so that the category of all predicted bounding boxes is determined as background, and the target is missed. If the detector does not predict the background class score in the classification score, all foreground scores are taken as the optimization object, so that the foreground score is reduced, and the target is missed. The loss function formula is:

[0134]

[0135] wherein, is the probability of the background label, is the probability of other labels (foreground labels).

[0136] Unperceptibility constraint: In order to limit the amplitude of the perturbation as much as possible, so that the adversarial sample is consistent with the input clean sample under human vision as much as possible, the L2 norm constraint is adopted:

[0137]

[0138] In summary, the overall attack objective function is as follows:

[0139]

[0140] wherein, is used to limit the unperceptibility of the perturbation, and is used to balance the relationship between the unperceptibility of the perturbation and other loss terms. By optimizing the objective function, an effective adversarial sample can be generated to destroy the performance of the target detector and achieve the purpose of attack.

[0141] The advantage of the embodiment is that it comprehensively combines deep learning and neural network explainability technology, takes high-resolution remote sensing images as input, can accurately extract significant features in the decision-making process of the target detection model, and realizes the transfer of model attention and interference of the decision-making process through the dynamic attention reversal module. Through the extraction of multi-level feature maps, the generation of reverse maps, and the design of corresponding attention reversal loss, the technology can effectively constrain the adversarial sample heat map to approach the reverse map, thereby enhancing the robustness of the model to adversarial attacks and avoiding the failure of the model when facing interference.

[0142] When the target detection model output is interfered, the embodiment designs a candidate box screening module, accurately screens the to-be-attacked box from a large number of predicted boxes through the intersection over union and classification score filtering principle, which makes the attack more targeted and efficient. In addition, the technology further improves the accuracy of the attack by setting the optimized target box after deformation and designing the bounding box drift loss, ensures that only the target box is attacked, and reduces the damage to irrelevant areas. The addition of imperceptibility constraints ensures that the generated adversarial samples remain consistent with clean samples in vision, increases the concealment of adversarial samples, and thus reduces the risk of being detected.

[0143] The embodiment shows good adaptability on various detector architectures, and can seamlessly support single-stage and two-stage detectors, including traditional anchor box models and new anchor-free models. Regardless of the structure of the detector, the modules and loss functions of the method can be universal, ensuring that the generated adversarial samples can effectively attack the model and significantly improve the attack performance of the adversarial samples.

[0144] The embodiment aims to introduce interference into the decision-making process of the target detection network based on neural network explainability technology, and to interfere with the decision-making results based on the bounding box screening strategy and the optimization target function design. The adversarial sample generation method improves the attack and concealment of the generated adversarial samples through various optimization mechanisms and module designs, and enhances the adaptability of the adversarial attack method to various target detectors in complex environments.

[0145] Corresponding to the above method, the application also provides a device / system, which comprises a computer device including a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory, so that the device / system implements the steps of the above method.

[0146] The embodiment of the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above edge computing server deployment method. The computer readable storage medium can be a tangible storage medium, such as random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the technical field.

[0147] In summary, the attention reversal adversarial attack method and device for remote sensing target detection of the present application takes high-resolution remote sensing images as input, designs an adversarial sample generation method for a remote sensing target detection model based on deep learning technology and neural network interpretability technology, inserts a dynamic attention reversal module after a deep backbone network for input clean samples and iteratively trained perturbed samples, extracts significant features affecting model decision-making through multi-level feature maps, forms a heat map, takes the inverse of the heat map of the clean sample as a reversal map, proposes a corresponding attention reversal loss, and constrains the heat map of the adversarial sample to approach the reversal map, thereby realizing interference with the decision-making process and shifting the model attention.

[0148] Those of ordinary skill in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination thereof. The decision to implement in hardware or software depends on the particular application and design constraints imposed on the technological solution. Those of ordinary skill can use different methods to implement the described functions for each particular application, but such implementation should not be considered beyond the scope of the present application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link.

[0149] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0150] In this disclosure, features described and / or illustrated with respect to one implementation can be used in the same manner or in an analogous manner in one or more other implementations, and / or in combination with or in place of features of other implementations.

[0151] The above descriptions are only the preferred embodiments of the present application, not intended to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall fall within the scope of the present application.

Claims

1. A method for attention inversion adversarial attacks targeting remote sensing target detection, characterized in that, The method includes the following steps: A differentiable tensor is created as an adversarial example, with the original remote sensing images in the preset dataset as clean samples, and the pixel values ​​of the clean samples as the initial values ​​of the tensor. The clean samples and the adversarial samples are input into the target detector. The feature maps of each layer in the target detector are reduced in dimension, linearly activated, and normalized to obtain the clean heatmap corresponding to the clean samples and the adversarial heatmap corresponding to the adversarial samples. An attention flip loss is constructed based on the inverted map of the clean heatmap corresponding to each layer in the target detector and the bias of the adversarial heatmap. By filtering with intersection-union ratio (IU) and classification score, candidate boxes to be attacked are selected. For each candidate box to be attacked, a corresponding optimized target box is set in the adversarial sample. The adversarial sample is selected by the target detector to predict the corresponding candidate box. The distance between the optimized target box and the predicted candidate box is combined with the IU of the predicted candidate box and the true bounding box in the clean sample to construct the bounding box translation loss. For the adversarial sample predicted by the target detector, the background score maximization loss is established by maximizing the classification probability of the background label. An imperceptible constraint loss is established based on the deviation of pixel values ​​between the adversarial sample and the clean sample; The overall attack objective function is constructed by combining the attention flipping loss, the bounding box translation loss, the background score maximization loss, and the imperceptibility constraint loss to update and optimize the adversarial sample, so as to obtain the target adversarial sample.

2. The attention inversion adversarial attack method for remote sensing target detection according to claim 1, characterized in that, The clean heatmap corresponding to the clean sample is obtained by performing dimensionality reduction, linear activation, and normalization on the feature map of each layer in the target detector. The calculation formula is as follows: ； Where i represents the current iteration round, , Indicates the first Width and height of each feature map Indicates the number of channels in the feature map; Used to filter negative values. It is a min-max normalization operation used to adjust the distribution of a heatmap to... Within the range.

3. The attention inversion adversarial attack method for remote sensing target detection according to claim 2, characterized in that, The inverted version of the clean heatmap is calculated as follows: ； in, Indicates when The clean heat map at that time, This indicates that the Z-score is normalized. Heatmap of clean sample The mean, express The standard deviation.

4. The attention inversion adversarial attack method for remote sensing target detection according to claim 1, characterized in that, By using intersection-union filtering and classification score filtering, candidate bounding boxes to be attacked are selected, including: Calculate the intersection-union ratio (CUP) between each predicted candidate box and the ground bounding box, and select the first set number of candidate boxes with the highest CUP. Sort each predicted candidate box according to its classification score, and select the top set number of second-class candidate boxes with the highest classification scores; A set number of candidate boxes are selected from the combination of the first type of candidate boxes and the second type of candidate boxes as the attack candidates.

5. The attention inversion adversarial attack method for remote sensing target detection according to claim 1, characterized in that, The formula for calculating the attention flipping loss is: ； in, This represents the heatmap of the adversarial sample in the i-th iteration. This represents the inverted version of the clean heatmap; SL1 represents the smoothing L1 function, calculated as follows: 。 6. The attention inversion adversarial attack method for remote sensing target detection according to claim 5, characterized in that, The formula for calculating the boundary box translation loss is: ； in, This indicates the size of the predicted candidate box. This indicates the size of the optimized target bounding box. This represents the coordinates of the center point of the predicted candidate box. This represents the coordinates of the center point of the actual bounding box. This represents the predicted candidate box. This represents the true bounding box in the clean sample.

7. The attention inversion adversarial attack method for remote sensing target detection according to claim 6, characterized in that, The background score maximization loss is calculated as follows: in, This indicates the probability of belonging to the background label. This indicates the probability of belonging to another label; The formula for calculating the loss due to the imperceptible constraint is as follows: ； in, This represents the pixel value of the adversarial sample. This represents the pixel value of the clean sample; The formula for calculating the overall attack objective function is: ； in, This is a parameter used to limit the imperceptibility of disturbances.

8. An attention inversion adversarial attack device for remote sensing target detection, comprising a processor, a memory, and a computer program or instructions stored in the memory, characterized in that, The processor is configured to execute the computer program or instructions, and when the computer program or instructions are executed, the device implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images