A defense method against remote sensing detection adversarial sample attack

Abstract

The application discloses a defense method against remote sensing detection adversarial sample attack, and belongs to the technical field of calculation, estimation or counting. First, a first-order gradient graph of a detection image is generated, the gradient graph is separated in RGB channels, the separated RGB channels are subjected to mutual elimination operation to eliminate the background, the position of an adversarial sample block is highlighted through saliency enhancement operation, and the adversarial sample block is positioned. The positioned adversarial sample block is eliminated, the eliminated area is completed by using an image completion technology to restore the image, and the Random Erasing algorithm is adopted to compensate for the performance decline caused by the image completion. The application first carries out defense against adversarial sample attack under remote sensing detection, adopts adversarial sample positioning, regional image completion and data enhancement technology to effectively defend against the adversarial sample. The framework can effectively eliminate the influence of the adversarial sample on the detector under remote sensing detection before target detection and identification and in training.

Description

Technical Field

[0001] This invention relates to remote sensing detection security technology, and more particularly to a method for improving the robustness of a neural network model against adversarial sample fraud attacks on remote sensing images. Specifically, it discloses a defense method against adversarial sample attacks on remote sensing detection, belonging to the technical field of calculation, estimation, or counting. Background Technology

[0002] With the development of deep learning technology, remote sensing detection has evolved from traditional methods to deep learning-based methods. However, deep neural networks are vulnerable to adversarial attacks, meaning they are susceptible to such attacks. The development of adversarial examples allows them to launch attacks in the physical domain, posing a significant threat to remote sensing detection. Adversarial attacks manifest as adversarial blocks, which are attached to or near the target to evade detection and can even mislead detection results. Adversaries can use this technology to render deep learning-based remote sensing detection ineffective, posing a significant threat to target detection technologies such as battlefield target detection, urban vehicle tracking, and aircraft detection. This results in substantial economic losses for civilian sectors and poses a major threat to national security.

[0003] Currently, there is no effective defense against adversarial attacks in the physical domain, but defense against such attacks is crucial. In the field of remote sensing detection, adversarial attacks are widely used and easy to execute. Therefore, this invention aims to propose a defense method against adversarial attacks in remote sensing detection. Taking advantage of the unique characteristics of remote sensing in detecting ground targets, the method locates the adversarial sample using an adversarial sample localization method. After removing the adversarial sample, image completion technology is used to complete the removed area. To address the issue of weak feature occlusion in the completed image, the Random Erasing algorithm is used to improve the model's robustness under weak feature occlusion. This invention can ensure that remote sensing detection maintains high detection accuracy and good robustness under adversarial attacks. Summary of the Invention

[0004] The purpose of this invention is to address the security issues mentioned above by providing a defense method against adversarial example attacks on remote sensing detection. Specifically, for adversarial example attacks existing in the physical domain, this method utilizes adversarial example localization methods to locate adversarial examples, and employs image completion techniques to complete the regions where adversarial examples have been removed. Furthermore, the RandomErasing algorithm is used to improve the model's robustness under weak feature occlusion. This restores the detection performance of the deep learning-based detector model before it was attacked by adversarial examples.

[0005] To achieve the above-mentioned objectives, the present invention employs the following technical solution:

[0006] A defense method against adversarial attacks by remote sensing detection includes the following five steps.

[0007] Step 1: Select a suitable dataset according to the target type to be detected, and use the Random Erasing algorithm to augment the data in the dataset.

[0008] Step 2: Train the constructed dataset using a detector model based on deep learning technology. The trained model can accurately identify the detection target.

[0009] Step 3: When performing target detection, the acquired optical remote sensing image is used as the model input. Adversarial examples are located and eliminated using the adversarial example localization method.

[0010] Step 4: Use image completion technology to complete the removed areas. The completed image is used as the final input of the model for prediction.

[0011] Step 5: Use the detector model to detect the final completed image and determine the location and category information of the detected targets contained in the image.

[0012] As a defensive method against adversarial attacks in remote sensing detection, step 1 uses the Random Erasing algorithm to augment the data in the dataset. The method of the Random Erasing algorithm in data augmentation can be summarized as follows.

[0013] The bounding box of the detected target is defined as obj. i Let i be the i-th detected target in the image, with coordinates (x, y). l y l x r y r The numbers () represent the top left and bottom right corners of the bounding box, respectively. Random erasing is performed within the bounding box of the detected target. The rectangle's width and height ratio is set to [ω1, w2]. During random erasing, a ratio value w is randomly selected within this range. c The width W and height H of the rectangular block in Random Erasing can be expressed as follows:

[0014]

[0015] Through (x l y l x ry r Within the area enclosed by the coordinates, reselect the top-left corner (x, y) of the rectangular block used for Random Erasing. c y c If x c +W≤x r -x l And y c +H≤y r -y l Then the coordinates (x, y) of the final Random Erasing rectangle can be obtained. c y c x c +W, y c +H).

[0016] The Random Erasing algorithm will locate the coordinate (x) c y c x c +W, y c The rectangle within (+H) is filled with all pixel values ​​of 1. For a training sample, if there are one or more detection targets on its data samples, a random erasing operation is performed within the bounding box of each detection target.

[0017] When training the model, each batch of training data is read in, the Random Erasing algorithm is enabled with a 50% probability. If the Random Erasing algorithm is enabled, the data augmentation is performed according to the above requirements. If the Random Erasing algorithm is not enabled, the read training data remains unchanged.

[0018] As a defensive method against adversarial attacks on remote sensing detection, step 2 requires training the model using the constructed dataset. The trained model needs to ensure that it has certain detection performance when performing target detection and that it has certain robustness when detecting occluded targets.

[0019] As a defense method against adversarial sample attacks detected by remote sensing, the adversarial sample localization method described in step 3 uses image processing technology to locate adversarial samples. This adversarial sample localization method includes steps 3-1 to 3-4.

[0020] Step 3-1: Obtain the frames to be detected, i.e., still images. First, calculate the first-order gradient of the image:

[0021]

[0022] Where a and b represent the horizontal and vertical directions of the image plane, the gradient map is calculated separately on the RGB three channels, and the gradient map is normalized to [0, 1], as shown below:

[0023]

[0024] Where g(x) represents the gradient map.

[0025] Step 3-2: Perform channel cancellation operation on the obtained RGB three-channel gradient map to enhance the salience of adversarial examples and reduce the salience of the background. Define the RGB channels of the gradient map g(x) as g(x). R , g(x) G With g(x) B Then the image g(x) after mutual cancellation is obtained. e for:

[0026]

[0027] in Representing the absolute value of the elements in the matrix, after mutual cancellation, the adversarial example block can be significantly separated from the background image. However, there may be other background noise in the environment, so it is necessary to enhance the saliency of the adversarial example block and suppress the background noise. The separation method is as follows:

[0028]

[0029] Where β is the normalized pixel offset, which is generally greater than 1, and α is the magnification factor, which is generally greater than 2.

[0030] Step 3-3: After saliency enhancement, the adversarial sample is clearly distinguishable from the background. Thresholding is performed on the image using an appropriate threshold to filter out the background and obtain the location of the adversarial sample within the image. Since there may be some very small high-frequency noise regions within the optical detection image, opening operations are needed to eliminate this noise. This involves first eroding the image and then dilating it, as shown in the following expression:

[0031] open(g(x))=dilate(erode(g(x),kernal1),kernal2)

[0032] kernel1 and kernel2 represent the size of the convolution kernel, erode(·) represents morphological erosion, and dilate(·) represents morphological dilation.

[0033] Steps 3-4: In the final localization step, the top-left and bottom-right image coordinates of the rectangular block formed by the connected regions are selected, and regions with fewer than the threshold number of connected pixels are removed. The remaining regions are those contaminated by adversarial sample blocks and also need to be removed. The location of the adversarial sample is found in the original input image and removed. Here, the removed regions are replaced with pixel values ​​all equal to 1.

[0034] As a defense method against adversarial sample attacks by remote sensing detection, step 4 requires image completion of the removed areas. The image completion algorithm includes steps 4-1 to 4-3.

[0035] Step 4-1: Utilize a coarse-to-fine image completion network. This network is first trained using ImageNet or other large datasets, and the trained network can effectively complete images.

[0036] Step 4-2: Use the image obtained in Step 3 after removing adversarial examples as the image completion target. Let the region to be completed (i.e., the adversarial example block) be the center region. Crop the image with n times the width and height of the region to be completed, creating a clipping block. Simultaneously, generate a rectangular block of the same size as the clipped region as a mask block. The area corresponding to the clipping region is filled with all pixels of 1, and the remaining areas are filled with all pixels of 0. Feed the clipping block and the mask block together into the image completion network to complete the missing parts of the clipping block.

[0037] Step 4-3: After completing the cut-out block, normalize the completed image to [0, 1]. Then restore it to the corresponding position in the image as the final image to be detected.

[0038] As a defensive method against adversarial attacks on remote sensing detection, step 5 performs robust detection on the detection image according to steps 5-1 to 5-3.

[0039] Step 5-1: Use the image to be detected obtained in step 4 as the input sample, fix the length and width of the image to be detected, and perform normalization.

[0040] Step 5-2: Send the processed input sample to be detected into a computer or other edge processing computer device, which contains a detector model that has been trained by the Random Erasing algorithm.

[0041] Step 5-3: Based on the adversarial example localization and completion method described in Steps 3 and 4, after the input sample is identified by the model, the detector can detect the location and category information of the target within the input sample.

[0042] The present invention, employing the above-mentioned technical solution, has the following beneficial effects: The present invention proposes a defense method against adversarial example attacks in remote sensing detection. Targeting the unique characteristics of remote sensing detection of ground targets, it locates the adversarial examples using an adversarial example localization method. After removing the adversarial examples, image completion technology is used to complete the removed areas. To address the issue of weak feature occlusion in the completed image, the Random Erasing algorithm is used to improve the model's robustness under weak feature occlusion. The present invention demonstrates excellent resistance to adversarial example attacks on remote sensing detection, can defend against different forms and types of adversarial example blocks, and can significantly improve the model's detection accuracy under adversarial example attacks. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the method for resisting adversarial sample blocks according to the present invention.

[0044] Figure 2 This is a schematic diagram of the adversarial sample block localization method of the present invention.

[0045] Figure 3 This is a schematic diagram of the image completion method of the present invention. Detailed Implementation

[0046] The technical solution of the invention will now be described in detail with reference to the accompanying drawings.

[0047] This invention aims to design a defense method against adversarial attacks on remote sensing detection by utilizing adversarial example localization technology, image completion technology, and data augmentation technology. This method can effectively resist adversarial attacks on remote sensing detection, thereby ensuring the robustness of the detector model against adversarial examples when performing remote sensing target detection.

[0048] The overall defense process is as follows: Figure 1 As shown, for the image to be detected, a first-order gradient map of the detection image is first generated. The gradient map is then separated into RGB channels. The separated RGB channels are used to eliminate the background through mutual cancellation. The location of the adversarial sample block is highlighted and located through saliency enhancement. The located adversarial sample block is then removed, and image completion technology is used to complete the removed region to restore the image. To compensate for the performance degradation caused by image completion, the Random Erasing algorithm is used.

[0049] like Figure 1 As shown, a defense method against adversarial sample attacks in remote sensing detection comprises three main parts:

[0050] The first part is data augmentation, the second part is adversarial example localization, and the third part is image completion. Overall, it can be divided into the following steps:

[0051] Step 1) Select a suitable dataset according to the target type to be detected, and use the Random Erasing algorithm to augment the data in the dataset.

[0052] Step 2) The constructed dataset is trained using a detector model based on deep learning technology. The trained model can accurately identify the detection target.

[0053] Step 3) When performing target detection, the acquired optical remote sensing image is used as the model input, and adversarial examples are located and removed using the adversarial example localization method.

[0054] Step 4) Use image completion technology to complete the removed areas. The completed image is used as the final input of the model for prediction.

[0055] Step 5) Use the detector model to detect the final completed image and determine the location and category information of the detected targets contained in the image.

[0056] The first part of a defense method against adversarial attacks on remote sensing detection is data augmentation, which serves as the training part. The training method is as follows:

[0057] The bounding box of the detected target is defined as obj. i Let i be the i-th detected target in the image, with coordinates (x, y). l y l x r y r The numbers () represent the top left and bottom right corners of the Bounding Box, respectively. Random Erasing is performed within the Bounding Box of the detected target. The rectangle's width and height ratio is set to [ω1, w2]. During Random Erasing, a ratio value w is randomly selected within this range. c The width W and height H of the rectangular block in Random Erasing can be expressed as follows:

[0058]

[0059] Through (x l y l x r y r Within the area enclosed by the coordinates, reselect the top-left corner (x, y) of the rectangular block used for Random Erasing. c y c If x c +W≤x r -x l And yc +H≤y r -y l Then the coordinates (x, y) of the final Random Erasing rectangle can be obtained. c y c x c +W, y c +H).

[0060] The Random Erasing algorithm will locate the coordinate (x) c y c x c +W, y c The rectangle within (+H) is filled with all pixel values ​​of 1. For a training sample, if there are one or more detection targets on its data samples, a random erasing operation is performed within the bounding box of each detection target.

[0061] During model training, for each batch of training data, the Random Erasing algorithm is activated with a 50% probability. If Random Erasing is activated, data augmentation is performed according to the above requirements; otherwise, the input training data remains unchanged. The model is then trained using the constructed dataset. The trained model must demonstrate certain detection performance in object detection and robustness in detecting occluded targets. At this point, model training is complete.

[0062] like Figure 2 As shown, the second part of a defense method against adversarial example attacks from remote sensing detection is the dimensional adversarial example localization part. This part is used in the image processing part before the model performs inference and consists of the following steps:

[0063] Step 1): Obtain the frames to be detected, i.e., still images. First, calculate the first-order gradient of the image:

[0064]

[0065] Where a and b represent the horizontal and vertical directions of the image plane, the gradient map is calculated separately on the RGB three channels. For subsequent processing, the gradient map is normalized to [0, 1], and the normalized representation is as follows:

[0066]

[0067] Where g(x) represents the gradient map.

[0068] Step 2): Perform channel cancellation operation on the obtained three-channel gradient map to enhance the salience of adversarial examples and reduce the salience of the background. Define the RGB channels of the gradient map g(x) as g(x). R , g(x) G With g(x) B Then the image g(x) after mutual cancellation is obtained. e for:

[0069]

[0070] in Representing the absolute value of the elements in the matrix, after mutual cancellation, the adversarial example block can be significantly separated from the background image. However, there may be other background noise in the environment, so it is necessary to enhance the saliency of the adversarial example block and suppress the background noise. The separation method is as follows:

[0071]

[0072] Where β is the normalized pixel offset, which is generally greater than 1, and α is the magnification factor, which is generally greater than 2.

[0073] Step 3): The saliency enhancement image makes the adversarial example significantly different from the background. Thresholding is performed on the image using an appropriate threshold to filter out the background and obtain the location of the adversarial example in the image. Since there may be some very small high-frequency noise regions within the optical detection image, opening operations are needed to eliminate this noise. This involves first eroding the image and then dilating it, as shown in the following expression:

[0074] open(g(x))=dilate(erode(g(x),kernal1),kernal2)

[0075] kernel1 and kernel2 represent the size of the convolution kernel, erode(·) represents morphological erosion, and dilate(·) represents morphological dilation.

[0076] Step 4): In the final localization step, select the top-left and bottom-right image coordinates of the rectangular block formed by the connected regions, and remove regions with fewer than the threshold number of connected pixels. The remaining regions are those contaminated by adversarial sample blocks and also need to be removed. Find the location of the adversarial sample in the original input image and remove it, replacing the removed regions with all pixel values ​​of 1.

[0077] like Figure 3As shown, the third part of a defense method against adversarial sample attacks from remote sensing detection is the image completion part. This part is used in the image processing part before the model performs inference. The implementation steps of the image completion part of this invention are as follows:

[0078] Step 1): Use a coarse-to-fine image completion network. This network is first trained on ImageNet or other large datasets. The trained network can effectively complete images.

[0079] Step 2): Use the image after removing adversarial examples as the image completion target. Center the image on the region to be completed (i.e., the adversarial example block). Crop the image to a size n times the width and height of the region to be completed, creating a clipping block. Simultaneously, generate a rectangular block of the same size as the clipped region as a mask block. Fill the area corresponding to the clipped region with all pixels of 1, and fill the remaining areas with all pixels of 0. Feed the clipping block and the mask block together into the image completion network to complete the missing parts of the clipped block.

[0080] Step 3): After completing the cut-out block, normalize the completed image to [0, 1]. Then restore it to the corresponding position in the image as the final image to be detected.

[0081] Table 1 shows the performance of the model detection method of the present invention in resisting adversarial sample attacks (model performance includes recall, precision and AP value).

[0082] This invention analyzes a dataset of tank images captured by remote sensing satellites and uses the YOLOv3 model to identify and detect tanks. Adversarial example blocks are directly attached to the detected targets to attack the detector; this attack is a vanishing attack, meaning the detector cannot detect the targets even if they are not detected. The detection confidence threshold is set from 0.01 to 0.9 to represent the model's defense capability against adversarial example attacks. As shown in Table 1, the algorithm of this patent enables the model to largely recover its original detection performance when attacked. The model's detection performance is very close to its original performance at different confidence thresholds. Therefore, this method enables the detector model to achieve higher detection accuracy in remote sensing detection.

[0083] Table 1

[0084]

[0085] In summary, this invention can prevent remote sensing detection failures caused by adversarial sample attacks, such as detection and identification of ground targets by UAVs and remote sensing satellites. This invention enables the target detector to still have a high detection capability under adversarial sample attacks and has strong robustness to different environments and adversarial samples.

Claims

1. A defense method against adversarial sample attacks from remote sensing detection, characterized in that... The process includes the following steps: Before remote sensing target detection and recognition, the image is first subjected to first-order gradient calculation and normalization to enhance the salience of adversarial examples in the image. The adversarial examples are separated from the image using a mutual elimination operation. After opening the image, image segmentation is performed to obtain the regions of adversarial examples in the image, thereby locating the positions of the adversarial examples in the image. The located adversarial example blocks are removed, and the removed regions are repaired using an image completion operation. Due to weak feature occlusion caused by image completion, the Random Erasing algorithm is used to augment the detector model. The image after image completion is used as the input of the detector trained with the Random Erasing data augmentation algorithm to perform robust detection on the remote sensing image to be detected.

2. The operational steps of the defense method against adversarial sample attacks by remote sensing detection according to claim 1, characterized in that, In adversarial example localization methods, the first step is to calculate the first-order gradient of the image: Where a and b represent the horizontal and vertical directions of the image plane; the gradient map is normalized to [0, 1]: Where g(x) represents the gradient map, which is normalized in the R, G, and B channels respectively; the adversarial examples are separated from the image by mutual cancellation operation using the gradient images of the three channels. The RGB channels of the gradient map g(x) are g(x) R , g(x) G With g(x) B Then the gradient map g(x) after significance enhancement e It can be represented as: The gradient image after mutual cancellation is enhanced for saliency and the background noise is reduced to improve the saliency of the adversarial sample block. Finally, image processing methods are used to locate the adversarial sample block.

3. The adversarial example localization method according to claim 2, characterized in that, The image processing method described above applies to g(x). e The specific method for locating adversarial sample blocks in the data is as follows: For g(x)... e Manual thresholding is performed to separate the background from the adversarial sample blocks by selecting an appropriate threshold. The binarized segmentation image first uses morphological erosion to eliminate high-frequency noise with extremely small areas, and then uses morphological dilation to make the adversarial sample block regions stand out as much as possible and connect them together. The image coordinates of the upper left and lower right corners of the rectangular blocks formed by the connected regions are calculated, and regions with less than the threshold of the number of connected pixels are removed. The final region obtained is the adversarial sample block.

4. The adversarial example localization method according to claim 2, characterized in that, The located adversarial sample block is filled with all pixel values ​​of 1. The image is cropped with the adversarial sample block as the center and its length and width are n times that of the adversarial sample block to form a cropping block. At the same time, a rectangular block of the same size as the cropping area is generated as a mask block. The area corresponding to the adversarial sample block is filled with all pixel values ​​of 1, and the remaining area is filled with all pixels of 0.

5. The operational steps of the defense method against adversarial attacks by remote sensing detection according to claim 1, characterized in that... The image completion algorithm takes a mask block and a clipping block as input to its network model, and outputs a normal image after completing the clipping block by filling in all pixels with a center value of 1.

6. The image completion algorithm according to claim 5, characterized in that, The image completion algorithm uses known background blocks to copy or borrow feature information, takes the cropped adversarial sample block as the foreground, and crops the region image with the adversarial sample block as the center and its length and width as n times to form a cropped block as the background block, and copies or borrows feature information from the background block.

7. The image completion algorithm according to claim 5, characterized in that, To address the issue of weak feature occlusion after image completion, the Random Erasing algorithm is used to augment the dataset. When training the detector model, the Random Erasing algorithm is enabled with a certain probability. The Random Erasing algorithm uses all pixel values ​​of 1 to randomly generate a rectangular region of a certain size to cover the detection target in order to augment the data.

Images