Attack resisting method and system for remote sensing target detection

Abstract

The invention discloses an anti-attack method and system for remote sensing target detection. The method comprises the following steps: acquiring a net remote sensing scene image; detecting a target object which is included in the remote sensing scene image and needs to resist an attack; constructing a confrontation sample image by using the target object; optimizing the constructed confrontation sample image to obtain an optimized confrontation sample image; the optimized confrontation sample image is used for confrontation attack on remote sensing target detection. According to the method, the problem that the attack performance of an existing texture-based antagonistic patch in a remote sensing scene is reduced can be solved.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, artificial intelligence and remote sensing image analysis, especially the application of deep learning in remote sensing target detection, and specifically relates to adversarial attack methods and systems for remote sensing target detection. Background Technology

[0002] With the rapid development of deep learning, object detection based on deep neural networks has shown remarkable performance in terms of accuracy and efficiency. However, many studies have shown that deep neural networks are susceptible to well-designed adversarial examples, where malicious perturbations or patches added to clean examples (i.e., original images without perturbations or patches) can easily fool object detection models.

[0003] Early research focused primarily on digital or physical attacks in close-range scenarios, such as facial recognition and autonomous driving. Adversarial attacks on aerial detection are more challenging, especially in the physical world, because remotely sensed targets are much smaller in distant scenarios. Adversarial examples mainly fall into two categories: perturbations that are difficult for humans to detect and image patches. The former typically manipulates every pixel in the entire image and is suitable for digital attacks in web applications. The latter is printable and can be attached to objects, making it the primary method for adversarial examples applied to real-world scenarios. Recently, researchers have conducted extensive research on physical adversarial attacks. These are often achieved by optimizing adversarial patches with special and complex textures, such as... Figure 1 As shown.

[0004] While these methods can fool detection models, adversarial attacks are only effective at close range (e.g., a few meters or tens of meters). Research on long-range physical attacks in remote sensing at distances of hundreds of meters is relatively limited, with most methods directly derived from general scene physical attack approaches. Because the image quality and color fidelity of texture-based adversarial patches in remote environments are easily affected by lighting conditions, atmospheric scattering, and refraction, these texture-based long-range scene adversarial examples become distorted, significantly reducing the performance of physical attacks. Tests showed that texture-based remote sensing YOLO attack patches achieved a 50% success rate at 50m, but completely lost their attack capability beyond 100m. Since the distance between the camera and the target is a fundamental characteristic of remote sensing scenes, low image quality and distortion are unavoidable. Therefore, even if an engineer designs a patch with excellent attack performance, it will lose its attack capability due to distortion in its physical implementation. Against this backdrop, developing an efficient physical adversarial attack method suitable for low-quality and distorted remote sensing images is a significant challenge. For deep neural network-based target detection models, texture information does provide important features, which may be the main reason why current research uses texture-based adversarial patches to mislead detectors by altering salient features. Nevertheless, in addition to texture details, relying on contour and edge information to locate objects is also very important for object detection.

[0005] Furthermore, in recent years, the transferability of adversarial patches has become a research hotspot, with researchers proposing various methods to improve the transferability between different models or images. However, the transferability between different modalities (i.e., visible light modal and infrared modal) has not received sufficient attention. Since visible light and infrared modal sensors have different imaging mechanisms, achieving cross-modal transfer attacks is challenging. Existing cross-modal attack work is achieved through joint training, which is fundamentally different from transferable attacks. Guided by joint information from visible light and infrared modalities, they optimize specific factors of patches (such as shape, location, and quantity). However, sometimes only data from one modality is available; in such cases, only transfer attack methods can be sought to attack other modalities. Moreover, during joint training of two modalities, because the color definitions for visible light and infrared modalities differ (visible light modal uses texture, infrared modal uses grayscale), it is impossible to simultaneously optimize color with other factors. Considering the significant impact of color on attack performance, the performance of a single modality will decline without color. In contrast, transfer attacks are based on single-modal optimization, allowing color to be incorporated into the overall synchronous optimization. Figure 2As shown, taking the transition from visible light mode to infrared mode as an example, researchers attempted to simultaneously optimize all four factors of the adversarial patch to improve cross-modal transferability while ensuring the effectiveness of the white-box mode. Considering that gradients can provide important guidance for attacks, the researchers tried to directly optimize color, shape, position, and quantity via gradient descent. However, differentiating all factors is a challenge. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention aims to provide an adversarial attack method and system for remote sensing target detection, so as to solve the problem of decreased attack performance of existing texture-based adversarial patches in remote sensing scenarios.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: Adversarial attack methods against remote sensing target detection include the following process: Acquire clean remote sensing scene images; Detect the target objects that need to be countered from attacks contained in the remotely sensed scene image; Construct adversarial sample images using the target object; The constructed adversarial sample image is optimized to obtain an optimized adversarial sample image; the optimized adversarial sample image is used for adversarial attacks on remote sensing target detection.

[0008] Preferably, the process of constructing an adversarial example image using the target object includes: setting an adversarial bar on top of the target object to construct an adversarial example image.

[0009] Preferred adversarial sample images The expression is as follows:

[0010] in, The single color indicates the contrast bar. It is a mask matrix The approximate matrix of the mask matrix It is a remotely sensed scene image that has been cleaned up. Binary masks with the same dimensions, Indicating the opposition clause , The value in Representing adversarial sample images The pixels at the corresponding positions in the middle belong to the anti-bar. It includes information on the location and thickness of the strip corresponding to the anti-countermeasure strip.

[0011] Preferred, approximate matrix Each value in the expression is evaluated as follows:

[0012] in, d ( x , y ) represents the pixel on the adversarial bar. The distance between the center line of the opposing strip; The function represents half the thickness of the anti-stripping strip. It is continuously differentiable. , Set to a set value, which cannot be greater than -100.

[0013] Preferably, when optimizing the constructed adversarial example image, the adversarial bar is optimized, including: By minimizing the target object confidence level To optimize color parameters Slope parameters And thickness parameters, where the thickness parameters are the thicknesses of the two sides of the anti-counter strip thickness divided by the center line, respectively; Iteratively update the center point and thickness of the adversarial strip. After each iteration, check the values ​​of the center point and thickness of the adversarial strip and trim the center point and thickness of the adversarial strip to a preset range. Specifically, a confrontation clause The optimization method is as follows:

[0014] in, For loss function, For the target object confidence level To train a well-functioning object detector, It is the set of detected target objects, target objects Use a tuple It means that, among them, and The coordinates represent the center of the bounding box, and w and h represent the width and height of the bounding box. and This represents the predicted class label and the prediction confidence score.

[0015] Preferably, the process of constructing adversarial sample images using the target object includes: constructing adversarial sample images using a set of fully factor-optimizable adversarial patches.

[0016] Preferred adversarial sample images The expression is as follows:

[0017] in, x To obtain clean remote sensing scene images, m i To counter the patch Each adversarial patch in the context of adversarial patching Includes information about the color, shape, location, and number of adversarial patches. , The location determines The shape and location of the patch For color variables.

[0018] Preferably, when optimizing the constructed adversarial sample image, the adversarial patch is... Optimize to find a suitable set of counter-patches Adding adversarial patches causes the target to... Disappeared, therefore, in response to the patch The optimized expression can be represented as:

[0019] in, To mitigate attack losses during the optimization process; It is a group of detected target objects ;Target It is a collection containing the location and category information of the detected object. , and Represents the coordinates of the center point of the object. and This represents the width and height of the object. Represents the category label of an object. The confidence score represents the test results. For adversarial sample images The test results .

[0020] Preferably, the process of optimizing the constructed adversarial example image includes: Use the center point A set of line segment lengths and a set of angles to indicate Adverse patches, Indicates connection to vertices and center point The angle between the line and the horizontal axis; using the following equation, , , These three sets of variables solve for the coordinates of all vertices:

[0021]

[0022] in, Represents the coordinates of the nth vertex, and each pair of adjacent vertices and and center point Forming a sector shape, the entire patch consists of N+1 sectors; for each point within the region, it will be placed in the set. The problem of taking 0 or 1 in the middle is transformed into the problem of whether the point is inside the sector. For a point Within the region, there always exists a straight line. and Intersect at Then middle The values ​​are mapped to the following relationship:

[0023] This mapping is approximated by a step function, which is expressed as follows:

[0024] Bundle Each point in Convert the value into a straight line and The problem of the ratio of lengths, in which, It is a straight line and At the intersection point, use gradient descent to update To optimize the shape of adversarial patches; after each update epoch, for each Execute splitting and merging: Specifically, in each round when the adversarial patch changes to a concave polygon, a splitting operation is performed to split the concave points, thereby ensuring that the adversarial patch is always a convex polygon. For polygons that overlap during the optimization process, a merging operation is performed to merge the overlapping polygons into a whole.

[0025] This invention also provides an adversarial attack system for remote sensing target detection, used to implement the aforementioned adversarial attack method for remote sensing target detection. The system includes: Image acquisition module: used to acquire net remote sensing scene images; Target object detection module: used to detect target objects in the remote sensing scene image that require countermeasures against attacks; Adversarial Sample Image Construction Module: Used to construct adversarial sample images using the target object; Image optimization module: used to optimize the constructed adversarial sample image to obtain an optimized adversarial sample image; the optimized adversarial sample image is used for adversarial attacks on remote sensing target detection.

[0026] The present invention has the following beneficial effects: The adversarial attack method provided by this invention effectively overcomes the inherent defect of existing technologies where attack performance drops sharply in remote sensing scenarios. Remote sensing images, due to their long shooting distance, generally suffer from blurred details, color distortion, and texture degradation, making traditional adversarial patches that heavily rely on complex textures almost ineffective beyond 100 meters. The advantage of this method lies in its departure from obsessively "entangled" with the detection model on distorted textures, instead shifting towards a more robust attack logic. By first accurately detecting specific target objects in the image and then constructing adversarial examples based on them, this process essentially guides the attack to focus on the essential features of the target—such as its contours and edge structures. This structural information is often preserved more completely in low-quality images than subtle textures, thus laying a solid foundation for generating more deceptive adversarial examples. Further optimization steps enable the final generated adversarial example image to simulate and resist various interferences caused by long-distance imaging. This ensures that when deployed in a real physical environment, it can still effectively mislead the target detection model, significantly improving the attack success rate at distances of 100 meters and even further. Attached Figure Description

[0027] Figure 1 The image shows a traditional texture-based adversarial patch (left) and the strip-based adversarial patch proposed in this invention (right), the latter exhibiting significantly higher adversarial attack performance in the physics domain.

[0028] Figure 2 This invention compares the cross-modal transfer attack results generated by three different methods: TOG, AS, and UAP. The method of this invention can simultaneously optimize the color, shape, position, and number of adversarial patches. The generated adversarial patches maintain good attack effectiveness in the visible modality and also allow for direct transfer attacks in the infrared modality.

[0029] Figure 3 This is a differentiable representation of the adversarial stripes in an embodiment of the invention. The adversarial stripes are represented by their center points. Slope parameters ,thickness and color It can be represented in a differentiable way.

[0030] Figure 4 The optimization process for the adversarial stripes in this embodiment of the invention mainly includes two steps in each iteration: the first step is to optimize the color parameters. Slope parameters and thickness parameters The second step is to update the center point of the adversarial strip. and thickness .

[0031] Figure 5 This is a representation of a fully factorial optimizable patch in an embodiment of the invention. The blue portion represents the differentiability of the patch; the yellow portion represents the patch splitting operation, which is performed when the adversarial patch is transformed into a concave shape; and the green portion represents the patch merging operation, which is performed when two opposing patches intersect.

[0032] Figure 6 This presents the detection results of adversarial samples generated under a white-box setting by various digital domain attack methods in embodiments of the present invention. TOG, APA, and PAAI have optimized texture-based adversarial patches, and the proposed method has optimized monochrome stripe-based adversarial patches.

[0033] Figure 7 This provides further examples of attacks on the digital domain in this embodiment of the invention.

[0034] Figure 8 This is an example of a physical domain attack in an embodiment of the present invention.

[0035] Figure 9 These are attack examples under different physical domain backgrounds in the embodiments of the present invention.

[0036] Figure 10 This is a visualization of the adversarial patches in an embodiment of the present invention. The upper part shows the attack success rate of different methods in the visible light mode and the attack success rate of the transition to the infrared mode; the lower part shows the optimization process of our method. Figure 11 In this embodiment of the invention, attacks on the physical world will occur at different distances and angles. The upper figure shows the attack results against the pedestrian detector, and the lower figure shows the attack results against the remote sensing detector. The blue curve represents the method proposed in this paper; the red curve represents TOG (Visible Mode Baseline Method); the cyan curve represents UAP (Infrared Mode Baseline Method); and the yellow curve represents the random adversarial patching method.

[0037] Figure 12 This is a visualization of the physical attacks in this embodiment of the invention. The top image shows the attack results against a pedestrian detector, and the bottom image shows the attack results against a remote sensing detector. The first three columns show the attack effects at different distances, and the last two columns show the attack effects at different angles. The lower left corner of each image displays the adversarial patch trained in the digital world. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] This invention relates to the fields of computer vision, artificial intelligence, and remote sensing image analysis, particularly the application of deep learning in remote sensing target detection. Specifically, this invention proposes a physical stripe-based adversarial attack method for target detection tasks in optical remote sensing images. This method optimizes the monochromatic stripes on the surface of a target object, making it impossible for the target detector to correctly identify and locate the target object, thereby improving the effectiveness of adversarial attacks. This method can be widely applied in scenarios such as remote sensing image analysis, military reconnaissance, UAV detection and evasion, and privacy protection.

[0040] To address the problems in existing technologies, this invention proposes a novel physical adversarial attack method that uses stripe-based patches to hide targets detected in remote sensing. The key idea is to mislead the detector's localization head prediction by attaching optimized monochrome stripes to the object. Once the target position is predicted incorrectly, the target detector's performance will significantly degrade. Specifically, this invention designs a differentiable representation method and an optimization method to update the position, thickness, and color of the adversarial stripes. Compared to complex textures, the proposed monochrome stripe method minimizes distortion in physical image processing, thus ensuring high attack performance in the physical domain. Extensive experiments were conducted on the DOTA / NWPU VHR-10 dataset and real-world scenarios to evaluate the effectiveness of various attack methods. Results show that adversarial stripes outperform previous texture-based adversarial patches when facing multiple white-box, black-box, and defensive detectors. More importantly, even in remote sensing scenarios at a height of 300 meters, the method maintains a high success rate in physical attacks, while the texture-based adversarial patches have minimal attack effectiveness.

[0041] This invention also proposes a cross-modal transfer attack method to enable targets to evade detection. This invention requires identifying common features across different modalities to transfer perturbations between them. The shape, location, and number of patches are common features that can be captured by sensors of different modalities, allowing these factors to carry attack information that can be detected by detectors of different modalities. Although color information cannot be transferred across modalities, considering the texture preference of deep neural networks, it can serve as a guarantee of maintaining attack success rate against known modalities. Figure 2As shown, taking the transfer from visible light mode to infrared mode as an example, this invention attempts to simultaneously optimize all four factors of the adversarial patch to improve cross-modal transferability while ensuring the effectiveness of the white-box mode. Considering that gradients can provide important guidance for attacks, this invention attempts to directly optimize color, shape, position, and quantity through gradient descent. However, differentiable computation of all factors is a challenge, and this invention requires a differentiable modeling method to unify all factors. Therefore, this invention proposes a novel modeling method that allows adversarial patches to automatically split and merge to guide the simultaneous optimization of these four factors. Specifically, this invention employs a differentiable adversarial patch representation method. Throughout the optimization process, this invention uses a splitting and merging strategy to form corresponding adversarial patches with optimal shape and color at specific locations on the target. This allows for the direct application of gradient-based methods to simultaneously find the optimal values ​​for the patch's color, shape, position, and quantity. This invention has undergone comprehensive experiments on the VEDAI, LLVIP datasets, and real-world scenarios. Compared to existing methods, the attack method of this invention achieves better cross-modal transfer performance in different scenarios (i.e., long-range and short-range scenarios) and different model architectures (i.e., CNN-based and transformer-based).

[0042] This invention proposes a novel physical attack method based on monochromatic stripes, achieving highly efficient adversarial attacks with lower computational and manufacturing costs. This method not only performs excellently in laboratory environments but also effectively deceives target detection systems in real-world long-range scenarios. Furthermore, this invention proposes a target detection attack method that can transfer between visible and infrared modes. To enhance the transferability between different modalities while maintaining high performance for individual modalities, this invention proposes a novel method to simultaneously optimize the color, shape, position, and number of adversarial patches. Specifically, this invention introduces a differentiable representation of adversarial patches, which allows us to split and merge adversarial patches after each update, thereby optimizing the full factor of the adversarial patches. Experiments conducted on multiple datasets and physical world environments demonstrate that the proposed method achieves state-of-the-art performance across various scenarios and model architectures.

[0043] The monochrome strip-based physical attack method provided in this invention can improve the attack success rate and cross-modal transferability in remote sensing target detection tasks. The former generates adversarial examples capable of deceiving target detectors by optimizing the color, position, thickness, and orientation of the stripes. Compared to traditional texture-based adversarial patches, this method performs better in long-range physical attack scenarios and can effectively counter various detectors, including white-box and black-box attacks. The latter achieves cross-modal transferability by simultaneously optimizing the color, position, number, and shape of the adversarial patches.

[0044] The method of the present invention will now be described in detail: The present invention provides an adversarial attack method for remote sensing target detection, comprising the following steps: Acquire clean remote sensing scene images; Detect the target objects that need to be countered from attacks contained in the remotely sensed scene image; Construct adversarial sample images using the target object; The constructed adversarial sample image is optimized to obtain an optimized adversarial sample image; the optimized adversarial sample image is used for adversarial attacks on remote sensing target detection.

[0045] Example 1 The steps for implementing a stripe attack in this embodiment are as follows: Input: Source image Object detector object

[0046] Output: Adversarial Examples

[0047] Step 1: Initialize the parameter center point ,color ,thickness slope

[0048] Step 2:

[0049] Step 3: for to

[0050] Step 4: Calculate the distance between each pixel and the center line.

[0051] Step 5: Calculate the approximate mask matrix of the left pixel.

[0052] Step 6: Calculate the approximate mask matrix of the right-side pixels.

[0053] Step 7: Calculate adversarial examples

[0054] Step 8: Obtain confidence level

[0055] Step 9: By minimizing To optimize and get

[0056] Step 10: Update the center point ,

[0057] Step 11: Update

[0058] Step 12: Cutting

[0059] Step 13: Return .

[0060] The specific implementation process of this embodiment is as follows: Given a clean remote sensing scene image A well-trained object detector It can correctly detect target objects contained in the image, that is ,in, It is the set of detected target objects. Each detected target object Both can be used as a tuple It means that among them and The coordinates represent the center of the bounding box, and w and h represent the width and height of the bounding box. and This represents the predicted class label and the predicted confidence score. The attack method proposed in this invention attempts to find the optimal adversarial condition. Attach it to the object The above constructs an adversarial example. (i.e., adversarial sample images). Then, You can use a deceptive object detector to make it miss objects. The detection. You can see... This is a hyperparameter indicating the maximum number of stripes to optimize. In the following content, we will use a single stripe as an example. Taking optimization as an example, the method of the present invention will be described in detail. For the case of multiple stripes, i.e. It only requires parallel optimization of multiple strips. One strip material The optimization can be expressed as: (1.1) Final adversarial example It can be represented as: (1.2) in, The single color indicates the contrast bar. Is with Binary masks with the same dimensions, Indicating the opposition clause . The value in express The corresponding pixel in the middle belongs to the adversarial stripe. Clearly, This includes information on the location and thickness of the strip. The objective of this invention is to optimize... and To obtain the optimal adversarial stripe. It should be emphasized that, in existing technologies, The attack process remains fixed because existing methods simply optimize the RGB values ​​of each pixel in a texture-based patch, the position of which is pre-defined before the attack. As shown in Equation (1.2), the adversarial bar is composed of color. and mask matrix Sure. This indicates that the stripes have uniform RGB values, ranging from 0 to 1. 255. However, It is a non-differentiable binary matrix, and therefore cannot be updated by gradient-based optimization. Therefore, it is necessary to design a differentiable representation to approximate it. This enables reverse propagation.

[0061] Figure 3 The left side illustrates the representation of the adversarial stripe. It consists of pixels whose distance from the center line (red line) of the adversarial stripe is less than half the thickness of the adversarial stripe; therefore, the mask matrix... It can be calculated as follows: (1.3) in, Represents pixels Distance from the center line, This represents half the strip thickness. (This is used to approximate the thickness in a differentiable manner.) First, the distance from each pixel to the center line is calculated, and a differentiable function is used to map the distance to 0 or 1. For optimizing only one adversarial strip, the center of the target is chosen as the initial center point of the strip. Therefore, the expression for the center line of the antagonistic strip is: (1.4) Where k is the slope parameter of the adversarial strip centerline. The distance for each pixel can be calculated as follows: (1.5) Next, we design a differentiable function to map the distance of each pixel as: (1.6) in, Set to -100, Figure 3 The mapping results are displayed on the right. You can see that if... If the value is less than 1, the function approximates the output as 1, while if... If the value is greater than 1, the function approximates the output as 0. Finally, the approximation matrix... Each value in can be calculated as: (1.7) Because of the function above It is continuously differentiable, so it can be used to propagate gradients during optimization. Therefore, the original binary mask... Approximately by the center point Slope parameters and thickness The combination of differentiable representations is replaced by, where and This determines the position of the adversarial stripes in the adversarial examples.

[0062] 2. Optimization process: Based on the above, optimizing adversarial stripes is equivalent to optimizing positional variables. and ,color and thickness Optimization. Among these parameters, , and The values ​​of these values ​​change continuously within a certain range and can be directly optimized through a gradient-based optimization process. However, the center point... This is a preset value, meaning that the adversarial strip optimizes its color, thickness, and slope only at a fixed point. To further improve the attack performance of this invention, an update method is designed to iteratively change the position of the center point of the adversarial strip. Figure 4 The iteration is given arrive Optimization process: First, the thickness of the strip (i.e., the resistance strip) is divided into two parts, left and right, by a centerline, and denoted as follows: and Center point and thickness parameters For iteration The update results. At the start of the iteration, and They are equal. During the optimization process, and This may differ; the center point should be moved towards the direction of the larger value. Specifically, for example... Figure 4 As shown, assuming optimization Enlarge (i.e.) ), become smaller (i.e.) ), along the tangent to the center line, the center point Update to the bottom right corner until... Again equals Center point The update process is as follows: (1.8) After moving the center point, the thickness should also be updated as follows: (1.9) In summary, the optimization process of one iteration mainly includes two steps. The first step is to minimize the objective... confidence level To optimize color parameters Slope parameters and thickness parameters That is, the loss function is expressed as: (1.10) The second step is to update the center point and thickness of the adversarial stripes, as shown in equations (1.8) and (1.9). After each iteration, the values ​​of these parameters are checked and trimmed to a reasonable range. express and The maximum thickness.

[0063] (1.11) This embodiment addresses an adversarial attack system for remote sensing target detection, used to implement the adversarial attack method for remote sensing target detection described in this embodiment. The system includes: Image acquisition module: used to acquire net remote sensing scene images; Target object detection module: used to detect target objects in the remote sensing scene image that require countermeasures against attacks; Adversarial Sample Image Construction Module: Used to construct adversarial sample images using the target object; Image optimization module: used to optimize the constructed adversarial sample image to obtain an optimized adversarial sample image; the optimized adversarial sample image is used for adversarial attacks on remote sensing target detection.

[0064] This embodiment verifies the attack performance of striping attacks in the digital world: Table 1 shows the success rates of white-box and black-box attacks. The success rates against YOLOv5 OBB and Oriented RCNN are shown on the left side of Table 1. On the DOTA dataset and the NWPU VHR-10 dataset, the success rates of the adversarial striping attacks against YOLOv5 OBB and Faster RCNN proposed in this invention (Ours) reach 88.4% / 70.1% and 87.2% / 78.9%, respectively. Compared with the texture-based adversarial patches proposed in TOG, APA, and PAAI, the method of this invention improves the success rate of attacks in digital domain white-box attacks by at least 10%. Figure 6 The detection results of adversarial examples generated by different attack methods are presented. It can be seen that all methods successfully hide objects with adversarial patches, preventing detection by YOLOv5 OBB and R-CNN in the digit domain. The main difference is that previous methods optimized for specific and complex textures, while the method of this invention optimizes for monochrome stripes. Furthermore, Figure 7 More subjective examples of bar-based attacks are shown, which successfully attacked various objects, including vehicles, baseball fields, ports, and football fields. Additionally, we present the success rate of black-box attacks, as shown on the right side of Table 1.

[0065] Table 1

[0066] IV. Attack Performance of Striped Attacks in the Physical World In addition to the aforementioned digital attacks, this invention also conducted experiments in real remote sensing scenarios to evaluate the physical attack performance of different methods. Specifically, this invention selected a black SUV and a gray sedan as target objects, and used a DJI Mini 2 drone to capture video from the ground to a height of 300 meters. The subsequent experiments primarily evaluated the physical attack performance between 120 meters and 300 meters. For close-range scenes, previous work has evaluated the effectiveness of texture-based adversarial patches. Furthermore, the detectors used in this invention's experiments were primarily trained on the DOTA dataset, which is suitable for remote sensing scenarios. At distances greater than 300 meters, objects in the captured images were too small to be detected. After obtaining real video, this invention first used YOLOv5 OBB to optimize the texture-based adversarial patch using TOG, APA, and PAAI, and then optimized the strip-based adversarial patch using the method proposed in this invention. These adversarial patches were then printed and affixed to the tops of the vehicles. Finally, this invention used a drone to capture real-world scenes to test the physical attack performance again. This invention evaluates physical white-box attacks against YOLOv5 OBB and physical black-box attacks against RCNN. Based on attack success rates using data captured at heights ranging from 120 to 300 meters, the proposed adversarial striping method achieves attack success rates of 76.2% and 64.1% against these two detectors, respectively, while texture-based adversarial patching attacks are largely ineffective. Figure 8 The first column visualizes optimized texture-based and stripe-based adversarial patches attached to vehicles in a real-world scene. The other three columns show the detection results of different methods at heights of 120m, 200m, and 300m. It can be seen that target objects with adversarial stripes were successfully hidden by the YOLOv5 OBB detector, while target objects with texture-based adversarial patches could still be detected. Furthermore, these experiments were conducted under both sunny and cloudy conditions, confirming that the proposed monochromatic adversarial stripes are more robust than texture-based adversarial patches when considering the effects of lighting conditions and atmospheric scattering in the physical world. To the applicant's knowledge, the applicant is the first institution to achieve a high success rate of physical attacks in remote sensing scenarios.

[0067] This invention also experimentally verified the attack performance of the same bar against different target detectors under different backgrounds. First, a bar chart was designed according to the method described above and affixed to a car. Then, the car was placed against different backgrounds. Next, video was captured from the ground to a height of 300 meters, and YOLOv5 OBB and directional R-CNN were used to detect the captured video. Figure 9The images show adversarial example detection results for YOLOv5 OBB and Oriented RCNN under two different backgrounds. The first column shows the detection results for cars without stripes, demonstrating that both detectors accurately detect the target car. The second and third columns show the detection results for cars with stripes under two different backgrounds. After installing stripes on the target car, both detectors ignore the target car and accurately detect other vehicles. Therefore, it can be concluded that the stripes generated by the method of this invention can simultaneously attack different detectors under different backgrounds.

[0068] Example 2 The steps of the cross-modal transferable attack implementation method in this embodiment are as follows: Input: Clean image Object detector object

[0069] Output: Adversarial Examples

[0070] Step 1: Initialize a set of parameters Center point A set of line segment lengths and a set of angles Color variables ; Step 2: for to

[0071] Step 3: Calculate the vertex coordinates of each patch Step 4: for to

[0072] Step 5: Calculate the corresponding formula for each patch.

[0073] Step 6: Obtain adversarial examples Step 7: Obtain The confidence score,

[0074] Step 8: By minimizing To optimize

[0075] Step 9:

[0076] Step 10: Split-Offensive Patch Step 11: Merge Against Patch Step 12: Binarize each And recalculate the adversarial examples Step 13: Return .

[0077] The specific implementation process of this embodiment is as follows: Given a trained object detector $f$, input a clean remotely sensed scene image. Receive test results ,Right now ,in It is a group of detected target objects . It is a collection containing the location and category information of the target object being detected. ,in and Represents the coordinates of the center point of the object. and This represents the width and height of the object. Represents the category label of an object. This represents the confidence score of the test results.

[0078] This invention attempts to find a set of adversarial patches that are optimizable across all factors, denoted as... For sets Each adversarial patch in: , The location determines The shape and location of the patch. Each patch has corresponding color information. Therefore, sets This contains information about the color, shape, location, and number of adversarial patches. It's worth noting that, for ease of implementation in the physical world, the color here represents a single hue. So, an adversarial example... It can be represented as: (1.12) Will The test results are recorded as The goal is to find a suitable set Adding adversarial patches causes the target to... Since it disappears, the problem can be expressed as: (1.13) in, To mitigate attack losses during the optimization process; 2. Optimization Methods As shown in equation (1.12), each adversarial patch is composed of a matrix. and color variables This indicates. However, Since the matrix is ​​a non-differentiable binary matrix, updating it directly via gradient is impractical. Therefore, a differentiable representation is needed to approximate it. This allows the optimization process to utilize gradients for rapid updates. Figure 5 The differentiable representation method of the present invention is described in the figure, in which the present invention uses a center point. A set of line segment lengths and a set of angles to indicate Adversarial patches. It's worth noting that... Indicates connection to vertices and center point The angle between the line and the horizontal axis. This can be expressed using the following equation, which uses these three sets of variables (i.e., ...). , , Solve for the coordinates of all vertices: (1.14) (1.15) in, This represents the coordinates of the nth vertex. Each pair of adjacent vertices... and and the center point This can form a sector. The entire patch can consist of N+1 sectors. For each point within the region, in the set... The problem of taking 0 or 1 can be transformed into the problem of whether the point is inside the sector. For a point Within the region, there always exists a straight line. and Intersect at . middle The value can be mapped to the following relationship: (1.16) Then, a step function is used to approximate this mapping. This function can be expressed as follows: (1.17) Based on the above discussion, Each point in Convert the value into a straight line and The problem of the ratio of lengths, in which It is a straight line and The intersection point. Using the gradient descent method, we can update... By updating This allows for optimization of the shape of adversarial patches. To further optimize the location and number of patches, this invention designs a split-merge method. After each update epoch, for each... The process involves splitting and merging. Specifically, in each round when the adversarial patch changes to a concave polygon, a splitting operation is performed, splitting the concave points to ensure that the adversarial patch remains a convex polygon, facilitating physical domain implementation. Secondly, overlapping polygons during optimization are merged into a single unit. This further optimizes the number and location of adversarial patches.

[0079] This embodiment addresses an adversarial attack system for remote sensing target detection, used to implement the adversarial attack method for remote sensing target detection described in this embodiment. The system includes: Image acquisition module: used to acquire net remote sensing scene images; Target object detection module: used to detect target objects in the remote sensing scene image that require countermeasures against attacks; Adversarial Sample Image Construction Module: Used to construct adversarial sample images using the target object; Image optimization module: used to optimize the constructed adversarial sample image to obtain an optimized adversarial sample image; the optimized adversarial sample image is used for adversarial attacks on remote sensing target detection.

[0080] This embodiment verifies the attack performance of cross-modal transferable attacks in the digital world: First, the effectiveness of various attack methods was tested in a white-box scenario. It's important to note that "white-box" refers to the visibility of parameters in the visible model, while the infrared model, sharing the same structure, remains opaque. Adversarial patches were generated in the visible light mode and then directly transferred to the infrared mode. Considering that thermal insulation materials are physically more accessible, the adversarial patches were implemented in black when switching to the infrared mode. Figure 10 The upper part illustrates the attack effects of different methods in the visible light mode and directly transferred to the infrared mode. It can be seen that the method of this invention can simultaneously optimize all four factors, thus exhibiting good cross-mode transferability. Figure 10 The second half details the optimization process of the method of the present invention.

[0081] This embodiment verifies the attack performance of cross-modal transferable attacks in the physical world: In addition to the aforementioned attacks targeting the digital world, this invention also conducted experiments in the real world to evaluate the effectiveness of the proposed method against attacks in the physical world. Figure 11 and Figure 12This invention presents statistical data on physical world attacks and some visual results. It can be seen that, compared to the best-performing baseline methods (i.e., TOG in the visible light mode and UAP in the infrared mode), the method of this invention still maintains better attack performance. However, when the distance is too close or too far, or the angle is too large, the attack effectiveness decreases due to the loss of adversarial patch information. This phenomenon is more pronounced for remote sensing. Maintaining performance at different distances and angles in adversarial attacks targeting target detection tasks remains a significant challenge, requiring the exploration of more effective solutions in future work.

[0082] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An adversarial attack method for remote sensing target detection, characterized in that, The method comprises the following steps: obtaining a pure remote sensing scene image; detecting a target object in the remote sensing scene image that needs to be attacked by an adversarial attack; constructing an adversarial sample image using the target object; optimizing the constructed adversarial sample image to obtain an optimized adversarial sample image; and 2. The method of claim 1, wherein, the optimized adversarial sample image is used for an adversarial attack on remote sensing target detection.

3. The method of claim 2, wherein, Adversarial sample image The expression is as follows: wherein, represents the single color of the adversarial strip, is an approximation matrix of the mask matrix that is a remotely sensed scene image taken with the same dimensions as the binary mask, represents the adversarial strip , the value in the adversarial sample image at the corresponding position belongs to the adversarial strip, contains the position and thickness information of the corresponding strip of the adversarial strip.

4. The method of claim 3, wherein, Approximate matrix Each value in the matrix is computed from the expression As follows: wherein, d x y represents the distance between the pixel on the adversarial strip and the center line of the adversarial strip; represents half of the thickness of the adversarial strip, the function is continuously differentiable, , is set to a set value, and the value is not greater than -100.​​ 5. The method of claim 3, wherein, The process of constructing an adversarial sample image using the target object comprises the following steps: By minimizing the confidence of the target object To optimize the color parameter , the slope parameter And the thickness parameter, where the thickness parameter, respectively, divides the thickness of the opposing strip into two side portions divided by the center line.​ an adversarial strip is arranged on the top of the target object to construct an adversarial sample image. In particular, one of the opposing strips is optimized in the following way: wherein, is a loss function, is a confidence score of the target object , is a well-trained object detector, is a set of detected target objects, a target object is represented by a tuple , wherein, and represent the coordinates of the center of the bounding box, and w and h represent the width and height of the bounding box, and represent the predicted class label and the predicted confidence score.

6. The method of claim 1, wherein, When the constructed adversarial sample image is optimized, the adversarial strip is optimized, comprising the following steps:

7. The adversarial attack method for remote sensing target detection according to claim 6, characterized in that, Adversarial sample image The expression is as follows: wherein, x is a remote sensing scene image, m i is an adversarial patch for each adversarial patch, the adversarial patch contains information about the color, shape, location and number of adversarial patches, , the area where the adversarial patch is located determines the shape and location of the patch, is a color variable.

8. The method of adversarial attack for remote sensing target detection according to claim 7, characterized in that, When optimizing the constructed adversarial sample image, the adversarial patch is optimized to find a suitable combination of adversarial patches , so that the addition of the adversarial patch causes the target to disappear, so the expression for optimizing the adversarial patch can be expressed as: in, To mitigate attack losses during the optimization process; It is a group of detected target objects ;Target It is a collection containing the location and category information of the detected object. , and Represents the coordinates of the center point of the object. and This represents the width and height of the object. Represents the category label of an object. The confidence score represents the test results. For adversarial sample images The test results .

9. The method of claim 8, wherein, iteratively updating the center point and thickness of the adversarial strip, and after each iteration, checking the values of the center point and thickness of the adversarial strip and clipping the center point and thickness of the adversarial strip to a preset range. Using the center point , a set of line segment lengths , and a set of angles to represent the patch, the angle between the line connecting the vertex and the center point and the horizontal axis; the coordinates of all vertices are solved by the following equations using , , these three sets of variables: wherein, denotes the coordinates of the nth vertex, each pair of adjacent vertices and and the center point form a sector, the entire patch being composed of N+1 sectors; for each point in the region, the question of taking 0 or 1 in the set is translated into the question of whether the point is inside the sector or not, for the point there is always a straight line intersecting at , the value of in is mapped to the following relationship: The process of constructing an adversarial sample image using the target object comprises the following steps: Bundle Each point in Convert the value into a straight line and The problem of the ratio of lengths, in which, It is a straight line and At the intersection point, use gradient descent to update To optimize the shape of adversarial patches; after each update epoch, for each Execute splitting and merging: Specifically, in each round when the adversarial patch changes to a concave polygon, a splitting operation is performed to split the concave points, thereby ensuring that the adversarial patch is always a convex polygon. For polygons that overlap during the optimization process, a merging operation is performed to merge the overlapping polygons into a whole.

10. A system for adversarial attack against remote sensing target detection, characterized in that, an adversarial patch is used to construct an adversarial sample image. The process of optimizing the constructed adversarial sample image comprises the following steps: a step function is used to approximate the mapping relationship, and the step function is expressed by the following formula: comprising: an image acquisition module for acquiring a pure remote sensing scene image; a target object detection module for detecting a target object in the remote sensing scene image that needs to be attacked by an adversarial attack; an adversarial sample image construction module for constructing an adversarial sample image using the target object; an image optimization module for optimizing the constructed adversarial sample image to obtain an optimized adversarial sample image; the optimized adversarial sample image is used for an adversarial attack on remote sensing target detection.

Images