An adversarial shape patch attack method for infrared target detection

Abstract

The application belongs to the technical field of artificial intelligence security, and relates to a method for an adversarial shape patch attack against infrared target detection, comprising the following steps: 1, parameter and model initialization, obtaining an original image, loading an infrared target detection model to be attacked and locking parameters; 2, Fourier shape parameter initialization and generation of an initial patch through a winding number differentiable mapping; 3, patch transformation and generation of an adversarial image; 4, inputting the adversarial image into a detector to obtain a target confidence score subset of all candidate boxes; 5, constructing a loss function and updating Fourier coefficients using an optimizer; 6, after repeating iteration for a maximum number of times, obtaining optimized Fourier coefficients and a final patch; the application realizes a differentiable mapping from a contour to a pixel mask by parameterizing a patch closed contour through a Fourier series and combining a winding number theorem, and solves the problems of insufficient shape representation capability, poor fineness and dependence on an explicit aggregation function in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence security technology, specifically relating to an adversarial shape patch attack method for infrared target detection. Background Technology

[0002] Infrared target detection is widely used in security and nighttime driving scenarios. Its imaging mechanism is based on thermal radiation rather than visible light reflection, which limits the effectiveness of traditional texture-based adversarial patches in the infrared domain. To address this, existing technologies are gradually shifting from texture optimization to physical heat shielding attacks based on infrared imaging mechanisms, using the patch's geometry as the primary adversarial carrier. Based on this, relevant existing technologies mainly include the following two representative schemes: The central ray star-shaped approach uses shape attributes as the semantic carrier of adversarial attacks. Building upon traditional adversarial patches that only optimize texture and have fixed shapes, it introduces deformable contour representations. The core idea is to construct a polygonal contour using a center point and multiple rays, and then use triangular geometric relationships to determine whether pixels are inside or outside the contour, thus differentiably mapping the contour to a patch. Furthermore, by jointly optimizing and updating both the patch texture and ray length—that is, updating the patch shape—and using the mask mean as an area penalty, it can achieve good attack results even with a smaller area.

[0003] An infrared adversarial patching method based on pixel grid direct learning of masks: This method focuses on physical adversarial patches for infrared target detection. Utilizing the imaging mechanism of an infrared camera, it causes the detector to miss targets by attaching heat-insulating material patches to them. The key is not optimizing the patch texture, but directly learning the shape and position of the patches: It proposes an aggregation regularization based on local clustering coefficients, allowing high-value pixels in the mask to automatically cluster into compact, connected, and fabricable shapes. Simultaneously, binary and sparsity constraints are added to make the mask tend towards 0 or 1 and control the patch area. Gradient descent can be used to simultaneously obtain the feasible shape and landing point, and the patch attachment to the target can be restricted by the target region mask.

[0004] However, existing infrared target detection methods have the following drawbacks: The method based on central ray star shape is based on star polygon modeling, which has insufficient shape representation ability, resulting in poor attack effect: In this technique, the patch outline is modeled as a polygon composed of a central point and multiple rays, which is essentially a star polygon representation. The shape can only be stretched along the ray direction, making it difficult to depict general concave shapes or more complex closed boundaries. Therefore, the searchable shape space is limited.

[0005] Infrared adversarial patching methods based on pixel-grid direct mask learning suffer from insufficient shape representation capabilities and poor precision. Furthermore, they require additional explicit aggregation functions for constraint, impacting offensive capabilities. Since directly optimizing patches on the pixel grid cannot effectively represent complex and irregular boundaries, the shape accuracy of the patches is limited, potentially resulting in shapes with insufficient detail and uneven boundaries, thus affecting adversarial effectiveness. Therefore, explicit aggregation functions are introduced to force high-value pixels to cluster into compact connected regions. While this improves shape stability and physical realizability, it also negatively impacts offensive capabilities, particularly in terms of shape representation capabilities and precision. Ultimately, the strength and adaptability of attacks are limited.

[0006] Therefore, a new method is needed that can simultaneously address the shortcomings of the two types of methods mentioned above. Summary of the Invention

[0007] This invention aims to induce a missed detection by the detector by attaching the resulting patch to the target object. Specifically, the shape is parameterized using Fourier series, and the patch image is mapped to a pixel mask using rotational integrals to generate an adversarial patch image. This image is then rendered onto the target image and input into the detector to calculate the loss. Backpropagation is used to optimize the Fourier coefficients, while high-frequency regularization is introduced to suppress noise, ensuring the patch shape is smooth and realistic. Finally, the generated adversarial patch is attached to the target surface, inducing a missed detection by the infrared detector.

[0008] This invention provides the following technical solution: a method for adversarial shape patching attacks against infrared target detection, comprising the following steps: Step 1: Parameter and model initialization, acquire the original image, load the infrared target detection model to be attacked and lock the parameters.

[0009] Step 2: Initialize Fourier shape parameters and generate initial patch by differentiable mapping of the number of turns.

[0010] Step 3: Patch transformation and generate adversarial image; The obtained initial patch is scaled to the previously set parameter multiple according to the length and width of the bounding box of the target to be attacked, and then randomly enhanced. It is then put into the renderer along with the original image to obtain the adversarial image.

[0011] Step 4: Input the adversarial image into the detector and obtain the molecular set of target confidence scores for all candidate boxes.

[0012] Step 5: Construct the loss function and update the Fourier coefficients using the optimizer.

[0013] Step 6: Repeat steps 2 to 5 until the maximum number of iterations is reached, then obtain the optimized Fourier coefficients and the final patch.

[0014] Preferably, in step 1, the locking parameters include: the patch scaling ratio relative to the target box, the loss weight and the high-frequency constraint coefficient, the optimizer type and the learning rate.

[0015] Preferably, step 2 includes the following sub-steps: Step 2-1: Randomly initialize the Fourier coefficient set as shape parameters, set the Fourier series order, and generate a continuous closed contour.

[0016] Step 2-2: Determine whether each pixel is inside or outside the contour based on the winding number integral. The winding number inside the contour is close to 1; the winding number outside the contour is close to 0, thus obtaining the original winding number image.

[0017] Steps 2-3: After cropping the original wrap-around image, pixel-level initial shape patches are obtained.

[0018] More preferably, in step 2-1, the closed contour is: (2) in, Let i represent the Cartesian coordinates of the contour boundary, where i is the imaginary unit. K Represents the highest frequency component. These represent the Fourier series coefficients corresponding to different frequency components.

[0019] In step 2-2, the number of turns integral is: (3) in, Represents the coordinates of pixels on the wraparound map. , Each represents The x-coordinate and y-coordinate of the point.

[0020] In steps 2-3, the formula for cropping the obtained original wrap-around image is as follows: (4) in, Indicates truncation, limitation The maximum value; This represents the original wrap-around number image.

[0021] Preferably, step 5 includes the following sub-steps: Step 5-1: Construct adversarial loss. Step 5-2: Calculate the fundamental frequency amplitude based on the current Fourier coefficients as the scale reference for the main shape of the contour; iterate through all high-frequency terms, and if the amplitude exceeds the threshold, construct a regularization loss.

[0022] Step 5-3: Construct the total loss using the loss functions obtained in Steps 5-1 and 5-2.

[0023] Step 5-4: Based on the loss obtained in step 5.3, use the optimizer to update the Fourier coefficients in reverse.

[0024] More preferably, in step 5-1, the adversarial loss is: (5) in, I Represents the original image. This indicates the initial patch. This indicates the scaling ratio of the patch relative to the target bounding box, where B represents the target bounding box. Represents geometric transformation.

[0025] In step 5-2, the regularization loss is: (7) in, These represent the Fourier series coefficients corresponding to different frequency components. Represents the high-frequency constraint coefficient. , This represents the fundamental frequency component.

[0026] In step 5-3, the total loss is: (8) in, This represents the hyperparameter used to balance shape regularity.

[0027] The beneficial effects of this invention are: 1. This invention uses a small number of learnable Fourier coefficients to parametrically model the closed contour of a patch, and based on the winding number theorem, rasterizes the continuous contour into a pixel-level shape mask in a differentiable manner, achieving end-to-end differentiable mapping from shape parameters to the patch mask. Subsequently, the patch is adaptively scaled and attached according to the target bounding box, generating an adversarial image input detector. A joint suppression attack loss is constructed on the NMS candidate detection boxes output by the detector, causing an overall decrease in the confidence of the target-related candidate boxes to induce missed detections. To avoid excessive high-frequency noise at the shape boundary, this technique uses frequency domain regularization with high-frequency energy suppression to limit the amplitude of high-frequency harmonics, improving the smoothness of the patch. Finally, a trimmed template can be exported and fabricated using heat-insulating material, then attached to the target surface to achieve infrared detection attack.

[0028] 2. The Fourier series representation of this invention is not constrained by star-shaped structures. It can progressively expand the shape complexity through the highest frequency component, covering more and more complex closed contour shapes, thereby increasing the searchable shape space and attack limit, and thus having better shape representation capabilities. Moreover, this invention introduces a high-frequency energy suppression regularization to limit the growth of high-frequency harmonic amplitude relative to the fundamental frequency, suppressing jagged and spiked boundaries, improving boundary smoothness while maintaining expressive power, and ultimately obtaining more stable infrared missed detection attacks.

[0029] 3. This invention uses Fourier coefficients to parameterize closed contours in low dimension and achieves analytical differentiable rasterization of contours to pixel masks through the winding theorem, making the shape more coherent. At the same time, this invention does not require explicit aggregation functions, resulting in better precision. Furthermore, on the detection side, the target-related candidate boxes are jointly suppressed, making the missed detection effect more stable under different confidence threshold settings. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating an adversarial shape patch attack method for infrared target detection according to the present invention. Figure 2 This is a schematic diagram of the visualization results of an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the optimization results of whether or not shape regularization constraint adversarial patching is used in an embodiment of the present invention; Figure 4 This is a schematic diagram of the method steps of the present invention. Detailed Implementation

[0031] The relevant technologies of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0032] like Figures 1-4 As shown, the specific implementation process of the adversarial shape patch attack method for infrared target detection in this embodiment follows the following steps: Step 1: Parameter and model initialization; obtain the original image. I， Load the infrared target detection model to be attacked f And lock the parameters; including: the patch scaling ratio relative to the target bounding box. Loss weights , and high-frequency constraint coefficient Optimizer type and learning rate, etc.

[0033] Step 2: Initialize Fourier shape parameters and generate initial patches by differentiable mapping of the number of turns.

[0034] Step 2-1: Randomly initialize the set of Fourier coefficients As a shape parameter, the Fourier series order K is set, and a continuous closed contour is generated using formula (2).

[0035] Step 2-2: Based on the winding integral of formula (3), determine whether each pixel is inside or outside the contour. Inside the contour, the winding number is close to 1; outside, it is close to 0, thus obtaining the original winding number image. .

[0036] Steps 2-3: Obtain the After cropping to [0,1] as shown in formula (4), a pixel-level initial shape patch is obtained. . Step 3: Patch transformation and generate adversarial image; the resulting initial patch Scale the target to the specified dimensions based on the bounding box of the target. Then, it is randomly enhanced and compared with the original image. I Put it into the renderer to get the adversarial image .

[0037] Step 4: Apply adversarial images Input detector f In the process, obtain the molecular set of target confidence scores for all candidate boxes. P .

[0038] Step 5: Construct the loss function and update the Fourier coefficients using the optimizer.

[0039] Step 5-1: Before NMS, the same target often corresponds to multiple candidate boxes and has a high confidence level. If only the highest-scoring candidate box is suppressed, other candidate boxes may still be retained, resulting in the target still being detected. Therefore, formula (6) is used to construct the adversarial loss. .

[0040] Step 5-2: Calculate the fundamental frequency amplitude based on the current Fourier coefficients. Used as a scale reference for the main shape of the outline. Then iterate through all high-frequency terms. If its amplitude Exceeding the threshold Then, the regularization loss is constructed using formula (7).

[0041] Step 5-3: Construct the total loss using the loss functions obtained in steps 5.1 and 5.2 according to formula (8). .

[0042] Step 5-4: Based on the loss obtained in Step 5.3, use the optimizer to update the Fourier coefficients in reverse. .

[0043] Step 6: Repeat steps 2-5 until the maximum number of iterations T is reached, then obtain the optimized Fourier coefficients. Get the final patch.

[0044] This invention sets the infrared image as The truth box of the target to be attacked is The infrared detector is The detector outputs a set of candidates that have not undergone NMS. ,in Let represent the candidate bounding box and its confidence score, respectively. A set of learnable Fourier coefficients is used. Define a closed contour and use a differentiable mapping module. G Generate shape mask: (1) The closed contour is parameterized by a Fourier series as follows: (2) in Let i represent the Cartesian coordinates of the contour boundary, where i is the imaginary unit. K The highest frequency component determines the complexity of the shape.

[0045] To transform a continuous closed contour into a mask usable by a pixel mesh, this invention employs an analytical mapping based on the number of turns theorem: first, the number of turns at each pixel is calculated to obtain the original number of turns map. As shown in formula (3), the final mask is obtained by absolute value normalization, as shown in formula (4): (3) (4) get Subsequently, this application defines a geometric transformation. ; by scaling right Scale the image to make its size proportional to the target bounding box; then translate the scaled patch to the center of the target bounding box to obtain a proportionally scaled image; finally, multiply the shape patch element-wise with the original image to obtain the adversarial image. (5) Will Input detector This yields the set P of all candidate detection boxes output before non-maximum suppression (NMS). From this set, a subset associated with the target box B is selected. The confidence level of this subset is used as the target of attack suppression.

[0046] To achieve stable false negatives, this method employs a multi-candidate box joint suppression strategy, which simultaneously reduces the confidence of all candidate boxes related to the target. Therefore, the attack loss is written as: (6) To avoid excessive high-frequency noise at the shape boundaries, we use low-frequency dominant constraints based on the Fourier coefficients: when The harmonic amplitude exceeds Regarding penalties, this makes shape patches practically feasible in physical deployments: (7) Therefore, the total loss is: (8) in It is a hyperparameter used to balance shape regularity.

[0047] This implementation method is based on an end-to-end adversarial shape framework using Fourier series modeling. It uses Fourier series to parametrically model the closed adversarial patch contour and utilizes the winding number theorem to achieve a differentiable raster mapping from the contour to the pixel-level mask. This constructs an infrared physical adversarial patch generation framework that can jointly optimize the external contour shape and attachment scale position end-to-end, thereby achieving overall suppression of the confidence of infrared target detectors and stable missed detection.

[0048] This implementation uses a regularization method based on high-frequency energy constraints. By limiting the ratio of the amplitude of the high-frequency terms in the Fourier series to the fundamental frequency, it suppresses jagged edges and sharp boundaries generated during the optimization process. This regularization strategy effectively keeps the shape dominated by low frequencies, avoiding excessively complex details, thereby improving the smoothness and cutibility of the shape, and enhancing robustness to physical cutting errors.

[0049] Example To demonstrate the effectiveness and feasibility of the adversarial patching attack method for infrared target detection proposed in this invention, this embodiment uses the LLVIP infrared modal pedestrian target detection dataset for adversarial attack experiments. For this dataset, approximately 100 pedestrian images with clear outlines are randomly selected as targets for attack. Adversarial patching optimization is performed on each frame of the human image, and attack evaluation is conducted.

[0050] In the experiment, parameters for a human target detector targeting the infrared modality were loaded and fixed. Shape-based adversarial patches with Fourier series representations were generated according to the technology proposed in the patent. The patches were scaled, randomly enhanced, and then rendered onto the surface of a human target, allowing the patched human to evade detector recognition. The entire process employed end-to-end patch optimization technology using the Adam optimizer with a learning rate of 0.002, a maximum of 500 iterations per image, and the highest frequency component K set to 6. This optimized the Fourier coefficients and obtained the target patch shape. During evaluation, both the target confidence threshold and the IoU threshold were set to 0.5, and the adversarial success rate across all targets was calculated.

[0051] The comparative methods include DeformP, an adversarial patch generation technique based on a central ray star shape; InfraP, an adversarial patch generation technique based on a pixel mesh mask; and UNIPatch, an adversarial patch generation technique based on spline curves. For the YOLOv3 infrared target detector, the attack success rates (ASR, %) of the four methods on the same series of human targets at different confidence thresholds are shown in Table 1 below:

[0052] The results show that the method of the present invention has better attack effect under various difficulty levels (the lower the confidence threshold, the more difficult it is to successfully counterattack), and has obvious advantages. This verifies that the present invention has stronger adversarial performance than other methods in infrared modal target detection tasks.

[0053] Figure 2 Visualization results of this invention are presented. The shape representation method proposed in this invention has a high degree of freedom and higher adversarial potential compared to other methods. In addition, the shape regularization loss proposed in this method has a significant effect on reducing the shape complexity of adversarial patches and accelerating convergence.

[0054] Figure 3 The results of adversarial patch optimization with and without shape regularization are shown. In the confidence loss versus iteration number graph, the blue line represents the result with shape regularization loss, and the red line represents the result without shape regularization loss. The results show that shape regularization loss can prevent shape optimization from becoming overly complex, while accelerating the convergence of the optimization process.

[0055] In summary, the adversarial shape patch attack method for infrared target detection proposed in this invention effectively solves the problems of insufficient shape representation capability, poor precision, and reliance on explicit aggregation functions in existing technologies by parameterizing the closed contour of the patch using Fourier series and combining it with the winding number theorem to achieve differentiable mapping from the contour to the pixel mask. This method requires no additional explicit aggregation constraints and can generate smooth and complex closed contours. By constructing an attack loss for joint suppression of multiple candidate boxes and a high-frequency energy suppression regularization, it can significantly reduce the overall confidence of target-related candidate boxes, achieving stable missed detection attacks against infrared target detectors. Simultaneously, it ensures the physical fabrication of the patch shape and robustness to clipping errors, providing an efficient and practical new approach for adversarial attack research in infrared target detection.

[0056] It should be emphasized that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method for adversarial shape patch attacks against infrared target detection, characterized in that, Includes the following steps: Step 1: Parameter and model initialization, acquire the original image, load the infrared target detection model to be attacked and lock the parameters; Step 2: Initialize Fourier shape parameters and generate initial patches by differentiable mapping of the number of turns; Step 3: Patch transformation and generate adversarial image; The obtained initial patch is scaled to the previously set parameter multiple according to the length and width of the bounding box of the target to be attacked, and then randomly enhanced. It is then put into the renderer along with the original image to obtain the adversarial image. Step 4: Input the adversarial image into the detector and obtain the molecular set of target confidence scores for all candidate boxes; Step 5: Construct the loss function and update the Fourier coefficients using the optimizer; Step 6: Repeat steps 2 to 5 until the maximum number of iterations is reached, then obtain the optimized Fourier coefficients and the final patch. Step 5 includes the following sub-steps: Step 5-1: Construct adversarial loss; Step 5-2: Calculate the fundamental frequency amplitude based on the current Fourier coefficients as the scale reference for the main shape of the contour; iterate through all high-frequency terms, and if the amplitude exceeds the threshold, construct a regularization loss. Step 5-3: Construct the total loss using the loss functions obtained in steps 5-1 and 5-2; Step 5-4: Based on the loss obtained in step 5-3, use the optimizer to update the Fourier coefficients in reverse. In step 5-1, the adversarial loss is: （5） in, I Represents the original image. This indicates the initial patch. This indicates the scaling ratio of the patch relative to the target bounding box, where B represents the target bounding box. Represents geometric transformations; In step 5-2, the attack loss is: （6） in, Indicates the candidate box. This represents a subset associated with the target box. Indicates confidence level; The regularization loss is: （7） in, These represent the Fourier series coefficients corresponding to different frequency components. Represents the high-frequency constraint coefficient. , Indicates the fundamental frequency component; In step 5-3, the total loss is: （8） in, This represents the hyperparameter used to balance shape regularity.

2. The method for adversarial shape patch attacks against infrared target detection according to claim 1, characterized in that, In step 1, the locking parameters include: the patch scaling ratio relative to the target box, the loss weight and the high-frequency constraint coefficient, the optimizer type and the learning rate.

3. The method for adversarial shape patch attacks against infrared target detection according to claim 1, characterized in that, Step 2 includes the following sub-steps: Step 2-1: Randomly initialize the set of Fourier coefficients as shape parameters, set the order of the Fourier series, and generate a continuous closed contour; Step 2-2: Determine whether each pixel is inside or outside the contour based on the wrapping number integral. The wrapping number inside the contour is close to 1; the wrapping number outside the contour is close to 0, thus obtaining the original wrapping number image. Steps 2-3: After cropping the original wrap-around image, pixel-level initial shape patches are obtained.

4. The method for adversarial shape patch attacks against infrared target detection according to claim 3, characterized in that, In step 2-1, the closed contour is: （2） in, Let i represent the Cartesian coordinates of the contour boundary, where i is the imaginary unit. K Represents the highest frequency component. These represent the Fourier series coefficients corresponding to different frequency components; In step 2-2, the number of turns integral is: （3） in, Represents the coordinates of pixels on the wraparound map. , Each represents The x-coordinate and y-coordinate of the point; In steps 2-3, the formula for cropping the obtained original wrap-around image is as follows: （4） in, Indicates truncation, limitation The maximum value; This represents the original wrap-around number image.

Images