Infrared vehicle detection adversarial patch generation method based on particle swarm optimization algorithm

Abstract

The invention discloses an infrared vehicle detection adversarial patch generation method based on a particle swarm optimization algorithm, and belongs to the technical field of infrared image processing and security attack. The method comprises the following steps of: firstly, performing vehicle segmentation on an input infrared image by using an image segmentation model to obtain an accurate vehicle segmentation mask for positioning a subsequent patch; thirdly, initializing an adversarial patch according to the segmentation mask, inputting the adversarial patch into an infrared vehicle detection model (such as YOLOv5) for detection, and generating a preliminary adversarial sample; then, defining a new loss function, and optimizing the generated adversarial patch by minimizing the confidence score of a detector on the patch, so as to enhance the attack effect; and finally, iteratively optimizing the position and the shape of the patch by adopting an improved particle swarm optimization (PSO) algorithm in combination with a shape optimization module so as to maximize the attack success rate, and ensuring the robustness and the adaptability of the patch in a real environment in combination with an expectation transformation (EOT) technology. According to the method, the attack success rate of the infrared vehicle detector is remarkably improved, and an innovative and efficient solution is provided for attack resistance of the infrared vehicle detector.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and artificial intelligence technology, and in particular to the vehicle detection system in infrared images. A method for generating adversarial patches for infrared vehicle detection based on a particle swarm optimization algorithm is proposed, aiming to reveal the security issues of infrared vehicle detection. Background Art

[0002] With the continuous development of intelligent assisted driving systems, infrared thermal imaging cameras have gained widespread application in this field. These cameras can provide clear images in low-light conditions and in harsh environments such as rain, snow, fog, haze, and dust. They overcome the limitations of other sensors in complex environments and thus improve driving safety. In intelligent assisted driving systems, infrared target detection is particularly crucial for understanding the surrounding environment and ensuring road safety. Therefore, the robustness and accuracy of infrared target detection directly affect the overall performance of the system.

[0003] Similar to visible light object detection, infrared object detection relies on deep neural networks (DNNs), making it vulnerable to adversarial attacks. In recent years, research on adversarial attacks against visible light vehicle detection has garnered significant attention. For example, Full Cover Attack (FCA) successfully deceives detectors by using the full three-dimensional vehicle surface for powerful camouflage attacks. However, these methods rely on color and texture rendering and are not suitable for infrared images, as their features and representation differ from those of visible light images.

[0004] Existing adversarial attack methods for infrared target detection primarily focus on pedestrian detection, rather than vehicle detection, and most are not open source. Many methods employing adversarial patches rely on bounding boxes (BBOs). However, compared to pedestrians, vehicles are larger, more complex, and have varying dimensions. This leads to misalignment in the placement of adversarial patches, resulting in poor attack effectiveness and impractical physical implementation for vehicle detection. Some methods also suffer from low algorithmic efficiency and prolonged optimization time, limiting their application in vehicle target detection. Furthermore, some methods face challenges and high costs in physical implementation, potentially making the target conspicuous, limiting their adaptability to diverse environments. These attack methods typically focus on patch optimization on a single image, resulting in a unique adversarial patch for each image and a lack of generalizability. Consequently, a patch that successfully attacks a single frame may not be effective in subsequent frames, especially when the viewpoint and environmental conditions change.

[0005] To address the limitations of existing infrared vehicle detection countermeasures, this paper proposes InfVAP (Infrared Vehicle Countermeasures Patch Generation Method). This method is an efficient and robust attack method that is easy to implement in the physical field. Summary of the Invention

[0006] This paper proposes a method for generating adversarial patches for infrared vehicle detection based on a particle swarm optimization algorithm. By integrating particle swarm optimization with precise segmentation, adversarial sample initialization, a dedicated loss function, and dynamic parameter adjustment, along with the Expected Transform (EOT) technique, this method generates adversarial patches suitable for attacking infrared vehicle detectors such as YOLOv5. This method balances the differences between the digital and physical domains, maintaining a high attack success rate across multiple scenarios while facilitating practical physical deployment.

[0007] The present invention is achieved through the following technical solutions:

[0008] Step 1: Use the image segmentation model to segment the vehicle in the input infrared image and obtain an accurate vehicle segmentation mask, which is represented as the boundary of the vehicle area.

[0009] Step 2: Initialize the adversarial patch based on the segmentation mask and input it into the infrared vehicle detection model for detection to generate preliminary adversarial samples.

[0010] Step 3: Define a new loss function to optimize the generated adversarial patch by minimizing the detector's confidence score for the patch.

[0011] Step 4: An improved particle swarm optimization (PSO) algorithm is used to iteratively optimize the patch's position and shape, combined with a shape optimization module to maximize the attack success rate. The expected transformation (EOT) technique is also used to transform the patch at multiple angles and scales, enhancing its robustness and adaptability in real-world environments.

[0012] Furthermore, the step 1 is specifically as follows: using the DeepLab-v3+ semantic segmentation network to perform pixel-level segmentation on the input infrared image X to obtain the vehicle area mask Mask, which is expressed as:

[0013] Mask=u(X)

[0014] where u(·) represents the forward mapping of the DeepLab-v3+ network. The network uses dilated convolution to expand the receptive field, so that the segmentation IoU reaches about 0.89, providing an accurate reference for subsequent patch positioning.

[0015] Furthermore, the step 2 is specifically as follows: in the segmentation mask area, initialize m triangle adversarial sub-patches, each sub-patch is composed of a position set Shape matrix m∈{0,1}4×8 , where when the j-th element of the matrix M is 1, a small isosceles triangle is drawn at the corresponding grid position.

[0016] Furthermore, the step three is specifically as follows: using the infrared detection model f to detect X adv Output the confidence vector V of the first 1000 detection boxes obj (i), define the attack loss function:

[0017]

[0018] Where sort(·) indicates that the confidence is sorted in descending order. By minimizing L obj , which significantly reduces the detector's confidence in the target vehicle.

[0019] Furthermore, the step 4 is specifically as follows: in the iter-th iteration, the position of the i-th particle and speed Updates as follows:

[0020]

[0021] in, is the velocity of the i-th particle at time t+1, is the velocity of the i-th particle at time t, is the position of the i-th particle at time t, pbest i is the best historical position of the ith particle, gbest is the best position among all particles, w is the inertia weight, c1 and c2 are acceleration constants, and r1 and r2 are random numbers between [0,1].

[0022] During this process, the particle swarm optimization algorithm employs a dynamic adjustment mechanism, enabling it to optimize the search process as iterations progress. Specifically, the inertia weight w decreases with increasing iterations to balance global and local search capabilities. The acceleration constants c1 and c2 are also adjusted, with c1 decreasing and c2 increasing with increasing iterations. This encourages particles to explore a wider search space in the early stages and to focus more on local search in the later stages, thereby accelerating convergence.

[0023]

[0024] For each candidate adversarial example X adv Sampling multiple sets of geometric and noise transformations under the transformation distribution T (including rotation, scaling, translation, interpolation perturbation and Gaussian noise), calculate the expected adversarial sample:

[0025]

[0026] And f(X adv_eot ) output participates in L obj Calculation is performed to improve the attack stability of the patch at different perspectives and scales.

[0027] When the number of iterations iter reaches the preset maximum iter max or L obj When it is lower than the threshold, the algorithm terminates; the optimal adversarial patch is determined by the shape matrix and position set corresponding to the global optimal particle gbest, and the final patch template for physical pasting is output. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. It is obvious that the drawings described below are only some embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0029] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0030] Figure 2 This is a specific implementation process framework diagram of the method proposed in the present invention;

[0031] Figure 3 Schematic diagram of adversarial patch shape control and sub-patch initialization;

[0032] Figure 4 This is a schematic diagram of the physical domain adversarial patch deployment and infrared detection effect, showing the comparison results of YOLOv5 detection after the patch is attached to the vehicle surface. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] See also Figure 1-4 As shown, the present invention provides a method for generating countermeasure patches for infrared vehicle detection based on a particle swarm optimization algorithm, comprising the following steps:

[0035] Step 1: Use the image segmentation model to segment the vehicle in the input infrared image and obtain an accurate vehicle segmentation mask, which is represented as the boundary of the vehicle area.

[0036] Step 2: Initialize the adversarial patch based on the segmentation mask and input it into the infrared vehicle detection model for detection to generate preliminary adversarial samples.

[0037] Step 3: Define a new loss function to optimize the generated adversarial patch by minimizing the detector's confidence score for the patch.

[0038] Step 4: An improved particle swarm optimization (PSO) algorithm is used to iteratively optimize the patch's position and shape, combined with a shape optimization module to maximize the attack success rate. The expected transformation (EOT) technique is also used to transform the patch at multiple angles and scales, enhancing its robustness and adaptability in real-world environments.

[0039] As an optimization solution for the above embodiment, step 1 is specifically as follows: the image segmentation model is DeepLab-v3+, which uses a deep convolutional neural network (CNN) to perform pixel-level segmentation of the image and accurately separate the vehicle area. The DeepLab-v3+ model uses dilated convolution technology to expand the receptive field and improve segmentation accuracy, especially in complex backgrounds, and can accurately extract the outline of the vehicle. The segmentation result is generated by the following formula:

[0040] Mask=u(X)

[0041] Here, u(X) represents the segmentation output of the DeepLab-v3+ model, and Mask is the vehicle segmentation mask, which accurately identifies the location of the vehicle in the infrared image. The DeepLab-v3+ model achieves an 89% Intersection over Union (IoU) using efficient atrous convolution techniques, improving segmentation accuracy in vehicle regions and providing a precise reference for subsequent patch optimization.

[0042] As an optimization solution for the above embodiment, the step 2 is specifically as follows: initialize the adversarial patch and further control the shape of the patch through the matrix M. The matrix M is a 4x8 binary matrix, in which each grid is 0 or 1, indicating whether a small triangle is drawn, thereby generating an optimized patch shape.

[0043] In this way, the shape of the patch is precisely controlled during the optimization process. The final patch shape M is optimized to ensure that it adapts to different environments and detection conditions and can effectively interfere with infrared vehicle detectors.

[0044] As an optimization solution for the above embodiment, the step three is specifically as follows: the loss function minimizes the confidence of the target detector output, specifically minimizing the confidence of the first 1000 detection boxes, thereby enhancing the effect of the attack.

[0045] In step 3, the loss function Lobj By minimizing the confidence of the target detector output, the goal is to make the target detector unable to recognize the vehicle. The specific expression of the loss function is:

[0046]

[0047] Among them, V obj (i) represents the target confidence of the i-th detection box. By sorting the confidence of these detection boxes and minimizing the confidence of the first 1000 detection boxes, the detector will not be able to correctly identify the vehicle, thus achieving the purpose of the adversarial attack.

[0048] As an optimization solution of the above embodiment, the step 4 is specifically as follows: in the iter-th iteration, the position of the i-th particle and speed Updates as follows:

[0049]

[0050] in, is the velocity of the i-th particle at time t+1, is the velocity of the i-th particle at time t, is the position of the iiith particle at time t, pbest i is the best historical position of the ith particle, gbest is the best position among all particles, w is the inertia weight, c1 and c2 are acceleration constants, and r1 and r2 are random numbers between [0,1].

[0051] During this process, the particle swarm optimization algorithm employs a dynamic adjustment mechanism, enabling it to optimize the search process as iterations progress. Specifically, the inertia weight w decreases with increasing iterations to balance global and local search capabilities. The acceleration constants c1 and c2 are also adjusted, with c1 decreasing and c2 increasing with increasing iterations. This encourages particles to explore a wider search space in the early stages and to focus more on local search in the later stages, thereby accelerating convergence.

[0052]

[0053] For each candidate adversarial example X adv Sampling multiple sets of geometric and noise transformations under the transformation distribution T (including rotation, scaling, translation, interpolation perturbation and Gaussian noise), calculate the expected adversarial sample:

[0054]

[0055] And f(X adv_eot ) output participates in Lobj Calculation is performed to improve the attack stability of the patch at different perspectives and scales.

[0056] When the number of iterations iter reaches the preset maximum iter max or L obj When it is lower than the threshold, the algorithm terminates; the optimal adversarial patch is determined by the shape matrix and position set corresponding to the global optimal particle gbest, and the final patch template for physical pasting is output.

Claims

1. A method for generating adversarial patches for infrared vehicle detection based on a particle swarm optimization algorithm, comprising the following steps: Step 1: Use the image segmentation model to segment the vehicle in the input infrared image and obtain an accurate vehicle segmentation mask, which is represented as the boundary of the vehicle area. Step 2: Initialize the adversarial patch based on the segmentation mask and input it into the infrared vehicle detection model for detection to generate preliminary adversarial samples. Step 3: Define a new loss function to optimize the generated adversarial patch by minimizing the detector's confidence score for the patch. Step 4: An improved particle swarm optimization (PSO) algorithm is used to iteratively optimize the patch's position and shape, combined with a shape optimization module to maximize the attack success rate. The expected transformation (EOT) technique is also used to transform the patch at multiple angles and scales, enhancing its robustness and adaptability in real-world environments.

2. The method according to claim 1, characterized in that The image segmentation model used in step 1 is DeepLab-v3+, which uses a deep convolutional neural network (CNN) to perform pixel-level segmentation of the image, accurately isolating the vehicle area. The DeepLab-v3+ model uses dilated convolution technology to expand the receptive field and improve segmentation accuracy, especially in complex backgrounds, and can accurately extract the outline of the vehicle. The segmentation result is generated using the following formula: Mask=u(X) Here, u(X) represents the segmentation output of the DeepLab-v3+ model, and Mask is the vehicle segmentation mask, which accurately identifies the location of the vehicle in the infrared image. The DeepLab-v3+ model achieves an 89% Intersection over Union (IoU) using efficient atrous convolution techniques, improving segmentation accuracy in vehicle regions and providing a precise reference for subsequent patch optimization.

3. The method according to claim 1, characterized in that The initialized adversarial patch in step 2 further controls the shape of the patch through the matrix M. The matrix M is a 4x8 binary matrix, in which each grid is 0 or 1, indicating whether a small triangle is drawn, thereby generating an optimized patch shape. When initializing the adversarial patch, the matrix M is used to control the patch's shape. Matrix M is a 4x8 binary matrix, with each element being either 0 or 1, indicating whether a small triangle should be drawn at that location. This allows the patch's shape to be precisely controlled during the optimization process. The final patch shape M is optimized to ensure its adaptability to diverse environments and detection conditions, effectively disrupting infrared vehicle detectors.

4. The method according to claim 1, wherein The loss function in step 3 minimizes the confidence of the target detector output, specifically minimizing the confidence of the first 1000 detection boxes, thereby enhancing the effect of the attack. In step 3, the loss function L obj By minimizing the confidence of the target detector output, the goal is to make the target detector unable to recognize the vehicle. The specific expression of the loss function is: Among them, V obj (i) represents the target confidence of the i-th detection box. By sorting the confidence of these detection boxes and minimizing the confidence of the first 1000 detection boxes, the detector will not be able to correctly identify the vehicle, thus achieving the purpose of the adversarial attack.

5. The method according to claim 1, wherein The particle swarm optimization algorithm (PSO) in step 4 dynamically adjusts the speed and position of the particles and combines the shape optimization module to iteratively optimize the position and shape of the patch to maximize the attack success rate (ASR). The particle swarm optimization algorithm is updated using the following formula: in, is the velocity of the i-th particle at time t+1, is the velocity of the i-th particle at time t, is the position of the iiith particle at time t, pbest i is the best historical position of the ith particle, gbest is the best position among all particles, w is the inertia weight, c1 and c2 are acceleration constants, and r1 and r2 are random numbers between [0,1]. During this process, the particle swarm optimization algorithm employs a dynamic adjustment mechanism, enabling it to optimize the search process as iterations progress. Specifically, the inertia weight w decreases with increasing iterations to balance global and local search capabilities. The acceleration constants c1 and c2 are also adjusted, with c1 decreasing and c2 increasing with increasing iterations. This encourages particles to explore a wider search space in the early stages and to focus more on local search in the later stages, thereby accelerating convergence. Through this dynamic adjustment mechanism, the PSO algorithm can effectively avoid falling into the local optimal solution too early, while improving the search efficiency of the global optimal solution, thereby maximizing the attack effect of the adversarial patch.

6. The method according to claim 1, characterized in that The expected transformation (EOT) technique in step 4 above enhances the robustness of the patch in the real environment by transforming the patch at multiple angles and scales. The EOT technique transforms the patch using the following formula: Among them, X adv is the generated adversarial sample, X adv_eot is the adversarial sample after the desired transformation, T is the transformation set, is the expectation operator, which represents the expected value of all possible transformations of the sample under the transformation distribution T. The expected transformation module simulates how patches in the physical world change under different angles, ambient lighting, and motion. It enhances the robustness of patches through multi-scale transformations and the addition of random noise. This approach ensures that adversarial examples remain effective in a variety of real-world situations, improving the adaptability of patches in dynamic environments. When the number of iterations iter reaches the preset maximum iter max or L obj When it is lower than the threshold, the algorithm terminates; the optimal adversarial patch is determined by the shape matrix and position set corresponding to the global optimal particle gbest, and the final patch template for physical pasting is output.

Images