Anti-attack patch generation method based on particle swarm optimization

Abstract

The invention provides a particle swarm optimization-based adversarial attack patch generation method, which comprises the following steps of: dividing a target region into a plurality of sub-regions by adopting a multi-patch collaborative strategy, deploying an independent adversarial patch in each sub-region, and efficiently searching optimal patch configuration in a parameter space by utilizing a particle swarm optimization algorithm. A fitness function is defined to evaluate a reduction effect of the patch on the confidence coefficient of the target detector, and iterative optimization of an adversarial sample is realized; that is to say, by optimizing the shape, position and color parameters of the patch, the physical confrontation sample which is more hidden, higher in attack success rate and capable of effectively cheating the target detector can be generated, and the method is a novel security attack method for the computer vision system.

Description

Technical Field

[0001] This invention belongs to the field of computer vision security technology, and in particular relates to a method for generating adversarial attack patches based on particle swarm optimization. Background Technology

[0002] With the rapid development of deep learning technology, artificial intelligence-based systems have made breakthroughs in fields such as computer vision, natural language processing, and autonomous driving. However, the concept of adversarial examples, proposed by Szegedy et al. in 2014, revealed a serious flaw in deep learning models: by adding carefully designed small perturbations to the input data, the model can be caused to output incorrect results with high confidence. This discovery has sparked widespread attention in academia regarding the security of deep learning models, and adversarial attack techniques have gradually become a research hotspot.

[0003] Adversarial attack techniques are mainly divided into two categories: white-box attacks and black-box attacks. White-box attacks assume that the attacker has complete knowledge of the target model's structure and parameters, and can directly calculate gradients to generate adversarial examples; while black-box attacks are more in line with real-world scenarios, where the attacker can only infer the model's behavior through input-output queries. In the field of computer vision, adversarial attacks can be further divided into two forms: digital attacks and physical attacks. The former directly adds perturbations to digital images, while the latter needs to consider various constraints in transforming adversarial examples from the digital domain to the physical world.

[0004] Adversarial patching is a special form of physical adversarial attack that interferes with a model's recognition ability by adding specific patterns to the surface of a target object. Early research on adversarial patching focused primarily on the digital domain, optimizing algorithms to generate patch patterns capable of fooling classifiers. As research progressed, researchers began to focus on how to effectively apply these digital patches to the physical world, which requires considering various practical factors such as limitations of printing equipment, changes in lighting, and shifts in viewing angle.

[0005] Traditional adversarial patch generation methods typically rely on optimization algorithms such as gradient descent. While effective, these methods suffer from slow convergence and susceptibility to local optima. Particularly when dealing with high-dimensional parameter spaces, traditional optimization methods often struggle to find the global optimum, resulting in limited attack effectiveness of the generated patches. Furthermore, patches generated by existing methods often lack stealth and are easily detected by human observers, limiting their application in real-world scenarios.

[0006] Particle Swarm Optimization (PSO) is a swarm intelligence optimization algorithm inspired by the foraging behavior of birds. It has advantages such as few parameters, simple implementation, and fast convergence speed. Compared with evolutionary computation methods such as genetic algorithms, PSO does not require complex crossover and mutation operations. Instead, it guides the search direction through information sharing and individual experience among particles, making it particularly suitable for solving optimization problems in continuous spaces.

[0007] In recent years, the PSO algorithm has begun to be applied in the field of adversarial attacks. For example, some studies have combined PSO with chaotic mapping to generate text adversarial examples, solving the problem that traditional methods are prone to getting trapped in local optima. In the image domain, PSO has also been used to optimize the generation process of adversarial examples, and its swarm intelligence characteristics help to find effective attack patterns in complex high-dimensional spaces. However, most existing PSO-based adversarial attack methods focus on the digital domain, and research on physical adversarial patch generation is still relatively lacking.

[0008] Current adversarial patch generation techniques face several key challenges: First, the patch generation process typically requires significant computational resources, especially when multiple physical transformations need to be considered, making the optimization process exceptionally time-consuming; second, the conversion from the digital domain to the physical domain leads to a decrease in patch effectiveness, and existing methods struggle to effectively compensate for this performance loss; third, most methods generate patches that lack stealth and are easily identifiable by human observers, limiting their practical application value; finally, existing optimization algorithms perform poorly in balancing attack effectiveness and patch naturalness, often requiring compromises between these two objectives. Summary of the Invention

[0009] To address the aforementioned issues, this invention provides an adversarial patch generation method based on particle swarm optimization. This method utilizes an improved PSO algorithm to efficiently search a multi-dimensional parameter space containing patch shape, color, and position, while also considering physical constraints to generate adversarial patches that are both highly aggressive and maintain a natural appearance. This invention not only improves patch generation efficiency but also ensures the effectiveness of patches in the physical world through a unique fitness function design.

[0010] A method for generating adversarial attack patches based on particle swarm optimization includes the following steps:

[0011] S1: Under the set configuration parameters, set the initial value for the patch parameter quadruple of each particle in the particle swarm. The configuration parameters include the number of patches and the center coordinates, width, and height of the target area. The patch parameter quadruple includes the hue, saturation, brightness, and contrast of the patch.

[0012] S2: Apply the adversarial patch generated based on the patch parameter quadruple of each particle to the target region to obtain the current iteration adversarial image for each particle.

[0013] S3: Obtain the fitness value of each particle in the current iteration based on the adversarial image of each current iteration, and determine whether the current iteration meets the set convergence condition. If yes, complete the patch generation; otherwise, proceed to step S4.

[0014] S4: Compare the fitness values ​​of each particle in the current iteration and the previous iteration, and take the patch parameter quadruple corresponding to the larger value as the preferred patch parameter quadruple for each particle.

[0015] S5: Update the optimal patch parameter quadruple for each particle in the current particle swarm using the set rules, and then return to step S2 to enter the next iteration using the updated patch parameter quadruple.

[0016] Furthermore, the convergence condition is set as one of the following:

[0017] The current iteration count has reached the maximum iteration count;

[0018] The cumulative iteration time up to the current iteration has reached the time limit;

[0019] The maximum fitness value across all iterations up to the current iteration is greater than a set threshold.

[0020] The change in the patch parameters of the particle swarm in the current iteration relative to the patch parameters of the particle swarm in the previous iteration is less than a preset threshold.

[0021] Furthermore, the fitness value of any particle is calculated as follows:

[0022] fitness = 1 - confidence

[0023] Here, confidence is the detection confidence when using a target detector to detect targets in an adversarial image generated based on the patch parameters corresponding to the current particle.

[0024] Furthermore, the method for updating the patch parameters of any particle using predefined rules is as follows:

[0025]

[0026] in, This represents the update rate of particle i on the d-th patch parameter at the current iteration t. Let w represent the update velocity of particle i at the d-th patch parameter in the next iteration t+1, w represent the inertia weight, c1 represent the individual learning factor, c2 represent the social learning factor, r1 and r2 represent random numbers uniformly distributed in [0,1], and p represent the update velocity of particle i at the d-th patch parameter in the next iteration t+1. id p represents the value of the d-th patch parameter in the patch parameter quadruple corresponding to the maximum fitness value of particle i across all iterations. gd This represents the value of the d-th patch parameter in the patch parameter quadruple corresponding to the maximum fitness value of all particles in the entire particle swarm across all iterations. This represents the value of particle i in the d-th patch parameter at the current iteration t. This represents the value of the current particle i in the d-th patch parameter at the next iteration t+1.

[0027] Furthermore, after determining whether the current iteration satisfies the set convergence condition, the patch parameter corresponding to the particle characterization of the maximum fitness value in all iterations up to the current iteration is used as the patch parameter for generating the optimal patch.

[0028] Furthermore, after generating the optimal patch, bilinear interpolation is used to adjust the optimal patch to the size of the target area, and then the adjusted optimal patch is pasted onto the surface of the target area to achieve a physical adversarial attack.

[0029] Furthermore, the number of patches generated for the target region is 4, and the boundary constraint matrix of the configuration parameters and patch parameters is as follows:

[0030]

[0031] In this boundary constraint matrix, the four elements in the first column are the x-coordinate and y-coordinate of the center of the target region, the width of the target region, and the height of the target region, respectively. The last four columns of the boundary constraint matrix correspond to a patch, and the four elements in the last four columns are the hue, saturation, brightness, and contrast of the patch, respectively. Different rows of the boundary constraint matrix represent the patch parameters under different configuration parameters.

[0032] Furthermore, when the target area is larger than a set value, the target area is divided into multiple sub-regions, and independent adversarial patches are generated for each sub-region based on particle swarm optimization.

[0033] Beneficial effects:

[0034] This invention provides a method for generating adversarial attack patches based on particle swarm optimization. The invention employs a multi-patch collaborative strategy, dividing the target region into multiple sub-regions and deploying independent adversarial patches in each sub-region. It utilizes the particle swarm optimization algorithm to efficiently search for the optimal patch configuration in the parameter space and evaluates the patch's effect on reducing the target detector's confidence by defining a fitness function, thus achieving iterative optimization of the adversarial sample. In other words, by optimizing the shape, position, and color parameters of the patches, this invention can generate more covert, higher-success-rate, and more effective physical adversarial samples that deceive the target detector, representing a novel attack method for the security of computer vision systems. Attached Figure Description

[0035] Figure 1 A flowchart illustrating a method for generating adversarial attack patches based on particle swarm optimization, provided by this invention.

[0036] Figure 2 This is a schematic diagram illustrating the optimization iteration of the adversarial attack patch based on particle swarm optimization provided by the present invention.

[0037] Figure 3 A schematic diagram of the output convergence curve of an adversarial attack patch generation method based on particle swarm optimization provided by the present invention;

[0038] Figure 4 Images showing the patched application of this invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, a method for generating adversarial attack patches based on particle swarm optimization includes the following steps:

[0041] S1: Under the set configuration parameters, set the initial value for the patch parameter quadruple of each particle in the particle swarm. The configuration parameters include the number of patches and the center coordinates, width, and height of the target area. The patch parameter quadruple includes the hue, saturation, brightness, and contrast of the patch.

[0042] S2: Apply the adversarial patch generated based on the patch parameter quadruple of each particle to the target region to obtain the current iteration adversarial image for each particle.

[0043] S3: Obtain the fitness value of each particle in the current iteration based on the adversarial image of each current iteration, and determine whether the current iteration meets the set convergence condition. If yes, complete the patch generation; otherwise, proceed to step S4.

[0044] The convergence condition is set as one of the following:

[0045] The current iteration count has reached the maximum iteration count;

[0046] The cumulative iteration time up to the current iteration has reached the time limit;

[0047] The maximum fitness value across all iterations up to the current iteration is greater than a set threshold.

[0048] The change in the patch parameters of the particle swarm in the current iteration relative to the patch parameters of the particle swarm in the previous iteration is less than a preset threshold.

[0049] After determining whether the current iteration meets the set convergence condition, the patch parameters of the particle characterization corresponding to the maximum fitness value in all iterations up to the current iteration are used as the patch parameters for generating the optimal patch.

[0050] After generating the optimal patch, bilinear interpolation is used to adjust the optimal patch to the size of the target area, and then the adjusted optimal patch is pasted onto the surface of the target area to achieve a physical adversarial attack.

[0051] S4: Compare the fitness values ​​of each particle in the current iteration and the previous iteration, and take the patch parameter quadruple corresponding to the larger value as the preferred patch parameter quadruple for each particle.

[0052] The fitness value of any particle is calculated as follows:

[0053] fitness = 1 - confidence

[0054] Here, confidence is the detection confidence when using a target detector to detect targets in an adversarial image generated based on the patch parameters corresponding to the current particle.

[0055] S5: Update the optimal patch parameter quadruple for each particle in the current particle swarm using the set rules, and then return to step S2 to enter the next iteration using the updated patch parameter quadruple.

[0056] Furthermore, the method for updating the patch parameters of any particle using predefined rules is as follows:

[0057]

[0058] in, This represents the update rate of particle i on the d-th patch parameter at the current iteration t. Let w represent the update velocity of particle i at the d-th patch parameter in the next iteration t+1, w represent the inertia weight, c1 represent the individual learning factor, c2 represent the social learning factor, r1 and r2 represent random numbers uniformly distributed in [0,1], and p represent the update velocity of particle i at the d-th patch parameter in the next iteration t+1. id p represents the value of the d-th patch parameter in the patch parameter quadruple corresponding to the maximum fitness value of particle i across all iterations. gd This represents the value of the d-th patch parameter in the patch parameter quadruple corresponding to the maximum fitness value of all particles in the entire particle swarm across all iterations. This represents the value of particle i in the d-th patch parameter at the current iteration t. This represents the value of the current particle i in the d-th patch parameter at the next iteration t+1.

[0059] The following example, using the generation of four patches for the target region, further illustrates the particle swarm optimization-based adversarial attack patch generation method provided by this invention. Figure 2 As shown, the specific steps include:

[0060] S1: Define the function apply_patch_to_image to apply the generated adversarial patch to the target image. It supports the following functions:

[0061] (1) Check if the input image file exists, read the image and convert it to tensor format;

[0062] (2) The position and size of the patch can be dynamically adjusted to adapt to different target areas;

[0063] (3) Supports the direct application of pre-generated patches or random patch generation;

[0064] (4) The positional relationship of the four patches can make them form a larger rectangular patch.

[0065] S2: Define the `generate_random_patch` function to generate random colors and size-appropriate patches, which supports the following features:

[0066] (1) Generate a patch based on the input color parameters (hue, saturation, brightness, contrast);

[0067] (2) If no color parameter is provided, a color will be generated randomly;

[0068] (3) The patch size can be dynamically adjusted according to the target area to ensure that the patch matches the target area.

[0069] S3: Implement the particle swarm optimization algorithm, iteratively optimizing the patch parameters, as detailed below:

[0070] S31: Define the target detection region parameters and patch color parameters as optimization variables to construct the search space of the particle swarm algorithm;

[0071] S32: Initialize the particle swarm. Each particle represents a set of patch configuration parameters, including the center coordinates (x, y) of the target region, width (w), height (h), and color parameters (hue, saturation, brightness, contrast) of multiple sub-patches. Its boundary constraints are:

[0072]

[0073] In this boundary constraint matrix, the four elements in the first column are the x-coordinate and y-coordinate of the center of the target region, the width of the target region, and the height of the target region, respectively. The last four columns of the boundary constraint matrix correspond to a patch, and the four elements in the last four columns are the hue, saturation, brightness, and contrast of the patch, respectively. Different rows of the boundary constraint matrix represent the patch parameters under different configuration parameters.

[0074] S4: The main program performs PSO optimization, extracts optimization parameters, and generates the final adversarial sample.

[0075] For each particle, a corresponding rectangular adversarial patch is generated based on its parameters and applied to the test image. The processed image is then inspected using a target detector, with the detection confidence level used as feedback to calculate the particle's fitness value. In other words, this invention updates the particle's position and velocity using a particle swarm optimization algorithm, compares the fitness values ​​of individual particles before and after the changes, and compares the fitness values ​​of the entire particle swarm before and after the changes, preserving both the individual optimal and the global optimal solutions. Figure 3 The figure shows the output convergence curve of an adversarial attack patch generation method based on particle swarm optimization provided by the present invention.

[0076] It should be noted that when the target area is larger than a set value, the target area is divided into multiple sub-regions, and independent adversarial patches are generated for each sub-region based on particle swarm optimization. The specific process of patch generation includes the following steps:

[0077] Step 1: Calculate the center coordinates and dimensions of the four sub-regions based on the target region parameters;

[0078] Step 2: Generate an independent rectangular patch for each sub-region; the patch color is determined by the corresponding color parameter.

[0079] Step 3: Use bilinear interpolation to adjust the patch to the target size;

[0080] Step 4: Apply the patch to the specified location on the original image;

[0081] Step 5: Based on the optimized patch parameters, use a printer to print the image with the retained patch on white paper at an appropriate scale for the physical world.

[0082] Step 6: Paste the patch onto the object surface according to the optimized position parameters to achieve a physical adversarial attack, such as... Figure 4 The image shown is the result after applying the patch.

[0083] Therefore, compared with some existing physical adversarial attacks, the adversarial attack patch generation method based on particle swarm optimization provided by this invention significantly improves stealth and practicality while ensuring attack effectiveness by dynamically optimizing the position, size, and color parameters of multiple adversarial patches. Compared with traditional adversarial attacks, this invention innovatively adopts a multi-patch collaborative interference strategy, enabling patches to adaptively blend with the background and accurately cover key target areas, while strictly adhering to physical implementation constraints (such as minimum size limits and natural color ranges), ensuring that the generated adversarial samples effectively interfere with target detection systems and can be covertly deployed in real-world scenarios. This design, which combines intelligent optimization algorithms with physical attack requirements, allows adversarial patches to maintain a high attack success rate even in complex environments, providing a more efficient and reliable solution for generating physical adversarial samples.

[0084] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A method for generating adversarial attack patches based on particle swarm optimization, characterized in that, Includes the following steps: S1: Under the set configuration parameters, set the initial value for the patch parameter quadruple of each particle in the particle swarm. The configuration parameters include the number of patches and the center coordinates, width, and height of the target area. The patch parameter quadruple includes the hue, saturation, brightness, and contrast of the patch. S2: Apply the adversarial patch generated based on the patch parameter quadruple of each particle to the target region to obtain the current iteration adversarial image for each particle. S3: Obtain the fitness value of each particle in the current iteration based on the adversarial image of each current iteration, and determine whether the current iteration meets the set convergence condition. If yes, complete the patch generation; otherwise, proceed to step S4. S4: Compare the fitness values ​​of each particle in the current iteration and the previous iteration, and take the patch parameter quadruple corresponding to the larger value as the preferred patch parameter quadruple for each particle. S5: Update the optimal patch parameter quadruple for each particle in the current particle swarm using the set rules, and then return to step S2 to enter the next iteration using the updated patch parameter quadruple.

2. The method for generating adversarial attack patches based on particle swarm optimization as described in claim 1, characterized in that, Set one of the following convergence criteria: The current iteration count has reached the maximum iteration count; The cumulative iteration time up to the current iteration has reached the time limit; The maximum fitness value across all iterations up to the current iteration is greater than a set threshold. The change in the patch parameters of the particle swarm in the current iteration relative to the patch parameters of the particle swarm in the previous iteration is less than a preset threshold.

3. The method for generating adversarial attack patches based on particle swarm optimization as described in claim 1, characterized in that, The fitness value of any particle is calculated as follows: fitness = 1 - confidence Here, confidence is the detection confidence when using a target detector to detect targets in an adversarial image generated based on the patch parameters corresponding to the current particle.

4. The method for generating adversarial attack patches based on particle swarm optimization as described in claim 1, characterized in that, The method for updating the patch parameters of any particle using predefined rules is as follows: in, This represents the update rate of particle i on the d-th patch parameter at the current iteration t. Let w represent the update velocity of particle i at the d-th patch parameter in the next iteration t+1, w represent the inertia weight, c1 represent the individual learning factor, c2 represent the social learning factor, r1 and r2 represent random numbers uniformly distributed in [0,1], and p represent the update velocity of particle i at the d-th patch parameter in the next iteration t+1. id p represents the value of the d-th patch parameter in the patch parameter quadruple corresponding to the maximum fitness value of particle i across all iterations. gd This represents the value of the d-th patch parameter in the patch parameter quadruple corresponding to the maximum fitness value of all particles in the entire particle swarm across all iterations. This represents the value of particle i in the d-th patch parameter at the current iteration t. This represents the value of the current particle i in the d-th patch parameter at the next iteration t+1.

5. The method for generating adversarial attack patches based on particle swarm optimization as described in claim 1, characterized in that, After determining whether the current iteration meets the set convergence condition, the patch parameters of the particle characterization corresponding to the maximum fitness value in all iterations up to the current iteration are used as the patch parameters for generating the optimal patch.

6. The method for generating adversarial attack patches based on particle swarm optimization as described in claim 5, characterized in that, After generating the optimal patch, bilinear interpolation is used to adjust the optimal patch to the size of the target area, and then the adjusted optimal patch is pasted onto the surface of the target area to achieve a physical adversarial attack.

7. The method for generating adversarial attack patches based on particle swarm optimization as described in claim 1, characterized in that, Four patches are generated for the target region, and the boundary constraint matrix of the configuration parameters and patch parameters is as follows: In this boundary constraint matrix, the four elements in the first column are the x-coordinate and y-coordinate of the center of the target region, the width of the target region, and the height of the target region, respectively. The last four columns of the boundary constraint matrix correspond to a patch, and the four elements in the last four columns are the hue, saturation, brightness, and contrast of the patch, respectively. Different rows of the boundary constraint matrix represent the patch parameters under different configuration parameters.

8. The method for generating adversarial attack patches based on particle swarm optimization as described in claim 1, characterized in that, When the target area is larger than the set value, the target area is divided into multiple sub-regions, and independent adversarial patches are generated for each sub-region based on particle swarm optimization.

Images