A general transferable physical world optical adversarial attack method
The optical adversarial attack method optimized by adaptive graph generation and differential evolution algorithm solves the problems of insufficient universality and transferability of existing physical world attack methods, and realizes efficient, flexible and covert adversarial attacks in black box scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2026-01-19
- Publication Date
- 2026-05-29
Smart Images
Figure CN122116027A_ABST
Abstract
Description
Technical Field
[0001] This invention provides a general and transferable physical world optical adversarial attack method, which involves the implementation of an adversarial sample construction technology framework based on optical perturbation generation and optimization, belonging to the fields of artificial intelligence security and computer vision. Background Technology
[0002] With the rapid development of artificial intelligence, deep neural networks have been widely applied in key areas such as image recognition, autonomous driving, facial verification, and security surveillance. However, numerous studies have shown that deep neural network models are extremely sensitive to adversarial examples; even minute perturbations that are imperceptible to humans can lead to serious deviations in model output. This phenomenon not only threatens the robustness and reliability of AI systems but also raises widespread concerns in real-world scenarios involving personal and property safety. For example, misidentification of traffic signs could lead to dangerous decisions by autonomous driving systems; interference with facial recognition could directly jeopardize identity verification and information security.
[0003] In existing research on adversarial attacks, general adversarial perturbations and transferable attacks in digital environments have achieved some success. Generality refers to perturbations that are effective across different input samples; transferability means that the perturbation remains effective across different models and tasks. However, most of these studies are limited to virtual digital image spaces and are difficult to apply directly to real-world physical environments. Physical adversarial attacks targeting real-world scenarios are currently mainly divided into two categories: one is contact-based invasive methods, such as attacking by pasting special texture patches or stickers onto the surface of the target object; the other is non-contact optical methods, such as using shadows, light spots, or reflected light to interfere with imaging devices. While the former has some attack effect, its practical deployment suffers from problems such as cumbersome production, high deployment difficulty, and difficulty in rapid removal; the latter, while more flexible, mostly only works in individual instances and lacks generality and transferability.
[0004] Therefore, how to implement adversarial attacks that are universal, transferable, and deployable in the physical world has always been a pressing challenge in the field of AI security. Especially in black-box scenarios, attackers often cannot obtain the structure and parameters of the target model and can only rely on transferability to generate perturbations on a proxy model before projecting them onto the target system. This places higher demands on the effectiveness and universality of the perturbation design. While existing methods have explored approaches based on attention transfer and frequency domain enhancement, they still suffer from limitations such as insufficient universality, unstable attack effects, and complex physical deployment.
[0005] To address the aforementioned problems, this invention proposes a novel non-invasive optical-physical adversarial attack method. Unlike traditional patching, this method utilizes laser projection to directly apply complex textures generated by a special algorithm to the target object or background, thereby interfering with deep neural networks in the physical world. The main innovations of this method include: firstly, significantly improving the universality and cross-model, cross-domain transferability of the perturbation through an adaptive graphics generation algorithm and an irrelevant high-frequency feature enhancement strategy; secondly, obtaining efficient perturbations on small-sample proxy datasets using a differential evolution optimization algorithm, greatly reducing attack preparation costs; and thirdly, the deployment method based on optical projection is simple and flexible, enabling rapid projection and removal, and possessing strong concealment and practical application value. Summary of the Invention
[0006] (a) Purpose To address the lack of universality, portability, and deployment flexibility in existing physical attacks, this invention proposes a non-intrusive attack method based on optical perturbation. This method constructs complex and diverse optical patterns through an adaptive shape generation algorithm and combines local homogenization and global arraying strategies for irrelevant high-frequency feature enhancement, enabling the generated perturbations to possess strong universality and portability across models, domains, and tasks. In the optimization phase, a differential evolution algorithm is introduced as the core search mechanism, using the attack success rate as the fitness function, enabling efficient acquisition of optimal perturbation parameters in a black-box environment. This invention converts the optimized perturbation pattern into a vector file and projects it onto the target object or background area using a laser projector or diffractive optical element, achieving rapid, flexible, and non-contact attack deployment. This method is not only easy to operate in complex environments but also has the advantages of easy removal and strong concealment. It can be seen that this invention integrates the advantages of optical perturbation modeling, frequency domain feature enhancement, and evolutionary optimization, forming a physical world adversarial attack method that combines universality, portability, and practical application value.
[0007] (II) Technical Solution This invention provides a general and transferable physical-world optical adversarial attack method, the steps of which are as follows: Step 1: Build the surrogate model and select a small sample dataset as the training basis; Step 2: Define the optical adversarial patch parameters and generation space based on the adaptive graphics generation algorithm; Step 3: Introduce local and global perturbation repetition mechanisms to enhance irrelevant high-frequency features, thereby improving perturbation mobility; Step 4: Construct a general patch search framework based on differential evolution algorithm; Step 5: Use a density-aware fitness function to combine attack effectiveness and perturbation stealth, and evaluate candidate patches; Step 6: Use laser projection equipment to project adversarial graphics onto the target object to achieve a physical attack.
[0008] In step 1, “constructing the proxy model” involves preparing a black-box proxy model in a digital environment for generating and evaluating disturbances. This includes some classic convolutional neural network models used for classification.
[0009] In step 1, "selecting a small sample dataset" involves selecting a small sample dataset within a digital environment. This serves as the training basis for the surrogate model, used as a small sample training set. Among them, Indicates the input sample. For the corresponding tags, The number of samples in the dataset. The selection of the dataset follows a random sampling strategy, such as randomly selecting samples of different categories from a large dataset. This randomness reflects the effectiveness of this invention.
[0010] In step 2, "defining the parameters and generation space of the optical adversarial patch based on the adaptive graphics generation algorithm," this invention introduces an adaptive curve graphics generation algorithm to model the optical patch, avoiding the limitations imposed by manually setting the number of anchor points or polygon type in traditional methods. This algorithm can dynamically generate graphics patches with complex contours according to design requirements and can be flexibly adjusted according to the needs of different application scenarios. Formally, this invention defines a unified graphics generation operator G, which is based on a core contour function in polar coordinates. This maps the parameter set to a set of geometric points on a two-dimensional plane. We define the general parameter set as follows: Among them, the core contour function The topological structure of the graph is determined as follows:
[0011] Based on the above core outline, a unified graphics generation function is proposed. The mathematical expression for converting polar coordinates to a specific graph in the Cartesian coordinate system is as follows:
[0012] This function adjusts the geometry of a graphic by controlling different parameters. Control parameters Controlling symmetry determines where the patch is placed. The repetition period when the angle changes; Parameters and The parameters control the width and height of the shape, and their values are generally set to 1; The parameters control the overall size of the graphic; and the control parameters Further control over the sharpness and complexity of the shape. For example, larger... Values can result in sharper shapes, while smaller values can lead to sharper shapes. and This will make the graphic have a more rounded edge.
[0013] In this way, the present invention constructs a searchable generative space. This space contains all possible basic patch shapes. Specifically, the parameter set. By taking values within a predetermined range, different patch patterns can be generated. The fixed center position of the pattern. This refers to the size of the patch. To ensure diversity in patch shapes, the parameter set... Each dimension has a continuous range of values, allowing for the generation of various forms of optical patches through flexible adjustments.
[0014] This invention also introduces a density-aware differential evolution algorithm for generating space. The most suitable patch shape is selected. These patches will serve as candidate perturbations for subsequent attack experiments, ensuring the effectiveness of the generated perturbations on different models and datasets. Meanwhile, the definition of the generation space provides a mathematical basis for subsequent perturbation optimization, ensuring the adjustability and scalability of the perturbations.
[0015] Through this adaptive graphics generation algorithm, the present invention can design flexible and diverse optical adversarial patches, overcoming the limitations of fixed graphic shapes and structures in traditional methods, and providing a wider range of application possibilities for subsequent adversarial attacks.
[0016] In the "local and global perturbation repetition strategy" described in step 3, this invention combines the homogenization of local perturbations with the structured arrangement of global perturbations, enhancing the introduction of irrelevant high-frequency features and thus improving the mobility of the perturbations. Specifically, this method consists of two sub-methods: (a) Local perturbation repetition Recent studies have shown that adversarial perturbations with regional homogeneity can significantly enhance the transferability of adversarial examples. Since irrelevant high-frequency features introduced by a single shape within a local region are often inherently constrained, a proposal has been made to generate more complex locally irrelevant high-frequency features by superimposing multiple homogeneous patterns. To construct such locally homogeneous perturbation patterns, the original image is modeled as a combination of a set of homogeneous patterns that can induce irrelevant but learnable features that might be incorrectly prioritized by deep neural networks. Specifically, local regional homogeneity is achieved by superimposing multiple homogeneous patterns. In the physical world, such regionally homogeneous adversarial textures can be directly projected onto the environment via laser projection. This transformation can be formalized as follows:
[0017] in, This represents the generated local composite perturbation pattern. The shape function represents the generated shape, and the control parameters are... Controlling the symmetry and complexity of patches, It is the base radius. The fixed center position of the pattern. It is the number of repeated steps of the local perturbation. This is a continuous scaling factor for the radius. In this way, perturbation patterns within a local region can enhance irrelevant high-frequency features, thereby enhancing the transferability of adversarial examples.
[0018] (ii) Global perturbation repeatability Strategically repeating local perturbations globally can further amplify the influence of irrelevant features, thus strengthening their impact on the model's decision-making process. Therefore, this invention applies a global structured arrangement to the locally homogenized perturbations, amplifying irrelevant features on a larger scale and enhancing the transferability against perturbations. This global repetition transformation can be formalized as follows:
[0019] in, This represents the final perturbation pattern after global structured arrangement. and These are globally repeated indices, representing scaling factors in the horizontal and vertical directions, respectively. By globally structuring local perturbations, irrelevant high-frequency features can be significantly enhanced globally, thereby improving the transferability of perturbations across models and tasks.
[0020] This strategy of repeating local and global perturbations, through meticulous design, effectively enhances irrelevant high-frequency features at multiple scales, thereby improving the effectiveness and transferability of the perturbations and ensuring good attack performance across different tasks and models.
[0021] In step 4, the "differential evolution algorithm-based" method described in this invention employs an optimization method based on differential evolution to efficiently search for the optimal parameter configuration against perturbations. Since the attack vector is inherently non-differentiable, traditional gradient-based optimization methods cannot be directly applied. This invention systematically explores the parameter space using a differential evolution algorithm to identify the optimal shape control configuration, thereby achieving a higher general attack success rate. It consists of four basic components: (i) Population initialization. First, the population is initialized randomly. Multiple parameters are set for each individual, and each individual ; in, The complexity, symmetry, and shape characteristics of the patch are controlled separately.
[0022] Control the size of the pattern. Control the degree of local repetition. Control the degree of global repetition. Control the pattern color. Each parameter is limited to a predefined range.
[0023] (ii) Mutation operation Mutation operations generate new candidate solutions by introducing diversity, exploring unexplored regions in the search space. Given a parent individual The mutation operation randomly selects two other individuals from the current population. and And generate new offspring individuals according to the following formula. :
[0024] in, It is differential weighting. It is a restriction operator used to ensure that the generated individuals are within a predefined range.
[0025] (iii) Cross-operation Crossover facilitates information exchange between individuals, allowing beneficial traits from the parents to be passed on to the current population. Given two parent individuals... and Cross-operation uses a binomial crossover strategy to construct new offspring individuals. Specifically, for each dimension in the parameter vector, the value of that dimension is independently determined based on the crossover probability, whether it is inherited from the mutated vector or the target vector.
[0026] (iv) Select Operation To evaluate the effectiveness of each individual, a fitness function is used. Each individual in the population is evaluated, and individuals with high scores are selected for retention and participation in the next round of iterative optimization.
[0027] In this context, the "general patch search framework" described in step 4 refers to minimizing the loss function. A general-purpose patch is an iterative optimization process aimed at achieving the desired outcome. It refers to an adversarial perturbation that can be transferred across multiple models, tasks, or datasets. Unlike custom patches designed for a single model or task, general-purpose patches are designed to effectively disrupt the model's decision-making process in a variety of environments. Therefore, general-purpose patches are applicable across models, tasks, and even datasets.
[0028] In the "density-aware fitness function" described in step 5, the present invention introduces a fitness function. This is used to evaluate the attack effectiveness and perturbation stealth of each individual, ensuring that the optimization process not only focuses on the attack success rate but also controls the spatial density of the perturbation. The fitness function comprehensively considers both the attack success rate and the spatial distribution of the perturbation, and is specifically defined as follows:
[0029] in, Indicates the attack success rate, measuring the adversarial perturbation in the dataset. The effect on the surface. Specifically, Indicates via patch parameters The generated adversarial pattern and These represent the input sample and its corresponding label, respectively. The penalty term represents the physically perceptible intensity (or spatial density) of the perturbation pattern. This represents a penalty term for the physically perceptible intensity (or spatial density) of the perturbation pattern. Furthermore, This represents the spatial density of the perturbation, used to control the distribution of adversarial perturbations in the image. This term is quantized using the following formula:
[0030] in, It is an indicator function used to calculate the presence or absence of a disturbance. It's a sliding window. Indicates the opposing pattern in the sliding window The intensity of perturbations within the density. This density-aware fitness function can balance the aggressiveness and stealth of adversarial perturbations during the optimization process, ensuring that the generated patches have strong transferability across multiple models and tasks. This is the calculated spatial density value of the disturbance. The smaller the value, the more sparse and concealed the disturbance appears visually.
[0031] In step 6, which describes "projecting adversarial graphics onto a target object using a laser projection device to achieve a physical attack," the invention exports the final patch pattern parameter representation output by the algorithm into a vector format file recognizable by the optical projection device and configures the corresponding projection parameters. Specifically, this includes representing and saving the optimal graphic using vector graphics or a device-specific format to preserve geometric accuracy. Finally, the configured vector graphics are loaded into the laser projection control system and saved as a reusable deployment file for quick subsequent recall and removal.
[0032] Through the above steps, a general adversarial perturbation generation and deployment process can be completed, from digital agent training to physical optical projection: First, candidate perturbations are constructed and evaluated on the agent model and small sample data. Then, an adaptive curve graph generation method is used to obtain the basic graph. Further, irrelevant high-frequency features are enhanced through local and global arraying. Next, a differential evolution algorithm is used to search for the optimal graph in the parameter space and export it as a vector format. Finally, the configured vector graph is precisely projected onto the target area through a laser projection device to achieve adversarial effects in the physical world. This invention is applicable to adversarial testing and security assessment of vision-based deep learning systems such as image classification, object detection, and segmentation in physical scenarios. Furthermore, its non-intrusive deployment and rapid projection and removal capabilities give it high engineering value.
[0033] The advantages of this invention compared to existing technologies are as follows: Existing physical adversarial methods either rely on invasive patches or stickers, which are difficult to create and deploy and hard to remove quickly, or optical / lighting methods can only achieve case-by-case attacks and lack versatility and transferability. This method combines adaptive graphics generation with irrelevant high-frequency feature enhancement, thereby constructing perturbations with both versatility and transferability at the physical level. Secondly, this method significantly reduces the dependence on target model information and data scale by using differential evolution for black-box optimization on a small-sample proxy model. Finally, this method uses vectorized output and laser projection deployment, which ensures both the geometric accuracy and repeatability of the physical graphics, and achieves rapid deployment and removal with a certain degree of stealth. In summary, this invention achieves a balance between practicality, transferability, and deployability, becoming an effective technical means for evaluating and studying the robustness of deep models in real physical environments. Attached Figure Description
[0034] Figure 1 This is a flowchart illustrating the overall steps of the present invention.
[0035] Figure 2 This is an example diagram showing different parameters of the adaptive graphics module of the present invention.
[0036] Figure 3 This is a flowchart of the digital training process of the present invention.
[0037] Figure 4 This is a flowchart of the physical deployment process of the present invention.
[0038] Figure 5 This is a visualization of the interference of digital and physical factors on the model's attention in this invention. Detailed Implementation
[0039] To make the technical problems, technical solutions, and advantages of this invention clearer, a detailed description will be provided below in conjunction with the accompanying drawings.
[0040] This invention provides a general and transferable physical-world optical adversarial attack method. Using this method, efficient, transferable, and universal adversarial perturbations can be generated and projected into the physical world using laser projection technology, thereby achieving attacks on deep neural network systems. In this technique, firstly, an adaptive graph generation algorithm is used to determine the basic shape of the perturbation pattern, and a high-frequency independent feature enhancement strategy is employed to further improve the diversity and transferability of the perturbation. Then, a differential evolution algorithm is used to optimize the perturbation pattern, and black-box testing is used to evaluate the effectiveness of the optimized perturbation in different tasks and models. Finally, the optimized adversarial perturbation is projected onto a target object, and its attack success rate in the physical environment is evaluated. Through this process, this invention enables efficient cross-domain and cross-task adversarial attacks in different models, tasks, and physical environments, significantly improving the practicality and stealth of the attack.
[0041] This invention provides a general and transferable method for physical-world optical adversarial attacks, such as... Figure 1 As shown. The specific construction steps for targeting adversarial attacks on autonomous driving are as follows: Step 1: Construct an autonomous driving perception agent model and select a small sample dataset as the training basis. The purpose of this step is to build an autonomous driving perception agent model that can be used for black-box search and evaluation. With accompanying small sample training set ,in, Indicates the input sample. For the corresponding tags, The dataset contains a sample size that provides a fast, reproducible, and transferable evaluation benchmark for the discretization search of adversarial patch parameters. Specifically, relevant datasets such as GTSRB and CNN classification models for tasks like traffic light recognition and traffic sign recognition can be selected. Formally, let... Y represents the set of input samples (such as image data) in the target domain, and Y represents the corresponding set of true labels. Proxy model Mathematically represented as , that is, the class probability vector output after inputting image x; let the adversarial example after projection perturbation of a single input be:
[0042] in, Represents the original clean input image For parameterized optical patches, For the disturbance intensity, Ensure pixel intensity and physical feasibility. The training and sample selection of the surrogate model must ensure two basic properties: first, it should be able to demonstrate the model's sensitivity to high-frequency, regional perturbations with a small sample size; second, it should facilitate the transfer of patches optimized on the surrogate model to the unknown target model, which is also the core of the subsequent objective function.
[0043] In practical implementation, it is recommended to construct and select according to the following embodiments. Preferably, a small sample training set is used. Composed of a representative subset from the target application domain, the GTSRB subset that can be used is: selecting 10 categories, with 10 samples per category, for a total of The images were used as the basis for digital training; during evaluation, transfer testing was performed on a larger-scale TT-100K dataset and multiple fine-grained datasets to verify universality. The surrogate model was selected as one or more common convolutional backbones (e.g., ResNet-50 / ResNet-18 / DenseNet-121, etc.) to obtain representations sensitive to high-frequency texture perturbations by training or fine-tuning on small samples.
[0044] Step 2: Define the optical adversarial patch parameters and generation space based on the adaptive graphics generation algorithm.
[0045] The goal of this step is to formally characterize the geometric and color properties of the optical adversarial patch using a small number of controllable parameters, thereby transforming the design problem of the adversarial patch into a search problem in a low-dimensional parameter space, so that subsequent gradient-free optimization algorithms can run efficiently under controllable constraints.
[0046] (I) Introduction to Adaptive Graphics Generation Algorithms To avoid the subjectivity of traditional manual setting of anchor points or polygon shapes, this embodiment introduces an adaptive graphics generation algorithm based on hyperformations. This algorithm utilizes a small number of hyperparameters (such as...) It can describe a variety of natural geometric shapes, such as Figure 2 As shown, the number above each sub-figure represents the specific shape parameters used to generate that figure. This algorithm can flexibly generate a variety of different shapes, such as polygons, spirals, and leaf shapes. By adjusting these parameters, it can continuously generate diverse graphics ranging from simple to complex. Compared to manual shape design, this method has higher flexibility and generation efficiency, and can construct geometric units with rich textures within a finite-dimensional parameter space.
[0047] (II) Introduction to Optical Patch Modeling and Parameters In the modeling of optical countermeasures patches, a patch is defined as a perturbation controlled by structured parameters. Its key parameter set is as follows: Let the parameter vector be: , in, The shape control parameters used for adaptive graphics. Control the degree of local repetition. Control the degree of global repetition. These are color channel values, and each component is subject to the physical constraints of the laser projector and imaging system.
[0048] (III) Definition of Generative Space To ensure that the generated patch is both effective in attacking and physically feasible, this embodiment will generate a space. Defined as:
[0049] in, For the boundary of shape parameters, This is the boundary of the symmetry parameters. Within this space, each set of parameters... Mapping operator Generate the corresponding patch .
[0050] Within this space, each set of parameters Mapping operator Generate the corresponding patch Simultaneously, a projection operator is introduced. This ensures that the samples remain within the legal pixel range and comply with the physical limitations of the projection device (such as brightness and color gamut constraints) after the perturbation is superimposed. Furthermore, the perturbation intensity factor... As an additional control variable, it is used to adjust the transparency and visibility of the patch in the image. The final generated space not only covers rich shapes and textures, but also ensures that the patch has good mobility and physical deployability in the black-box environment through density and intensity constraints.
[0051] Step 3: Introduce a repetition mechanism to enhance irrelevant high-frequency features, thereby improving perturbation mobility.
[0052] The repetition of adversarial patches is a key technique in this method to improve transferability and physical robustness. Repetition mechanisms can be categorized into local and global repetition, which introduce irrelevant high-frequency structures at different scales and spatial ranges, respectively, thereby disrupting the model's low-level / mid-level feature extraction and promoting cross-model, cross-task, and cross-domain transferability. The visual effects of the repetition mechanism are... Figure 3 This is shown in the text.
[0053] (a) Local perturbation repetition Local repetition focuses on introducing dense, hierarchical textures and high-frequency features within a single local unit of the patch. This makes the local area a "high-frequency information source" in the local receptive field of the convolutional network, thereby preempting or contaminating the original semantic features and increasing the probability of misclassification across models. Let the basic form be... Local repetition is achieved by layering and enhancing the details of the basic shape to obtain local composite units:
[0054] in, This represents the generated local composite perturbation pattern; The shape function represents the generated shape, and the control parameters are... Controlling the symmetry and complexity of patches, The fixed center position of the pattern. A continuous scaling factor for the radius (which varies with the number of repetition steps q to achieve gradient scaling of local perturbations). In this way, perturbation patterns within local regions can enhance irrelevant high-frequency features, thereby enhancing the transferability of adversarial examples.
[0055] (ii) Global perturbation repeatability Global repetition, by arraying or tiling local complex units across the patch plane, causes high-frequency irrelevant features to appear in multiple locations across the entire image, forming consistent interference across the receptive field. This spatially repetitive structure allows features at different levels (shallow, mid-, and even deep) to access irrelevant repetitive details, improving coverage of different network architectures and the universality of attacks. In practical experiments, the effect of global repetition was evaluated using different layouts from 1×1 to 5×5, and it was found that 4×4 achieves a good trade-off between ASR and SSIM. This process can be formally transformed into:
[0056] Where j and k are globally repeated indices ( Corresponding to the horizontal direction, The corresponding vertical direction has a value range of [value]. (These are the maximum number of permutations in the horizontal and vertical directions, respectively), controlling the perturbation permutation density in the horizontal and vertical directions; respectively representing the perturbation density in the horizontal and vertical directions. and Scaling factor in dimensions. By globally structuring local perturbations, irrelevant high-frequency features can be significantly enhanced globally, thereby improving the transferability of perturbations across models and tasks. Indicates the horizontal direction. The arrangement positions coordinate, Indicates the vertical direction of the first The arrangement positions coordinate; It is the number of repetition steps of the local perturbation ( The range of values is (The threshold for the local maximum number of repetition steps within a single global permutation unit). A continuous scaling factor for the radius (in the local perturbation repeatability) (Consistent definition).
[0057] This strategy of repeating local and global perturbations, through meticulous design, effectively enhances irrelevant high-frequency features at multiple scales, thereby improving the effectiveness and transferability of the perturbations and ensuring good attack performance across different tasks and models.
[0058] Step 4: Construct the differential evolution algorithm and initialize the population individuals.
[0059] The goal of this build is to generate mappings in the parameterized patch. Above, a gradient-free differential evolution algorithm is used in a low-dimensional parameter space. The search identified a universal optical patch that boasts a high attack success rate across multiple models, tasks, and physical environments while satisfying physical projectibility constraints. The overall optimization problem is formalized as minimizing the fitness function. :
[0060] This represents the physically perceptible intensity penalty term for the perturbation pattern; subsequently, a gradient-free optimization algorithm framework is used to optimize the problem. The following operations are performed in the initialization phase and operator design: (a) Initial population First, the population is initialized randomly. Multiple parameters are set for each individual, and each individual ; in, The complexity, symmetry, and shape characteristics of the patch are controlled separately. Control the size of the pattern. Control the degree of repetition. and Controls the position and color of the pattern. Each parameter is constrained within a predefined range. Population size. A value of 30 is recommended, but should be adjusted based on search budget and parallel processing capabilities. Each parameter is initialized by random, uniform sampling within its predefined range.
[0061] (ii) Mutation operation Population size It is recommended to set the value to between 30 and 40, adjusting based on search budget and parallel processing capabilities. Each parameter is randomly and uniformly sampled within its predefined range to initialize the mutation operation. This introduces diversity to generate new candidate solutions and explores unexplored regions in the search space. Given a parent individual... The mutation operation randomly selects two other individuals from the current population. and And generate new offspring individuals according to the following formula. :
[0062] in, It is differential weighting. It is a restriction operator used to ensure that the generated individuals are within a predefined range. The recommended value is 0.5.
[0063] (iii) Cross-operation Crossover facilitates information exchange between individuals, allowing beneficial traits from the parents to be passed on to the current population. Given two parent individuals... and Crossover operation randomly from or Select each parameter to construct a new offspring individual .
[0064]
[0065] Where CR is the crossover probability. To ensure that at least one dimension comes from the random index of the mutation vector, this invention can continuously generate complex and diverse candidate patch shapes through the above crossover and mutation processes for subsequent fitness function selection. The parameter represents the i-th individual (target vector) in the g-th generation population. Represents the j-th individual in the same generation. The generated random number is used to determine whether to perform cross-replacement in the current dimension. (iv) Select Operation To evaluate the effectiveness of each individual, a fitness function is used. Each individual in the population is evaluated, and individuals with high scores are selected for retention and participation in the next round of iterative optimization.
[0066] This step presents an engineerable differential evolution search framework: from individual encoding, population design, mutation / crossover / selection implementation, to model validation, the process is as follows: Figure 3 As shown, the aim is to obtain a universal patch with strong transferability and physical projectability under realistic projection conditions. Key implementation details and ablation values (such as λ=1e-2, α=0.5, 4×4 layout) can be directly used for the embodiments or experimental reproduction in the specification.
[0067] Step 5: Use a density-aware fitness function to combine attack effectiveness and perturbation stealth to evaluate candidate patches.
[0068] The goal of this step is to construct a fitness function that simultaneously reflects attack effectiveness and perturbation stealth, enabling differential evolution black-box search to find an acceptable compromise between these two conflicting objectives. To prevent the pursuit of high adversarial attack success rates from resulting in unprojectable or extremely conspicuous patterns, local over-aggregation or overall density exceeding the limit should be explicitly penalized within the fitness range. This invention introduces a fitness function... This is used to evaluate the attack effectiveness and perturbation stealth of each individual, ensuring that the optimization process not only focuses on the attack success rate but also controls the spatial density of the perturbation. The fitness function comprehensively considers both the attack success rate and the spatial distribution of the perturbation, and is specifically defined as follows:
[0069] in, Indicates the attack success rate, measuring the adversarial perturbation in the dataset. The effect on the surface. Specifically, Indicates via patch parameters The generated adversarial pattern and These represent the input sample and its corresponding label, respectively.
[0070] also, This represents the spatial density of the perturbation, used to control the distribution of adversarial perturbations in the image. This term is quantized using the following formula:
[0071] in, It is an indicator function used to calculate the presence or absence of a disturbance. It's a sliding window. The opposing pattern is displayed in the window. The intensity of perturbations within the density. This density-aware fitness function can balance the aggressiveness and stealth of adversarial perturbations during the optimization process, ensuring that the generated patches have strong transferability across multiple models and tasks.
[0072] Step 6: Use laser projection equipment to project adversarial graphics onto the target object to achieve a physical attack.
[0073] After optimizing the parameters of the adversarial patch and generating a vectorized perturbation pattern, it is imported into a laser projector or diffractive optical element. The perturbation pattern is then directly projected onto the surface of the target object or its surrounding background area via an optical system. The laser projection device uses a high-brightness beam as a carrier to superimpose the designed geometric shape and color information onto the target recognition scene of the autonomous driving system. This causes the camera or sensor to simultaneously receive the object's own light reflection and the perturbation projection during imaging, resulting in strong interference to the deep neural network model during the feature extraction stage, ultimately leading to incorrect recognition results. Since this method does not require contact with the object's surface, the attack process is highly covert and flexible. The projection position and angle can be adjusted in real time according to the environment to ensure the perturbation remains effective under different viewing angles and lighting conditions. In the experiment, this invention used a standard laser projector to complete the projection operation, such as... Figure 4 As shown, the object under test is a canvas bag. By clearly superimposing the optimized perturbation pattern onto the surface of the target object, the classifier can be misclassified as a torch with a confidence level of 0.63. At the same time, the attacking device maintains a certain distance from the object to achieve stable optical interference. Figure 5 This is a visualization of the digital and physical interference with the model's attention as presented in this invention. The figure contains 10 sets of comparative examples, each set consisting of four sub-figures, from top to bottom: the original input image, the image after applying adversarial perturbation, the attention heatmap of the original image in the model, and the heatmap of the model's attention after applying perturbation.
[0074] Through the above steps, a general adversarial perturbation generation and deployment process can be completed, from digital agent training to physical optical projection: First, candidate perturbations are constructed and evaluated on the agent model and small sample data. Then, an adaptive curve graph generation method is used to obtain the basic graph. Further, irrelevant high-frequency features are enhanced through local homogenization and global arraying. Next, the optimal graph is searched in the parameter space using a differential evolution algorithm and exported as a vector format. Finally, the configured vector graph is precisely projected onto the target area using a laser projection device to achieve adversarial effects in the physical world. This invention is applicable to adversarial testing and security assessment of vision-based deep learning systems such as image classification, object detection, and segmentation in physical scenarios. Furthermore, its non-intrusive deployment and rapid projection and removal capabilities give it high engineering value.
[0075] The parts of this invention not described in detail are well-known in the field.
[0076] The above description is only a part of the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.
Claims
1. A universal and transferable physical-world optical adversarial attack method, characterized in that, The steps include the following: Step 1: Build the surrogate model and select a small sample dataset as the training basis; Step 2: Define the optical adversarial patch parameters and generation space based on the adaptive graphics generation algorithm; Step 3: Introduce local and global perturbation repetition mechanisms to enhance irrelevant high-frequency features, thereby improving perturbation mobility; Step 4: Construct a general patch search framework based on differential evolution algorithm; Step 5: Use a density-aware fitness function to combine attack effectiveness and perturbation stealth, and evaluate candidate patches; Step 6: Use laser projection equipment to project adversarial graphics onto the target object to achieve a physical attack.
2. The universal and transferable physical-world optical countermeasure attack method according to claim 1, characterized in that: In step 1, a black-box surrogate model for generating and evaluating perturbations is prepared in a digital environment, including a convolutional neural network model for classification; specifically, a small sample dataset is selected in the digital environment. It serves as the training basis for the surrogate model and is used as a small sample training set; among which, Indicates the input sample. For the corresponding tags, The number of samples in the dataset; the selection of the dataset follows a random sampling strategy.
3. The universal and transferable physical-world optical countermeasure attack method according to claim 1, characterized in that: In step 2, an adaptive curve generation algorithm is introduced to model the optical patch. A unified curve generation operator G is defined, which is based on the core contour function in polar coordinates. This maps the parameter set to a set of geometric points on a two-dimensional plane; the core contour function The topological structure of the graph is determined as follows: ； Based on the core outline, a unified graphics generation function is implemented. The mathematical expression for converting polar coordinates to a specific graph in the Cartesian coordinate system is: ； The geometric shape of the graphic can be adjusted by controlling different parameters; control parameters Controlling symmetry determines where the patch is placed. Repetition period during angle change; parameters and parameters Control the width and height of the shape separately; parameters Control the overall size of the graphic; control parameters Further control over the sharpness and complexity of the shape.
4. The universal and transferable physical-world optical countermeasure attack method according to claim 1, characterized in that: Step 3 further includes local perturbation repeatability; Uniformity in local areas is achieved by superimposing multiple homogeneous patterns; in the physical world, this uniform adversarial texture is directly projected onto the environment via laser projection; this transformation can be formalized as follows: ； in, This represents the generated local composite perturbation pattern; The shape function represents the generated shape, and the control parameters are... Controlling the symmetry and complexity of patches, The fixed center position of the pattern. The continuous scaling factor for the radius varies with the number of repetition steps q, achieving gradient scaling of local perturbations.
5. A universal and transferable physical-world optical adversarial attack method according to claim 1 or 4, characterized in that: Step 3 further includes global perturbation repeatability; By applying a global structured arrangement to the perturbation of local homogenization, this global repetitive transformation is formalized as follows: ； in, This represents the final perturbation pattern after global structured arrangement; j and k are the indices of global repetition. Corresponding to the horizontal direction, The corresponding vertical direction has a value range of [value]. The maximum number of permutations in the horizontal and vertical directions are thresholds, which control the perturbation permutation density in the horizontal and vertical directions, respectively. Indicates the horizontal direction. The arrangement positions coordinate, Indicates the vertical direction of the first The arrangement positions coordinate; It is the number of repeated steps of the local perturbation. The range of values is This is the threshold for the maximum number of local repetition steps within a single global permutation unit. is a continuous scaling factor for the radius.
6. The universal and transferable physical-world optical countermeasure attack method according to claim 1, characterized in that: In step 4, the parameter space is systematically explored using a differential evolution algorithm to identify the optimal shape control configuration; this includes: Population initialization: First, the population is initialized randomly. Multiple parameters are set for each individual, and each individual ; in, The complexity, symmetry, and shape characteristics of the patch are controlled separately. ; Control the size of the pattern. Control the degree of local repetition. Control the degree of global repetition. Control the pattern color; each parameter is limited to a predefined range; Mutation operation: Mutation operations generate new candidate solutions by introducing diversity, exploring unexplored regions in the search space; given a parent individual The mutation operation randomly selects two other individuals from the current population. and And generate new offspring individuals according to the following formula. : ； in, It is differential weighting. It is a restriction operator used to ensure that the generated individuals are within a predefined range; Cross operation: Crossover facilitates information exchange between individuals, allowing beneficial traits from the parent generation to be passed on to the current population; given two parent individuals... and Cross-operation uses a binomial crossover strategy to construct new offspring individuals. Specifically, for each dimension in the parameter vector, the value of that dimension is independently determined based on the crossover probability, whether it is inherited from the mutated vector or from the target vector. Select operation: To evaluate the effectiveness of each individual, a fitness function is used. Each individual in the population is evaluated, and individuals with high scores are selected for retention and participation in the next round of iterative optimization.
7. A universal and transferable physical-world optical adversarial attack method according to claim 6, characterized in that: In step 4, the general patch search framework refers to minimizing the loss function. The iterative optimization process with the objective of [missing information]; general patch refers to adversarial perturbations that are migrated between multiple models, tasks, or datasets to ensure that they can effectively disrupt the model's decision-making process in various different environments.
8. A universal and transferable physical-world optical countermeasure attack method according to claim 1, characterized in that: In step 5, a fitness function is introduced. This is used to evaluate the attack effectiveness and perturbation stealth of each individual; the fitness function comprehensively considers the attack success rate and the spatial distribution of the perturbation, and is specifically defined as follows: ； in, Indicates the attack success rate, measuring the adversarial perturbation in the dataset. The effect on; specifically, Indicates via patch parameters The generated adversarial pattern and These represent the input sample and its corresponding label, respectively. The physically perceptible intensity penalty term for the perturbation pattern; This represents the physically perceptible intensity penalty term for the perturbation pattern; furthermore... This represents the spatial density of the perturbation, used to control the distribution of adversarial perturbations in the image.
9. A universal and transferable physical-world optical countermeasure attack method according to claim 8, characterized in that: Quantification is performed using the following formula: ； in, It is an indicator function used to calculate the presence or absence of a disturbance. It's a sliding window. Indicates the opposing pattern in the sliding window The intensity of the disturbance within; This is the calculated spatial density value of the disturbance. The smaller the value, the more sparse and concealed the disturbance appears visually.
10. A universal and transferable physical-world optical adversarial attack method according to claim 1, characterized in that: In step 6, the patch pattern parameters are exported as a vector format file that can be recognized by the optical projection device and the corresponding projection parameters are configured. Specifically, the optimal graphic is represented and saved using vector graphics or a device-specific format to preserve geometric accuracy. Finally, the configured vector graphics are loaded into the laser projection control system and saved as a reusable deployment file for quick recall and removal in the future.