Black box sparse adversarial attack method based on constraint multi-modal multi-target resonance optimization

Abstract

The invention relates to the technical field of intelligent algorithm evaluation and optimization and image confrontation attack, in particular to a black box sparse confrontation attack method based on constraint multi-modal multi-target resonance optimization, which comprises the following steps: determining an input image and an image classifier to be confronted, and initializing disturbance optimization parameters; the fitness of constrained multi-modal multi-objective optimization is determined; generating a parent population according to the input image and disturbance optimization parameters; performing clustering, sorting, screening, crossing and variation on a current parent population to generate a child population; when the number of the adversarial samples is greater than a preset number, clustering and redundancy elimination are carried out on disturbance modes of the offspring individuals; sorting union sets of the parent population and the child population, and updating the current parent population; when the maximum query frequency in the disturbance optimization parameters is reached, generating a final confrontation sample based on the disturbance mode of the individuals in the current offspring population; according to the invention, high-quality adversarial samples can be generated.

Description

Technical Field

[0001] This invention relates to the fields of intelligent algorithm evaluation and optimization as well as image adversarial attack technology, specifically to a black-box sparse adversarial attack method based on constrained multimodal multi-target resonance optimization. Background Technology

[0002] In recent years, deep learning technology has made groundbreaking progress in the application of computer vision. At the same time, the security of intelligent systems integrating deep learning has received increasing attention. Research has found that these systems are susceptible to being deceived by carefully designed adversarial examples, leading to erroneous decisions. To test the reliability of intelligent systems used for image classification tasks under image adversarial attack scenarios, various image adversarial attack methods have been proposed. Based on the amount of information that can be obtained from the attacked model, image adversarial attacks can be divided into white-box and black-box attacks. Compared to white-box attacks, which require complete internal information about the model, black-box attacks, which only require the model's output, are closer to real-world application scenarios and therefore have higher feasibility and research value. However, the high-dimensional image pixel search space not only brings computational cost challenges to black-box attacks but also makes attack algorithms prone to getting trapped in local optima, reducing the diversity of generated perturbation patterns and affecting the attack success rate. Therefore, designing an advanced and efficient perturbation generation method to address these problems has become the primary challenge in improving the performance of black-box attacks.

[0003] Among numerous black-box image attack methods, black-box sparse adversarial attacks, due to their modification of only a very small number of pixels in the image, show great potential in improving perturbation concealment and reducing attack costs. However, the core challenge of this method lies in how to effectively balance the degree of perturbation with the attack success rate under sparsity constraints. Constrained multi-objective optimization (CMO) strategies offer a solution, seeking the optimal balance between perturbation and attack effectiveness through CMO algorithms. However, during optimization, CMO algorithms typically focus only on finding a trade-off solution in the target space, neglecting the diversity of perturbation patterns in the decision variable space. Current mainstream models often employ adversarial training, gradient masking, and defensive distillation to enhance the anti-interference capabilities of intelligent systems. The lack of diversity in perturbation patterns limits the effectiveness of black-box image sparse adversarial attack methods based on CMO.

[0004] Therefore, how to provide a black-box image sparse adversarial attack algorithm that can balance the success rate of the attack and the degree of perturbation, while ensuring the diversity of perturbation patterns, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present invention provides a black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization, which solves the technical problems of weak diversity and concealment of image adversarial examples generated in the prior art.

[0006] This invention provides a black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization, comprising the following steps: Step S1: Determine the input image and the image classifier to be challenged, and initialize the perturbation optimization parameters; Determining the fitness of constrained multimodal multi-objective optimization includes: establishing the fitness based on the classification loss of the image classifier, the size of the perturbation, the sparsity of the perturbation, and whether the adversarial mechanism is successful; Step S2: Generate a parent population based on the input image and perturbation optimization parameters. The parent population contains multiple individuals, each representing a perturbation mode; calculate the fitness of each individual. Step S3: The current parent population is processed using a topological incremental clustering algorithm based on adaptive resonance theory, and then the current parent population is sorted according to the constraint dominance rule, crowding distance and individual fitness. Step S4: Filter, crossover, and mutate the current parent population to generate offspring population, and acquire and accumulate individuals belonging to adversarial examples in the offspring population; when the number of individuals belonging to adversarial examples is greater than the preset number, cluster and de-redundancy processing is performed on the perturbation patterns of individuals belonging to adversarial examples. Step S5: Sort the union of the parent and offspring populations according to the constraint dominance rules, crowding distance, and individual fitness, and update the current parent population according to the sorting results. Step S6: Return to step S3 until the maximum number of queries in the perturbation optimization parameters is reached, and generate the final adversarial sample based on the perturbation pattern of individuals in the current offspring population.

[0007] Preferably, in step S1, the perturbation optimization parameters include the maximum number of perturbation pixels, the color channel perturbation amplitude, and the maximum number of queries; The fitness includes an optimization objective and constraints. The optimization objective is established based on the classification loss of the image classifier and the magnitude of the perturbation. The constraints are established based on the sparsity of the perturbation and whether the adversarial mechanism is successful. The expressions for the optimization objective and constraints are as follows:

[0008]

[0009] in, Indicates minimization. Indicating a response to the disturbance The two objective functions, Represents the classification loss function. This represents the trained image classifier for adversarial training. Indicates the input image. Indicates disturbance of Norm; Indicates constraints. The dimension is The real number field, These represent the width, height, and number of channels of the input image, respectively. Indicates disturbance of Norm, Indicates the maximum number of disturbed pixels. This indicates the category containing the highest confidence score. For category indexing, The total number of categories, This indicates that the image classifier will classify the scrambled image into the first... Confidence level of each category Indicates the true category.

[0010] Preferably, in step S2, the step of generating the parent population specifically includes: Randomly select from all pixels of the input image. A set of non-repeating pixel positions are used as perturbation pixel positions; for each perturbation pixel position, three color channel perturbation values ​​are generated, with the perturbation value for each color channel starting from... The perturbation pattern is formed by randomly selecting the perturbation pixel position and the perturbation value of the color channel of each pixel. The perturbation pattern is recorded as an individual, and multiple individuals constitute the parent population. The steps for calculating the fitness of each individual specifically include: overlaying the perturbation pattern of the individual onto the original image, processing the pixel values ​​using a cropping function to generate perturbation samples; inputting the perturbation samples into the image classifier to be challenged to obtain the predicted confidence scores for each category, and calculating the fitness based on the predicted confidence scores for each category.

[0011] Preferably, step S3 specifically includes: Step S3-1: Perform cluster initialization, winning node selection, warning value testing, and node learning processing on the current parent population in sequence to obtain the neighborhood of each individual; Step S3-2: Sort the parent population according to the constraint dominance rule and stratify the individuals in the neighborhood of each individual, and then sort the individuals in the same layer according to the crowding distance.

[0012] Preferably, step S3-1 specifically includes: Step S3-1-1, Clustering Initialization: Input individuals as data points into the topology adaptive resonance network, and set the first two data points input into the topology adaptive resonance network as two nodes of the topology network respectively; Step S3-1-2, Winning Node Selection: Whenever a new data point is added... The data is input into the topology network, and the current data point is compared with the existing set of nodes. CIM between nodes in the middle; The two nodes with the lowest CIM values ​​among the current data points are determined as the first and second winning nodes. Step S3-1-3, Warning Value Test: The CIM between the input data point and the first and second winning nodes are respectively and ; when At that time, the input data points are generated as new nodes in the topological adaptive resonant network; when At that time, update the number of data points that match the first winning node; when When the first winning node is reached, update the number of data points that match the first winning node, and update the neighbor nodes of the first winning node; Step S3-1-4, Node Learning: Update the positions of all winning nodes; Step S3-1-5: Return to step S3-1-2 until all individuals in the parent population have been processed. Determine the subpopulation to which each individual belongs based on the final topology network. Combine all individuals in the subpopulation to which the individual belongs, as well as all individuals in the subpopulations adjacent to the subpopulations to which all individuals belong, to form the neighborhood of the current individual.

[0013] Preferably, in step S3-1-3, the expression for updating the neighbor nodes of the first winning node is:

[0014] in, This represents the neighboring nodes of the first winning node. This indicates the number of data points contained in the neighboring node; In step S3-1-4, the expression for updating the positions of all winning nodes is:

[0015] in express The number of data points contained in a node.

[0016] Preferably, step S3-2 specifically includes: Step S3-2-1: Determine the constraint dominance rules for a given individual. and An individual is considered to be an individual if any of the following conditions are met. Constraint and Domination of Individuals : (1) The perturbated image is an adversarial example. The perturbated image is not an adversarial example; (2) and The perturbated images are not adversarial examples, and < That is, classifier pair The confidence level of the perturbed image belonging to the true category is less than [a certain value]. The confidence level that the disturbed image belongs to the true category; (3) and The perturbated images are all adversarial samples, and < ,Right now The Euclidean distance between the perturbed image and the original image is less than 1. The Euclidean distance between the perturbed image and the original image; Step S3-2-2: Sort the parent population according to the constraint domination rule, including: dividing all individuals that are not dominated by any other individual into the first layer, then finding those that are not dominated from the remaining individuals and putting them into the second layer, and so on, until all individuals are stratified, thus completing the sorting. Step S3-2-3: For individuals in the same layer, calculate the crowding distance of the individuals, and sort them according to the requirement that individuals with larger crowding distances are ranked higher within the same layer; The expression for calculating congestion distance is:

[0017] in, Represents an individual Crowded distance, Indicates the dimension of the target space. and They represent the first time. The target space sorts the individuals in the current layer in ascending order of their target values, and then assigns adjacent individuals to the target space. The target values ​​of adjacent individuals on both sides, and They represent the first The maximum and minimum values ​​of the solutions that exist in the target space.

[0018] Preferably, step S4 specifically includes: Step S4-1: Select individuals from the first half of the current parent population. For each individual, use probability... Perform a crossover operation between the current individual and the top-ranked individual in its neighborhood, with probability... Randomly select an individual from the population that is different from the current individual and perform a crossover operation to obtain offspring individuals; Step S4-2: Perform mutation operations on the offspring individuals, including randomly replacing the pixel positions of the perturbation patterns of the offspring individuals and the corresponding channel perturbation values, to form an offspring population.

[0019] Step S4-3: Obtain and accumulate offspring individuals belonging to adversarial examples. When the number of individuals belonging to adversarial examples is greater than the preset number, cluster and de-redundancy processing is performed on the perturbation patterns of individuals belonging to adversarial examples.

[0020] Preferably, in step S4-3, the step of clustering the perturbation patterns belonging to adversarial sample individuals includes: When the number of adversarial examples exceeds the preset number When the perturbation pixel coordinates of all adversarial samples are input into the topological incremental clustering algorithm based on adaptive resonance theory for processing, clustering is completed. The perturbation redundancy removal process specifically includes the following steps: Set the color channel perturbation value of the pixel position in the cluster containing interference pixels less than the preset threshold to [0,0,0]. Then input the individual with the adjusted perturbation mode into the classifier for evaluation. If it is still an adversarial example, replace the current individual with the individual with the adjusted perturbation mode.

[0021] Preferably, step S5 specifically includes: The union of the parent and offspring populations is sorted according to the constraint dominance rule, crowding distance, and individual fitness. When there are no adversarial examples in the sorting results, multiple individual sets are generated. The best individual is selected from each individual set without repetition and added to the new population until the number of individuals in the new population reaches the total number of individuals in the parent population. When there are adversarial examples in the ranking results, multiple adversarial example sets are generated according to the category to which the adversarial examples belong, and a non-adversarial example set is generated. Individuals are first selected from the adversarial example set without repetition to the new population, and then individuals are selected from the non-adversarial example set without repetition to the new population, until the number of individuals in the parent population reaches the total number of individuals in the parent population. Finally, the parent population is updated to the new population.

[0022] Compared with the prior art, the present invention has at least the following beneficial effects: (1) This invention models the black-box image adversarial attack problem as a constrained multimodal multi-objective optimization problem. The multi-objective optimization aims to achieve two objectives simultaneously: first, to minimize the loss function between the predicted class and the true class of the perturbed image; and second, to minimize the difference between the perturbed image and the original image. Multimodality refers to multiple equivalent solutions that satisfy the above objectives, i.e., combinations of perturbed pixel positions and color channel perturbation values. The constraint condition is that the candidate solution must be an adversarial example and the number of perturbed pixels must be less than or equal to a set value. Unlike constrained multi-objective optimization problems that only focus on the diversity of solutions in the target space, constrained multimodal multi-objective optimization problems, while pursuing the diversity of solutions in the target space, also ensure the diversity of perturbation modes in the input space, making the attack more difficult to defend against.

[0023] (2) This invention addresses the constrained multimodal multi-objective optimization problem in black-box sparse adversarial attacks by designing a constrained multimodal multi-objective genetic algorithm based on incremental clustering to solve the optimization problem. The perturbation pattern of the original image is encoded into individuals in the genetic algorithm. Unlike traditional constrained multi-objective optimization algorithms, this algorithm maintains the diversity of perturbation patterns through a crossover strategy based on multi-population neighborhood search, mutation operators, and a diversity maintenance strategy based on subpopulation mapping. It also uses constraint dominance rules and crowding distance sorting to ensure fast convergence, thereby improving the success rate and efficiency of the attack.

[0024] (3) This invention uses a pixel-dimensional incremental clustering algorithm to discover pixel perturbations with low attack contribution in different adversarial samples. It uses a dynamic perturbation elimination strategy to remove redundant perturbations in each round of algorithm iteration. Unlike the perturbation degree control method that relies solely on norm constraints, this strategy can significantly reduce the number of perturbation pixels in adversarial samples, further reduce the difference between the perturbated image and the original image, and improve the quality of adversarial samples.

[0025] (4) This invention designs incremental clustering strategies at two different levels: macroscopic and microscopic. At the macroscopic level of population management, incremental clustering is used to efficiently and dynamically divide the constantly evolving perturbation patterns into subpopulations. This provides an adaptive structural basis for subsequent neighborhood search and diversity preservation strategies, thereby ensuring the diversity of perturbation patterns. At the microscopic level of perturbation sparsity, incremental clustering is used to analyze the perturbation pixel positions in adversarial examples at the pixel dimension. As new adversarial examples are generated, the existing clusters are continuously updated, thereby accurately and efficiently discovering redundant pixels with low attack contribution, providing accurate information for dynamic perturbation elimination strategies, and significantly improving the sparsity and concealment of perturbations. Attached Figure Description

[0026] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0027] Figure 1The flowchart shows the overall steps of the black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization provided by this invention.

[0028] Figure 2 A detailed flowchart of the black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization provided by this invention.

[0029] Figure 3 The flowchart illustrates the construction of subpopulations using the topological incremental clustering algorithm based on adaptive resonance theory provided by this invention.

[0030] Figure 4 This is a schematic diagram of adversarial samples generated by the black-box sparse adversarial attack method based on constrained multi-objective optimization provided by the present invention.

[0031] Figure 5 This is another schematic diagram of the adversarial sample generated by the black-box sparse adversarial attack method based on constrained multi-objective optimization provided by the present invention. Detailed Implementation

[0032] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0033] This invention proposes a black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization. The black-box image adversarial attack problem is modeled as a constrained multimodal multi-objective optimization problem. This problem effectively balances multiple objective functions in the target space while ensuring the diversity of input combinations in the decision space. The process of finding various perturbation combinations of pixel positions and color channel values ​​to simultaneously optimize the reduction of model confidence and the minimization of image perturbation is modeled as a constrained multimodal multi-objective optimization problem. This optimization problem is solved using a constrained multimodal multi-objective genetic algorithm based on incremental clustering. The perturbation patterns of the original image are encoded into individuals in the genetic algorithm. Through multi-population neighborhood search and diversity preservation strategies, diverse perturbation patterns are generated. Furthermore, a pixel-level dynamic perturbation elimination strategy removes redundant perturbations. During the evolutionary iteration process, an effective attack on the classifier is achieved, while generating high-quality adversarial examples with diversity and strong concealment, thus improving the reliability and security of the intelligent system.

[0034] To illustrate the effectiveness of the method proposed in this invention, the following detailed description of the above technical solution is provided through a specific embodiment, such as... Figure 1 , Figure 2 As shown, this invention discloses a black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization. The specific implementation steps are as follows: Step S1: Determine the input image and the image classifier to be challenged, and initialize the perturbation optimization parameters; Determining the fitness of multimodal multi-objective optimization includes: establishing the fitness based on the classification loss of the image classifier, the size of the perturbation, the sparsity of the perturbation, and whether the adversarial mechanism is successful; The fitness includes an optimization objective and constraints. The optimization objective is established based on the classification loss of the image classifier and the magnitude of the perturbation. The constraints are established based on the sparsity of the perturbation and whether the adversarial mechanism is successful. This invention first identifies the target of a black-box sparse adversarial attack based on multimodal, multi-target resonance optimization. The attack involves superimposing interfering pixels onto the input original image, then inputting the scrambled image into an image classifier to be attacked. The image classifier outputs the class confidence score of the scrambled image. The perturbation pattern represents the pattern of superimposing interfering pixels into the input image. This invention optimizes the perturbation pattern of the scrambled original image and models the optimization process.

[0035] In this step, the present invention first obtains the input image and the image classifier to be challenged.

[0036] In some embodiments, the input image may be an image from a publicly available dataset, such as the ImageNet dataset, or a self-collected natural image; the adversarial image classifier may be an existing deep network for image classification, such as a Deep Residual Network (ResNet) model.

[0037] In this step, the invention determines the initial values ​​of the perturbation optimization parameters for optimizing the perturbation mode. Specifically, the perturbation optimization parameters include: the maximum number of perturbation pixels, the color channel perturbation amplitude, and the maximum number of queries.

[0038] Maximum number of perturbation pixels This represents the maximum number of interfering pixels superimposed on the original image; the color channel perturbation magnitude represents... The extent of color channel overlay modification, and the maximum number of queries. This indicates the maximum number of query iterations required for perturbation optimization.

[0039] In some embodiments, the maximum number of perturbed pixels It can be 0 of the total number of pixels in the input image. . 30%, for example, when the input image is an image from the ImageNet dataset, with a total of 50176 pixels, then the maximum number of perturbed pixels is... The value is 150. Considering the desired adversarial effect and the perturbation norm constraint, the color channel perturbation amplitude can be determined. The value is 1, which represents the maximum number of queries. Q The value can be set to 5000.

[0040] By setting the above parameters, this invention determines the modeling and initial requirements for the subsequent optimization process.

[0041] This invention models the black-box sparse adversarial attack problem as a constrained multimodal multi-objective optimization problem. This invention determines that the fitness of multimodal multi-objective optimization includes two objective functions and two constraints, which are described in detail below.

[0042] The two objective functions are minimizing the model's confidence in the true class and minimizing image perturbation. Finding multiple combinations of perturbation pixel locations and color channel perturbation values ​​that achieve both objectives is considered a constrained multimodal multi-objective optimization problem.

[0043] The two constraints are: the number of disturbed pixels cannot exceed the maximum number of disturbed pixels. Furthermore, the classifier's classification of the perturbed image does not match the image's true category.

[0044] The expressions for the two objective functions and the two constraints are as follows:

[0045]

[0046] in, Indicates minimization. Indicating a response to the disturbance The two objective functions, Represents the classification loss function. This represents the trained image classifier for adversarial training. Indicates the input image. Indicates disturbance of Norm; Indicates constraints. The dimension is The real number field, These represent the width, height, and number of channels of the input image, respectively. Indicates disturbance of Norm, Indicates the maximum number of disturbed pixels. This indicates the category containing the highest confidence score. For category indexing, The total number of categories, This indicates that the image classifier will classify the scrambled image into the first... Confidence level of each category Indicates the true category.

[0047] As shown in the above formulas, for the optimization objective, this invention uses the boundary loss function to calculate the difference between the confidence scores of the perturbed image and the most likely category other than the true category, which serves as the first optimization objective. A smaller objective value indicates that the model is more likely to misclassify the perturbed image. The Euclidean distance between the perturbed image and the original image is used as the second optimization objective. A smaller objective value indicates that the pixel distribution of the perturbed image deviates less from that of the original image. Regarding the constraints, this invention defines the first constraint of the optimization problem as the number of perturbed pixels not exceeding the maximum number of perturbed pixels; and the second constraint is that the classifier's classification of the perturbed image should be inconsistent with the image's true category. When the second constraint is satisfied, the perturbed image can be called an adversarial example.

[0048] Through the above steps, this invention determines the fitness of multimodal multi-objective optimization, including the optimization objective and constraints. The fitness of multi-objective optimization can be applied to subsequent optimization steps to evaluate the effectiveness of the perturbation mode.

[0049] Step S2: Generate a parent population based on the input image and perturbation optimization parameters. The parent population contains multiple individuals, each representing a perturbation mode; calculate the fitness of each individual. The perturbation pattern refers to the pattern of superimposing interfering pixels in the input image; In this step, the step of generating the parent population based on the input image and perturbation optimization parameters specifically includes: for a total number of pixels of The image, from all pixels Random selection The non-repeating pixel positions are used as the perturbation pixel positions, denoted as . For each selected pixel position Generate three color channel perturbation values, with the perturbation value for each color channel starting from... The perturbation pattern is formed by randomly selecting the perturbation pixel position and the color channel perturbation value of each pixel, and can be regarded as an individual of the parent population in the genetic algorithm. ,Depend on Each individual constitutes the initial paternal population. .

[0050] For each individual in the population, the perturbation pattern is superimposed on the original image, and the pixel values ​​are restricted to a specified range using a cropping function to generate a perturbation sample. Then, the sample is input into the image classifier to be challenged to obtain the confidence of each category. The fitness of each perturbation pattern is calculated based on the classification loss, perturbation size, perturbation sparsity, and whether the adversarial process is successful.

[0051] The clipping function expression is:

[0052] in, This represents the clipping function. This represents the image after adding the perturbation mode. This represents the minimum value within the valid interval. This represents the maximum value within the valid range. When pixel values ​​are normalized... and The values ​​are 0 and 1 respectively.

[0053] Step S3: The current parent population is processed using a topological incremental clustering algorithm based on adaptive resonance theory, and then the current parent population is sorted according to the constraint dominance rule, crowding distance and individual fitness. This includes: obtaining the neighborhood of each individual; sorting the individuals in the initial population according to the constraint dominance rule, crowding distance, and individual fitness, and sorting the individuals within the neighborhood of each individual; In this step, the present invention first processes the individual inputs from the parent population using a topological incremental clustering algorithm based on adaptive resonance theory, adaptively generating several clusters, i.e., multiple partitioned populations, to characterize the distribution of candidate solutions in the decision space. Then, the adjacency relationship of the partitioned populations is determined based on similarity, and the neighborhood of each individual is obtained, with multiple individuals in each individual's neighborhood. The parent population is sorted according to the constraint dominance rule, and the individuals within each individual's neighborhood are stratified. Then, the crowding distance is used to sort the individuals within the same stratum, thereby maximizing population diversity while retaining high-quality solutions. The details are described below.

[0054] Step S3-1: Perform cluster initialization, winning node selection, warning value testing, and node learning processing on the current parent population in sequence to obtain the neighborhood of each individual; Incremental clustering is an unsupervised learning method that can group new data points in real time without reprocessing the entire dataset. Unlike traditional clustering algorithms that require loading all data at once, incremental clustering is particularly suitable for handling continuously generated data streams. Adaptive resonance theory is one of the incremental learning methods inspired by the learning behavior of the human brain. In the learning process of an adaptive resonance network, the activation states between neurons interact and adjust until a stable resonance state is reached. The topological incremental clustering algorithm based on adaptive resonance theory considers how the system can adapt to new situations by updating existing knowledge while maintaining stable memory of old knowledge. It can solve the stability and plasticity dilemma. The introduction of topological structure can learn the connectivity relationships between data clusters, so that the clustering results can reflect the true neighborhood relationships between data, rather than just discovering a few isolated groups, thus exhibiting good incremental clustering performance.

[0055] This step takes the current parent population P containing N individuals as input and processes it using a topological incremental clustering algorithm based on adaptive resonance theory to obtain a structured population. This process treats each individual as a node and groups them in the decision space. The algorithm proceeds through the following steps (1) to (4), as follows: Figure 3 As shown, individuals in the population are processed one by one, clusters are dynamically created and adjusted to form subpopulations, and connections between them are established.

[0056] (1) Clustering initialization: Individuals are input as data points into the topological adaptive resonant network. For the first two data points input into the topological adaptive resonant network, they are automatically made into two nodes of the topological network. The kernel function bandwidth between nodes is determined. The kernel function bandwidth is calculated using kernel density estimation, and the expression is:

[0057] in, This represents the kernel function bandwidth, which includes all dimensions. For dimensional indexing, It was before The first input data Standard deviation of the dimension.

[0058] Will The median is determined as the inter-node kernel bandwidth. .

[0059] (2) Winning node selection: Whenever a new data point is added The data is input into the topology network, and the current data point is compared with the existing set of nodes. The similarity between nodes in the dataset is measured using the Correlation Entropy Induced Metric (CIM). The expression for CIM is:

[0060]

[0061]

[0062] in, This represents the inductive measure of related entropy. Indicates the input data points. Indicates the first An existing node Indicates the kernel bandwidth between nodes. Indicates the total number of data points. Represent the input data points and the first... The first existing node Dimension value, and These represent the indices of the first and second winning nodes, respectively. This indicates that the minimum value has been obtained. This represents an existing set of nodes. Indicates the first winning node. This represents the set of nodes remaining after removing the first node.

[0063] Will The two nodes with the lowest CIM value (i.e., the highest similarity) to the current data point are determined as the winning nodes.

[0064] (3) Warning value test: In this step, it is determined whether the CIM between the input data point and the first and second nodes exceeds the warning value. The similarity between the input data point and the first and second nodes is expressed as follows:

[0065]

[0066] The results of the alert value test can be divided into the following three categories: Scenario I: When When the matching degree between the two winning nodes and the input data points exceeds a predefined warning threshold, a new node will be generated in the topology network. The node position is consistent with the input data point.

[0067] Situation II: When When, it means the first winning node With input data points The matching degree is within the warning value, while the second winning node With input data points The matching degree is not within the warning value, at which point it is necessary to update the matching degree. Number of matching data points .

[0068] Situation III: When At this point, the matching degree between the two winning nodes and the input data points is within the warning value, and it is also necessary to update the... Number of matching data points .

[0069] (4) Node learning: After passing the alert value test, the winning node is updated in the following way. Location:

[0070] in express The number of data points contained in a node, when the winning node meets condition III, in addition to updating the node. outside, Node's neighboring nodes It will also be updated through the following methods:

[0071] in, This represents the neighboring nodes of the first winning node. This indicates the number of data points contained in the neighboring node.

[0072] (5) Return to step (2) until all individuals in the parent population are completed. Determine the subpopulation to which each individual belongs based on the final topology network. Combine all individuals in the subpopulation to which the individual belongs, as well as all individuals in all subpopulations adjacent to that subpopulation, to form the neighborhood of the current individual.

[0073] Step S3-2: Sort the parent population according to the constraint dominance rule and stratify the individuals in the neighborhood of each individual. Then, sort the individuals in the same stratum using the crowding distance.

[0074] In this step, the present invention first determines the constraint dominating rule for a given individual. and An individual is considered to be an individual if any of the following conditions are met. Constraint and Domination of Individuals : (1) The perturbated image is an adversarial example. The perturbated image is not an adversarial example; (2) and The perturbated images are not adversarial examples, and < That is, classifier pair The confidence level of the perturbed image belonging to the true category is less than [a certain value]. The confidence level that the disturbed image belongs to the true category; (3) and The perturbated images are all adversarial samples, and < ,Right now The Euclidean distance between the perturbed image and the original image is less than 1. The Euclidean distance between the perturbed image and the original image.

[0075] The parent population is sorted according to the constraint dominance rule in the above manner. This includes: according to the constraint dominance rule, all individuals that are not dominated by any other individual are assigned to the first layer. Then, from the remaining individuals, those that are not dominated are placed into the second layer, and so on, until all individuals are stratified, thus completing the sorting.

[0076] When individuals do not dominate each other, they are assigned to the same non-dominated stratum. For individuals in the same non-dominated stratum, the crowding distance is used to assess their distribution within the non-dominated stratum. The expression for calculating the crowding distance is as follows:

[0077] in, Represents an individual Crowded distance, Indicates the dimension of the target space. and They represent the first time. The target space sorts the individuals in the current layer in ascending order of their target values, and then assigns adjacent individuals to the target space. The target values ​​of adjacent individuals on both sides, and They represent the first The maximum and minimum values ​​of the solutions that exist in the target space.

[0078] Through the above steps, individuals within the same non-dominated layer were identified. The crowding distance is used to rank individuals within the same non-dominated layer, based on the requirement that individuals with larger crowding distances rank higher.

[0079] Step S4: Filter, crossover, and mutate the current parent population to generate offspring population, and acquire and accumulate individuals belonging to adversarial examples in the offspring population; when the number of individuals belonging to adversarial examples is greater than the preset number, cluster and de-redundancy processing is performed on the perturbation patterns of individuals belonging to adversarial examples. In this step, the first half of the individuals are selected from the sorted population. Then, each selected parent individual has a high probability of crossover with the best individual in its neighborhood, or with other random individuals in the population, generating new pixel perturbation combinations. Subsequently, the new individuals generated by the crossover are mutated, that is, a small number of pixel positions and their perturbation values ​​are randomly changed to introduce new possibilities. Through these operations, a batch of new perturbation patterns (offspring) are generated, and they are also applied to the original image. The classifier is then queried to calculate their fitness, as described in detail below.

[0080] Step S4-1: Select individuals from the first half of the current parent population. For each individual, use probability... Perform a crossover operation between the current individual and the top-ranked individual in its neighborhood, with probability... Randomly select an individual from the population that is different from the current individual and perform a crossover operation to obtain offspring individuals; Specifically, since the perturbation pixel position is a set of pixel positions Since the random integers in the solution are used, a crossover operator is designed based on the idea of ​​discrete crossover. By exchanging some pixels and the corresponding channel perturbation values ​​of the two parent solutions, two new offspring individuals are generated. The pixel positions involved in the exchange cannot be those that already exist in the other parent solution. The number of pixels crossed is determined by the number of non-overlapping pixel positions in the two parent solutions and the crossover probability.

[0081] Step S4-2: Perform mutation operations on the offspring individuals, including randomly replacing the pixel positions of the perturbation patterns of the offspring individuals and the corresponding channel perturbation values, to form an offspring population.

[0082] Specifically, the new pixel location generated by the mutation cannot be a pixel location that already exists in the individual, and the number of mutated pixels... Determined by the maximum number of perturbed pixels and variability They jointly decided to introduce new perturbation patterns to candidate solutions in this way, thereby promoting the diversity of perturbation samples.

[0083] Step S4-3: Obtain and accumulate offspring individuals belonging to adversarial examples. When the number of individuals belonging to adversarial examples is greater than the preset number, cluster and de-redundancy processing is performed on the perturbation patterns of individuals belonging to adversarial examples.

[0084] In this step, the step of clustering the perturbation patterns of offspring individuals specifically includes: When the number of adversarial examples exceeds the preset number In this case, the coordinates of each perturbation pixel in all adversarial samples are input into the topological incremental clustering algorithm based on adaptive resonance theory, and the similarity index is used to cluster pixels with close positions into the same cluster.

[0085] The perturbation redundancy removal process specifically includes the following steps: If a cluster contains more interfering pixels, it means that almost all adversarial examples have changed the color channel value of the pixel at that location. This indicates that the attacked classifier is more sensitive to changes in pixels near that cluster, and the pixels in that cluster contribute more to the attack effect. On the other hand, if a cluster contains fewer interfering pixels, it means that the attack may succeed even without changing the pixels near that cluster, and the pixels in that cluster contribute less to the attack effect.

[0086] In this step, the color channel perturbation value corresponding to the pixel position in the cluster containing interfering pixels less than a preset threshold is set to [0,0,0], meaning that the pixel at that position in the original image no longer changes. Then, the individual with the adjusted perturbation mode is input into the classifier for evaluation. If it is still an adversarial example, it means that these perturbations with low attack contribution can be eliminated, and the current individual is updated and replaced with the individual with the adjusted perturbation mode.

[0087] Step S5: Sort the union of the parent and offspring populations according to the constraint dominance rules, crowding distance, and individual fitness, and update the current parent population according to the sorting results. In this step, the parent and offspring populations are merged into a large set, which is then sorted again using the constraint dominance rule and crowding distance. Based on the diversity preservation strategy, several of the best individuals are selected from the sorted large set to form a new generation population, as described below.

[0088] The union of the parent and offspring populations is sorted according to the constraint dominance rule, crowding distance, and individual fitness. When there are no adversarial examples in the sorting results, multiple individual sets are generated. The best individual is selected from each individual set without repetition and added to the new population until the number of individuals in the new population reaches the total number of individuals in the parent population. When there are adversarial examples in the ranking results, multiple adversarial example sets are generated according to the category to which the adversarial examples belong, and a non-adversarial example set is generated. Individuals are first selected from the adversarial example set without repetition to the new population, and then individuals are selected from the non-adversarial example set without repetition to the new population, until the number of individuals in the parent population reaches the total number of individuals in the parent population. Finally, the parent population is updated to the new population.

[0089] Step S6: Return to step S3 until the maximum number of queries in the perturbation optimization parameters is reached, and generate the final adversarial sample based on the perturbation pattern of individuals in the current offspring population.

[0090] In this step, we return to step S3 and repeat the selection, crossover, mutation, and redundant perturbation elimination processes from the previous steps until the maximum number of queries in the perturbation optimization parameters is reached. Then, we generate the final adversarial sample from the perturbation patterns of individuals in the current offspring population.

[0091] Figure 4 To compare the adversarial sample obtained by attacking an image using the black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization provided in this invention, we can use the following method: Figure 5 A black-box sparse adversarial attack method based on multi-objective optimization is provided to attack adversarial samples obtained from the same image. Figure 4 and Figure 5 The left image shows the images obtained after applying perturbations to the original image using two different methods, while the right image shows the perturbation patterns applied to the original image. The maximum number of pixels that can be perturbed is set for each method. k The values ​​are all 150, color channel perturbation amplitude m The values ​​are all 1, and the maximum number of queries is [number missing]. Q The values ​​are all 5000. By comparison, it can be found that, while the attacked deep residual network misclassifies an image originally classified as a transport plane as a passenger plane, the black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization only needs to perturb 4 pixels to successfully attack the model. The index measuring the degree of perturbation between the adversarial example and the original image... The norm value is only 0.68, while black-box sparse adversarial attack methods based on multi-objective optimization require perturbation of 142 pixels to achieve the same effect of causing the attacked model to misclassify. The norm value is 8.4, indicating a significant increase in perturbation. This demonstrates that the black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization can guarantee the attack effect on the classifier and reduce the attack cost through perturbation pattern search and preservation strategies in the decision space and dynamic perturbation elimination strategies in the pixel dimension. Simultaneously, it generates low-visibility perturbation images, providing a novel attack capability for real-world attack scenarios that combines diversity, stealth, and high efficiency.

[0092] As can be seen from the above technical solution, the present invention discloses a black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization, which has the following advantages compared with the prior art: First, this invention models the black-box image adversarial attack problem as a constrained multimodal multi-objective optimization problem. The multi-objective optimization aims to simultaneously achieve two objectives: first, minimizing the loss function between the predicted and true classes of the perturbed image; and second, minimizing the difference between the perturbed and original images. Multimodality refers to multiple equivalent solutions satisfying the above objectives, i.e., combinations of different perturbed pixel positions and color channel perturbation values. The constraints are that candidate solutions must be adversarial examples and the number of perturbed pixels must be less than or equal to a set value. Unlike constrained multi-objective optimization problems that only focus on the diversity of solutions in the target space, the constrained multimodal multi-objective optimization problem, while pursuing diversity in the target space solutions, also ensures diversity in perturbation patterns in the input space, making attacks more difficult to defend against.

[0093] Secondly, this invention addresses the constrained multimodal multi-objective optimization problem in black-box sparse adversarial attacks by designing a constrained multimodal multi-objective genetic algorithm based on incremental clustering. This algorithm encodes the perturbation patterns of the original image into individuals within the genetic algorithm. Unlike traditional constrained multi-objective optimization algorithms, this algorithm maintains the diversity of perturbation patterns through a crossover strategy based on multi-population neighborhood search, mutation operators, and a diversity preservation strategy based on subpopulation mapping. Furthermore, it utilizes constraint dominance rules and crowding distance sorting to ensure rapid convergence, thereby improving attack success rate and efficiency.

[0094] In addition, this invention utilizes a pixel-dimensional incremental clustering algorithm to discover pixel perturbations with low attack contribution in different adversarial examples, and uses a dynamic perturbation elimination strategy to remove redundant perturbations in each round of algorithm iteration. Unlike perturbation degree control methods that rely solely on norm constraints, this strategy can significantly reduce the number of perturbation pixels in adversarial examples, further reducing the difference between the perturbated image and the original image, and improving the quality of adversarial examples.

[0095] Finally, this invention designs incremental clustering strategies at both macro and micro levels. At the macro level of population management, incremental clustering efficiently and dynamically divides the constantly evolving perturbation patterns into subpopulations. This provides an adaptive structural foundation for subsequent neighborhood search and diversity preservation strategies, thus ensuring the diversity of perturbation patterns. At the micro level of perturbation sparsity, incremental clustering analyzes the positions of perturbation pixels in adversarial examples at the pixel dimension. As new adversarial examples are generated, existing clusters are continuously updated, thereby accurately and efficiently discovering redundant pixels with low attack contribution. This provides precise information for dynamic perturbation elimination strategies, significantly improving the sparsity and concealment of perturbations.

[0096] In summary, this invention can reduce the distortion of adversarial examples while ensuring the success rate of attacks under sparsity constraints. It can also generate adversarial examples of multiple categories, demonstrating good diversity and attack generalization ability. This is of great significance for further improving the security and robustness of intelligent systems in the face of complex attack scenarios.

[0097] While the specific embodiments of the present invention depict actions or steps in a particular order, this should be understood as requiring such actions or steps to be performed in the shown specific order or sequential order, or requiring all illustrated actions or steps to be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations. The above descriptions are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention.

[0098] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization, characterized in that, Includes the following steps: Step S1: Determine the input image and the image classifier to be challenged, and initialize the perturbation optimization parameters; Determining the fitness of constrained multimodal multi-objective optimization includes: establishing the fitness based on the classification loss of the image classifier, the size of the perturbation, the sparsity of the perturbation, and whether the adversarial mechanism is successful; Step S2: Generate a parent population based on the input image and perturbation optimization parameters. The parent population contains multiple individuals, each representing a perturbation mode; calculate the fitness of each individual. Step S3: The current parent population is processed using a topological incremental clustering algorithm based on adaptive resonance theory, and then the current parent population is sorted according to the constraint dominance rule, crowding distance and individual fitness. Step S4: Filter, crossover, and mutate the current parent population to generate offspring population, and acquire and accumulate individuals belonging to adversarial examples in the offspring population; when the number of individuals belonging to adversarial examples is greater than the preset number, cluster and de-redundancy processing is performed on the perturbation patterns of individuals belonging to adversarial examples. Step S5: Sort the union of the parent and offspring populations according to the constraint dominance rules, crowding distance, and individual fitness, and update the current parent population according to the sorting results. Step S6: Return to step S3 until the maximum number of queries in the perturbation optimization parameters is reached, and generate the final adversarial sample based on the perturbation pattern of individuals in the current offspring population.

2. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 1, characterized in that, In step S1, the perturbation optimization parameters include the maximum number of perturbation pixels, the perturbation amplitude of the color channel, and the maximum number of queries; The fitness includes an optimization objective and constraints. The optimization objective is established based on the classification loss of the image classifier and the magnitude of the perturbation. The constraints are established based on the sparsity of the perturbation and whether the adversarial mechanism is successful. The expressions for the optimization objective and constraints are as follows: in, Indicates minimization. Indicating a response to the disturbance The two objective functions, Represents the classification loss function. This represents the trained image classifier for adversarial training. Indicates the input image. Indicates disturbance of Norm; Indicates constraints. The dimension is The real number field, These represent the width, height, and number of channels of the input image, respectively. Indicates disturbance of Norm, Indicates the maximum number of disturbed pixels. This indicates the category containing the highest confidence score. For category indexing, The total number of categories, This indicates that the image classifier will classify the scrambled image into the first... Confidence level of each category Indicates the true category.

3. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 2, characterized in that, In step S2, the step of generating the parent population specifically includes: Randomly select from all pixels of the input image A set of non-repeating pixel positions are used as perturbation pixel positions; for each perturbation pixel position, three color channel perturbation values ​​are generated, with the perturbation value for each color channel starting from... The perturbation pattern is formed by randomly selecting the perturbation pixel position and the perturbation value of the color channel of each pixel. The perturbation pattern is recorded as an individual, and multiple individuals constitute the parent population. The steps for calculating the fitness of each individual specifically include: overlaying the perturbation pattern of the individual onto the original image, processing the pixel values ​​using a cropping function to generate perturbation samples; inputting the perturbation samples into the image classifier to be challenged to obtain the predicted confidence scores for each category, and calculating the fitness based on the predicted confidence scores for each category.

4. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 3, characterized in that, Step S3 specifically includes: Step S3-1: Perform cluster initialization, winning node selection, warning value testing, and node learning processing on the current parent population in sequence to obtain the neighborhood of each individual; Step S3-2: Sort the parent population according to the constraint dominance rule and stratify the individuals in the neighborhood of each individual, and then sort the individuals in the same layer according to the crowding distance.

5. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 4, characterized in that, Step S3-1 specifically includes: Step S3-1-1, Clustering Initialization: Input individuals as data points into the topology adaptive resonance network, and set the first two data points input into the topology adaptive resonance network as two nodes of the topology network respectively; Step S3-1-2, Winning Node Selection: Whenever a new data point is added... The data is input into the topology network, and the current data point is compared with the existing set of nodes. CIM between nodes in the middle; The two nodes with the lowest CIM values ​​among the current data points are determined as the first and second winning nodes. Step S3-1-3, Warning Value Test: The CIM between the input data point and the first and second winning nodes are respectively and ; when At that time, the input data points are generated as new nodes in the topological adaptive resonant network; when At that time, update the number of data points that match the first winning node; when When the first winning node is reached, update the number of data points that match the first winning node, and update the neighbor nodes of the first winning node; Step S3-1-4, Node Learning: Update the positions of all winning nodes; Step S3-1-5: Return to step S3-1-2 until all individuals in the parent population have been processed. Determine the subpopulation to which each individual belongs based on the final topology network. Combine all individuals in the subpopulation to which the individual belongs, as well as all individuals in the subpopulations adjacent to the subpopulations to which all individuals belong, to form the neighborhood of the current individual.

6. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 5, characterized in that, In step S3-1-3, the expression for updating the neighbor nodes of the first winning node is: in, This represents the neighboring nodes of the first winning node. This indicates the number of data points contained in the neighboring node; In step S3-1-4, the expression for updating the positions of all winning nodes is: in express The number of data points contained in a node.

7. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 6, characterized in that, Step S3-2 specifically includes: Step S3-2-1: Determine the constraint dominance rules for a given individual. and An individual is considered to be an individual if any of the following conditions are met. Constraint and Domination of Individuals : (1) The perturbated image is an adversarial example. The perturbated image is not an adversarial example; (2) and The perturbated images are not adversarial examples, and < That is, classifier pair The confidence level of the perturbed image belonging to the true category is less than [a certain value]. The confidence level that the disturbed image belongs to the true category; (3) and The perturbated images are all adversarial samples, and < ,Right now The Euclidean distance between the perturbed image and the original image is less than 1. The Euclidean distance between the perturbed image and the original image; Step S3-2-2: Sort the parent population according to the constraint domination rule, including: dividing all individuals that are not dominated by any other individual into the first layer, then finding those that are not dominated from the remaining individuals and putting them into the second layer, and so on, until all individuals are stratified, thus completing the sorting. Step S3-2-3: For individuals in the same layer, calculate the crowding distance of the individuals, and sort them according to the requirement that individuals with larger crowding distances are ranked higher within the same layer; The expression for calculating congestion distance is: in, Represents an individual Crowded distance, Indicates the dimension of the target space. and They represent the first time. The target space sorts the individuals in the current layer in ascending order of their target values, and then assigns adjacent individuals to the target space. The target values ​​of adjacent individuals on both sides, and They represent the first The maximum and minimum values ​​of the solutions that exist in the target space.

8. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 7, characterized in that, Step S4 specifically includes: Step S4-1: Select individuals from the first half of the current parent population. For each individual, use probability... Perform a crossover operation between the current individual and the top-ranked individual in its neighborhood, with probability... Randomly select an individual from the population that is different from the current individual and perform a crossover operation to obtain offspring individuals; Step S4-2: Perform mutation operations on the offspring individuals, including randomly replacing the pixel positions of the perturbation patterns of the offspring individuals and the channel perturbation values ​​corresponding to the pixels, to form an offspring population; Step S4-3: Obtain and accumulate offspring individuals belonging to adversarial examples. When the number of individuals belonging to adversarial examples is greater than the preset number, cluster and de-redundancy processing is performed on the perturbation patterns of individuals belonging to adversarial examples.

9. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 8, characterized in that, In step S4-3, the step of clustering the perturbation patterns belonging to individual adversarial samples includes: When the number of adversarial examples exceeds the preset number When the perturbation pixel position coordinates of all adversarial samples are input into the topological incremental clustering algorithm based on adaptive resonance theory for processing, clustering is completed. The perturbation redundancy removal process specifically includes the following steps: Set the color channel perturbation value of the pixel position in the cluster containing interference pixels less than the preset threshold to [0,0,0]. Then input the individual with the adjusted perturbation mode into the classifier for evaluation. If it is still an adversarial example, replace the current individual with the individual with the adjusted perturbation mode.

10. The black-box sparse adversarial attack method based on constrained multimodal multi-objective resonance optimization according to claim 9, characterized in that, Step S5 specifically includes: The union of the parent and offspring populations is sorted according to the constraint dominance rule, crowding distance, and individual fitness. When there are no adversarial examples in the sorting results, multiple individual sets are generated. The best individual is selected from each individual set without repetition and added to the new population until the number of individuals in the new population reaches the total number of individuals in the parent population. When there are adversarial examples in the ranking results, multiple adversarial example sets are generated according to the category to which the adversarial examples belong, and a non-adversarial example set is generated. Individuals are first selected from the adversarial example set without repetition to the new population, and then individuals are selected from the non-adversarial example set without repetition to the new population, until the number of individuals in the parent population reaches the total number of individuals in the parent population. Finally, the parent population is updated to the new population.

Images