Black box anti-patch attack method for aerial image target detector
Optimizing adversarial patches in a black box environment through differential evolution algorithms has solved the problem that adversarial attack methods in the existing technology are difficult to apply to deep neural network models with unknown internal structures and parameters, and achieved efficient adversarial patch optimization and attack effects.
Patent Information
- Application Number
- CN202311207798.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-19
- Publication Date
- 2025-02-11
AI Technical Summary
The existing adversarial attack methods are difficult to apply to deep neural network models with unknown internal structures and parameters, resulting in poor attack effects of adversarial samples.
The differential evolution algorithm is used to optimize the adversarial patch in a black box environment. The final adversarial patch is obtained by pasting candidate adversarial patches on the target detection box position of the initial sample image, and using the fitness function to optimize and update the population iterative optimization and update.
Without the need for specific internal information of the target detector, optimized solutions for counter-patches are realized, avoid dimensional disasters and local optimal solutions, and improve optimization efficiency and attack effects against counter-patches.
Smart Images
Figure CN119418031A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a black box adversarial patch attack method for an aerial image target detector. Background Art
[0002] With the continuous development of remote sensing technology, the number of images and videos that can be collected by earth observation satellites or aircraft has increased dramatically. How to correctly and efficiently process and use these aerial images has become a key issue in the military, civilian and other fields.
[0003] In recent years, artificial intelligence technology represented by deep learning has been continuously developed and successfully applied in computer vision, natural language processing, speech recognition and other fields. It also provides a reliable solution for automatically extracting meaningful information from aerial images. Compared with traditional information extraction methods, deep neural networks have superior performance in solving visual problems, but studies have shown that deep neural networks are easily attacked by adversarial samples, resulting in incorrect predictions. Adversarial samples are defined as: adding maliciously designed perturbations to clean samples. Usually, such perturbations do not interfere with the judgment of the human eye, but can mislead some trained deep neural network models to produce incorrect prediction outputs. Adversarial samples were originally proposed for image classification tasks, and then expanded from image classification to target detection tasks and semantic segmentation tasks. In the field of aerial image target detection, adversarial samples can be roughly divided into two categories, including pixel-based adversarial samples and patch-based adversarial samples. Pixel-based adversarial samples generate adversarial perturbations by directly operating all or part of the pixels of the input image, thereby making the target detector based on the deep neural network invalid. Patch-based adversarial samples add perturbations in specific areas of the image, which tend to limit the scope of the perturbations rather than the intensity of the perturbations. Therefore, patch-based adversarial samples are easier to implement in the real world, posing greater security risks to the deployment and application of deep neural networks.
[0004] In practical applications, although the existence of adversarial samples will bring security threats to the application of target detectors based on deep neural network models, it can also be applied to scenarios such as anti-reconnaissance and anti-surveillance. Corresponding to the above two adversarial samples, adversarial attacks against target detectors based on deep neural network models can be divided into pixel-based adversarial attacks and patch-based adversarial attacks according to the different added disturbances. Among them, the pixel-based adversarial attack is to perturb the entire scene to obtain a specific or general camouflage attack style to achieve the purpose of deceiving the target detector. However, since this form of perturbation requires an attack on the entire scene, it is related to the background in which the target is located and is difficult to manufacture in the physical world, so it has little impact on specific applications. The patch-based adversarial attack is to limit the perturbation range in a specific area of the image, that is, to paste adversarial patches in a specific area of the image to achieve the purpose of deceiving the target detector. This attack method is usually independent of the background and can be implemented in the physical world, so it is currently also used in scenarios such as anti-reconnaissance and anti-surveillance. Specifically, when conducting anti-reconnaissance and anti-surveillance, adversarial patches are arranged in specific areas in the actual physical world so that adversarial patches exist in specific areas of the captured image, thereby preventing specific targets from being detected and identified by the target detector.
[0005] At present, most of the adversarial attack methods are aimed at the situation where the structure and parameters of the neural network model to be attacked are known. According to the information of the neural network model to be attacked, adversarial samples are generated by gradient updating to carry out adversarial attacks. However, in practical applications, the internal structure and parameter information of the neural network model to be attacked are usually unknown, which makes the existing adversarial attack methods difficult to apply and the attack effect of the generated adversarial samples is poor. Summary of the invention
[0006] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides a black box anti-patch attack method for aerial image target detector.
[0007] The technical solution of the present invention is as follows:
[0008] A black-box adversarial patch attack method for aerial imagery object detectors is provided, including:
[0009] Select an existing aerial image target detector or a trained deep neural network model that can be used for target detection;
[0010] Obtain a training data set, the training data including an initial sample image containing a target;
[0011] Set a confidence threshold, and use an aerial image target detector or a deep neural network model to obtain a target detection frame in the initial sample image whose confidence is not less than the confidence threshold;
[0012] Obtaining an initial population including a plurality of initial adversarial patches;
[0013] A candidate adversarial patch is pasted on the target detection frame position of the initial sample image to obtain a training sample image corresponding to the initial sample image. The training sample image is used as the input of the aerial image target detector or the deep neural network model to obtain the corresponding detection output result. Based on the initial population and the detection output result corresponding to the population obtained each time, the differential evolution algorithm is used to iteratively optimize and update the population to obtain the final adversarial patch. The candidate adversarial patch is obtained by dimensionality upgrading the adversarial patch in the population.
[0014] In some possible implementations, obtaining a target detection frame in an initial sample image with a confidence level not less than a confidence level threshold using an aerial image target detector or a deep neural network model includes:
[0015] The initial sample image is input into the aerial image target detector or the deep neural network model to obtain a target detection frame whose confidence is not less than a confidence threshold in the initial sample image output by the aerial image target detector or the deep neural network model.
[0016] In some possible implementations, an initial population including a plurality of initial adversarial patches is obtained using the following method:
[0017] Determine the initial size of the adversarial patch based on empirical values and the target size of the adversarial patch;
[0018] Based on the determined initial size, the initial adversarial patch is obtained by generating a grayscale image of a preset size or generating random pixels between [0, 1] for each pixel in the patch;
[0019] The process of obtaining the initial adversarial patch is repeated multiple times to obtain multiple initial adversarial patches.
[0020] In some possible implementations, the initial size of the adversarial patch has a width in the image width direction equal to 1 / N of the target size, and a height in the image height direction equal to 1 / M of the target size, where N and M are both positive integers.
[0021] In some possible implementations, a candidate adversarial patch is pasted at the target detection frame position of the initial sample image to obtain a training sample image corresponding to the initial sample image, the training sample image is used as an input of an aerial image target detector or a deep neural network model to obtain a corresponding detection output result, based on the initial population and the detection output result corresponding to the population obtained each time, a differential evolution algorithm is used to iteratively optimize and update the population to obtain a final adversarial patch, further comprising:
[0022] Step S51, obtaining a candidate adversarial patch corresponding to each initial adversarial patch in the initial population;
[0023] Step S52, pasting a candidate adversarial patch at the target detection frame position of the initial sample image to obtain a training sample image corresponding to the initial sample image, and using the training sample image as an input of an aerial image target detector or a deep neural network model to obtain a corresponding detection output result;
[0024] Step S53, calculating a preset fitness function value corresponding to each adversarial patch in the current population according to the acquired detection output result;
[0025] Step S54, determine whether the preset termination condition is met, if not, proceed to the next step, if yes, proceed to step S57;
[0026] Step S55, based on the current population and the fitness function, performing population mutation, crossover and selection operations to obtain an intermediate population including a plurality of adversarial patches;
[0027] Step S56, selecting multiple adversarial patches from the current population and the intermediate population as a new generation population, obtaining a candidate adversarial patch corresponding to each adversarial patch in the new generation population, and returning to step S52;
[0028] Step S57, selecting the adversarial patch with the smallest corresponding fitness function value in the current population as the final adversarial patch output.
[0029] In some possible implementations, candidate adversarial patches are obtained by:
[0030] The adversarial patch is copied, spliced and tiled along the width and height directions of the image until the width is consistent with the width in the target size and the height is consistent with the height in the target size, thereby obtaining a candidate adversarial patch with a size consistent with the target size.
[0031] In some possible implementations, the detection output results of the aerial image target detector or the deep neural network model include: a target detection box and its corresponding confidence level.
[0032] In some possible implementations, the fitness function is set to:
[0033] S(x)=P(x)-S0, where S(x) represents the fitness function value corresponding to the adversarial patch x in the population, P(x) represents the confidence of the target detection box with the candidate adversarial patch corresponding to the adversarial patch x pasted on it, and S0 represents the confidence threshold. If there are multiple target detection boxes with the candidate adversarial patch corresponding to the adversarial patch x pasted on them, then P(x) uses the average of the confidences of the corresponding multiple target detection boxes.
[0034] In some possible implementations, the termination condition is that there is at least one adversarial patch whose corresponding fitness function value is less than 0.
[0035] In some possible implementations, the target is at least one of an airplane, a ship, and a truck.
[0036] The main advantages of the technical solution of the present invention are as follows:
[0037] The black-box adversarial patch attack method for aerial image target detectors of the present invention can achieve the optimal solution of adversarial patches without the need for specific internal information of the target detector by adopting a differential evolution algorithm; and by performing dimensionality reduction processing on the adversarial patch variables to be solved, it can avoid the dimensionality disaster and the situation of falling into the local optimal solution when solving using the differential evolution algorithm, thereby improving the optimization efficiency and the attack effect of the adversarial patch finally obtained. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0039] Figure 1 A flowchart of a black box anti-patch attack method for an aerial image target detector according to an embodiment of the present invention;
[0040] Figure 2 A schematic diagram of the generation principle of an adversarial patch according to an embodiment of the present invention;
[0041] Figure 3 This is a training sample image when the target is an airplane according to an embodiment of the present invention. DETAILED DESCRIPTION
[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0043] The technical solution provided by the embodiments of the present invention is described in detail below with reference to the accompanying drawings.
[0044] refer to Figure 1 An embodiment of the present invention provides a black box anti-patch attack method for an aerial image target detector, the method comprising the following steps S1-S5:
[0045] Step S1, selecting an existing aerial image target detector or a trained deep neural network model that can be used for target detection.
[0046] Specifically, in one embodiment of the present invention, if there is an aerial image target detector that can be used for target detection, then any one of the existing aerial image target detectors is selected. If there is no aerial image target detector that can be used for target detection, then any one of the deep neural network models that have been trained and can be used for target detection is selected.
[0047] Existing target detection can be divided into a single-stage and a two-stage series. The single-stage target detection mainly includes the YOLO series neural network, and the two-stage target detection is represented by the Faster R-CNN neural network. Therefore, in one embodiment of the present invention, a deep neural network model corresponding to the aerial image target detector can be selected from the trained YOLO series neural network or the Faster R-CNN neural network. The YOLO series neural network includes neural networks such as YOLOv2, YOLOv3 and YOLOv4.
[0048] Step S2, obtaining a training data set, where the training data includes an initial sample image containing a target.
[0049] In one embodiment of the present invention, a training data set is obtained from an existing known data set.
[0050] Specifically, for aerial image targets, the existing DOTA-v1.0 dataset, RSOD dataset or NWPUVHR-10 dataset is selected as the training dataset. Among them, the DOTA-v1.0 dataset is a large-scale dataset containing 2806 images, including 15 common target categories including aircraft collected from different sensors and platforms. Since the image sizes in the DOTA-v1.0 dataset are in different ranges, when the existing DOTA-v1.0 dataset is selected as the training dataset, all images are divided into 608x608 pixel sub-images for subsequent processing in a consistent format; then based on the obtained images, according to the actual determined aerial image targets, the images are filtered to obtain images containing the corresponding targets as initial sample images to obtain the training dataset. When the existing RSOD dataset or NWPU VHR-10 dataset is selected as the training dataset, according to the actual determined aerial image targets, the images in the dataset are filtered to obtain images containing the corresponding targets as initial sample images to obtain the training dataset.
[0051] In one embodiment of the present invention, the target is determined according to actual needs, for example, the target is at least one of an airplane, a ship, and a truck.
[0052] Step S3, setting a confidence threshold, and using an aerial image target detector or a deep neural network model to obtain a target detection frame in the initial sample image whose confidence is not less than the confidence threshold.
[0053] When the current target detector is working, the final prediction result will apply a preset confidence threshold to filter out redundant detection results with lower confidence, and finally exclude detection frames with higher mutual overlap through a non-maximum suppression mechanism. The purpose of the adversarial patch generated by an embodiment of the present invention is to make the confidence value of the target detected and recognized by the target detector after the adversarial patch is pasted lower than the confidence threshold, so that the target detection frame is screened out, thereby making the target unable to be detected and recognized by the target detector, achieving the "camouflage" and "invisibility" effects of the target.
[0054] In one embodiment of the present invention, in order to ensure the adversarial attack effect of the subsequently generated adversarial patch, an aerial image target detector or a deep neural network model is used to obtain a target detection frame in the initial sample image whose confidence is not less than a confidence threshold, and the adversarial patch is optimized based on the target detection frame.
[0055] Furthermore, in one embodiment of the present invention, obtaining a target detection frame in an initial sample image with a confidence level not less than a confidence level threshold using an aerial image target detector or a deep neural network model includes:
[0056] The initial sample image is input into the aerial image target detector or the deep neural network model to obtain a target detection frame whose confidence is not less than a confidence threshold in the initial sample image output by the aerial image target detector or the deep neural network model.
[0057] In the above manner, an aerial image target detector or a deep neural network model can be used to obtain a target detection frame in the initial sample image whose confidence is not less than the confidence threshold, and the obtained target detection frame is used for subsequent generation and optimization of adversarial patches.
[0058] Step S4, obtaining an initial population including a plurality of initial adversarial patches.
[0059] In one embodiment of the present invention, an initial population including a plurality of initial adversarial patches is obtained by:
[0060] Determine the initial size of the adversarial patch based on empirical values and the target size of the adversarial patch;
[0061] Based on the determined initial size, the initial adversarial patch is obtained by generating a grayscale image of a preset size or generating random pixels between [0, 1] for each pixel in the patch;
[0062] The process of obtaining the initial adversarial patch is repeated multiple times to obtain multiple initial adversarial patches.
[0063] Among them, the population size, that is, the number of individuals in the population (the number of adversarial patches) is set according to the actual situation.
[0064] Since the differential evolution algorithm usually suffers from the curse of dimensionality when conducting high-dimensional search problems and is prone to falling into local optimal solutions, in order to improve the attack effect of the adversarial patches generated by the differential evolution algorithm in the subsequent process, in one embodiment of the present invention, the adversarial patches in the population are subjected to dimensionality reduction processing, that is, the size of the adversarial patches in the population (i.e., the initial size) is made smaller than the size of the adversarial patches actually used for pasting (i.e., the target size). The adversarial patches actually used for pasting are generated by enlarging the adversarial patch encoding in the population to a relatively large space, and are specifically generated by copying, splicing and tiling an adversarial patch in the population along the width and height directions of the image.
[0065] Furthermore, since the adversarial patch for actual pasting is generated by copying, splicing and tiling an adversarial patch in the population along the width and height directions of the image, in one embodiment of the present invention, the initial size of the adversarial patch is equal to 1 / N of the target size in the width direction of the image, and is equal to 1 / M of the target size in the height direction of the image, where N and M are both positive integers and can be set according to actual conditions.
[0066] Step S5, paste a candidate adversarial patch at the target detection frame position of the initial sample image, obtain a training sample image corresponding to the initial sample image, use the training sample image as the input of the aerial image target detector or the deep neural network model to obtain the corresponding detection output result, based on the initial population and the detection output result corresponding to the population obtained each time, use the differential evolution algorithm to iteratively optimize and update the population to obtain the final adversarial patch, and the candidate adversarial patch is obtained by dimensionality upgrading the adversarial patch in the population.
[0067] In one embodiment of the present invention, a candidate adversarial patch is pasted on the target detection frame position of the initial sample image to obtain a training sample image corresponding to the initial sample image, and the training sample image is used as the input of the aerial image target detector or the deep neural network model to obtain the corresponding detection output result. Based on the initial population and the detection output result corresponding to the population obtained each time, the differential evolution algorithm is used to perform iterative optimization and update of the population to obtain the final adversarial patch, further comprising the following steps S51-S57:
[0068] Step S51, obtaining a candidate adversarial patch corresponding to each initial adversarial patch in the initial population.
[0069] In one embodiment of the present invention, the candidate adversarial patches are obtained by performing dimension upgrading on the adversarial patches in the population.
[0070] Specifically, based on the setting of step S4 above, the candidate adversarial patch is obtained in the following way:
[0071] The adversarial patch is copied, spliced and tiled along the width and height directions of the image until the width is consistent with the width in the target size and the height is consistent with the height in the target size, thereby obtaining a candidate adversarial patch with a size consistent with the target size.
[0072] By using the above method to increase the dimension of the adversarial patch, the candidate adversarial patch with the image size of the target size can be obtained.
[0073] Step S52, paste a candidate adversarial patch at the target detection frame position of the initial sample image to obtain a training sample image corresponding to the initial sample image, and use the training sample image as the input of the aerial image target detector or the deep neural network model to obtain the corresponding detection output result.
[0074] In one embodiment of the present invention, in order to improve the optimization efficiency of the adversarial patch and ensure the adversarial attack effect of the subsequently generated adversarial patch, each candidate adversarial patch has at least one corresponding target detection frame, that is, each candidate adversarial patch has at least one target detection frame to which the candidate adversarial patch is pasted.
[0075] Optionally, in order to further improve the optimization efficiency of the adversarial patch and ensure the adversarial attack effect of the subsequently generated adversarial patch, in one embodiment of the present invention, the candidate adversarial patches pasted on the target detection frame of the same initial sample image are the same.
[0076] Among them, the pasting position of the candidate adversarial patch is the center position of the target detection box.
[0077] Furthermore, in one embodiment of the present invention, the detection output result of the aerial image target detector or the deep neural network model includes: a target detection frame and its corresponding confidence level.
[0078] Step S53, calculating a preset fitness function value corresponding to each adversarial patch in the current population according to the acquired detection output result.
[0079] In one embodiment of the present invention, the fitness function is set as:
[0080] S(x)=P(x)-S0
[0081] Among them, S(x) represents the fitness function value corresponding to the adversarial patch x in the population, P(x) represents the confidence of the target detection box with the candidate adversarial patch corresponding to the adversarial patch x pasted on it, S0 represents the confidence threshold, if there are multiple target detection boxes with the candidate adversarial patch corresponding to the adversarial patch x pasted on them, then P(x) adopts the average of the confidences of the corresponding multiple target detection boxes.
[0082] Furthermore, based on the fitness function set above, the fitness function value corresponding to each adversarial patch in the population is calculated according to the obtained detection output result.
[0083] Step S54, determine whether the preset termination condition is reached, if not, proceed to the next step, if yes, proceed to step S57.
[0084] In one embodiment of the present invention, the termination condition is specifically set according to actual conditions.
[0085] Furthermore, since the role of the final adversarial patch is to make the confidence value of the target detected and recognized by the target detector after the adversarial patch is pasted lower than the confidence threshold so that the target detection box is screened out, for this reason, the termination condition can be set to that there is at least one adversarial patch corresponding to a fitness function value less than 0.
[0086] Step S55: Based on the current population and the fitness function, population mutation, crossover and selection operations are performed to obtain an intermediate population including a plurality of adversarial patches.
[0087] In one embodiment of the present invention, based on the current population and the fitness function, population mutation, crossover and selection operations are performed to obtain an intermediate population including a plurality of adversarial patches, further comprising:
[0088] Based on the current population, perform mutation operations to obtain multiple mutant individuals;
[0089] According to the current population and its corresponding mutant individuals, a crossover operation is performed to obtain the offspring population corresponding to the current population;
[0090] According to the current population and its corresponding offspring population, the selection operation is performed with the goal of minimizing the fitness function value corresponding to the individual to obtain the intermediate population.
[0091] Among them, the obtained intermediate population is the next generation population of the current population.
[0092] Specifically, set the current population to:
[0093]
[0094] Among them, P t represents the current population, that is, the tth generation population, P t (i) represents the population P t The i-th individual in represents the range of individual variables, N represents the total number of individuals in the population, and T represents the total number of iterations. To combat patch attacks, the upper and lower bounds of all variables are [0,1].
[0095] The mutation operation is expressed as:
[0096] PM (i) = P t (r1)+F(P t (r2)-P t (r3)) r1≠r2≠r3≠i
[0097] Among them, P M (i) represents individual P t (i) The corresponding variant individual, F represents the mutation factor, P t (r1),P t (r2) and P t (r3) respectively represent the population P t The r1th individual, r2th individual and r3th individual in , r1, r2 and r3 represent individual indexes, satisfying the unequal constraint.
[0098] The crossover operation is expressed as:
[0099]
[0100] Among them, P C (i) represents the population P t The corresponding offspring population P C The i-th individual in N individuals P C (i) (i = 1, 2, ..., N) constitutes the offspring population P C , C r represents the mutation probability, and rand represents random sampling from the interval [0,1].
[0101] The selection operation is represented as:
[0102]
[0103] Among them, P t+1 (i) represents the population P of the t+1th generation t+1 The i-th individual in f(P t (i)) represents individual P t (i) The corresponding fitness function value, f(P C (i)) represents individual P C (i) A corresponding fitness function value, where the fitness function value is determined according to the aforementioned acquisition calculation method.
[0104] In one embodiment of the present invention, by setting the adversarial patches in the population to be smaller in size and lower in dimension, operations such as mutation, crossover and selection in the differential evolution algorithm can be performed only in a low-dimensional space, effectively preventing the optimization problem from falling into a local optimal solution.
[0105] Step S56, selecting multiple adversarial patches from the current population and the intermediate population as a new generation population, obtaining a candidate adversarial patch corresponding to each adversarial patch in the new generation population, and returning to step S52.
[0106] In one embodiment of the present invention, a preset number of adversarial patches with the smallest fitness function values are selected from the current population and the intermediate population as individuals in the population, serving as a new generation population.
[0107] In one embodiment of the present invention, the number of individuals in different generations of populations is the same, that is, the number of adversarial patches is the same. In this case, the preset number is the same as the number of initial adversarial patches in the initial population.
[0108] Furthermore, for the determined new generation population, the adversarial patches in the population are dimensionally upgraded to obtain candidate adversarial patches corresponding to each adversarial patch in the new generation population.
[0109] The method for obtaining the candidate adversarial patch corresponding to each adversarial patch in the new generation population is the same as the method for obtaining the candidate adversarial patch corresponding to each initial adversarial patch in the initial population in step S51.
[0110] Step S57, selecting the adversarial patch with the smallest corresponding fitness function value in the current population as the final adversarial patch output.
[0111] In one embodiment of the present invention, when the termination condition is met, the adversarial patch with the smallest corresponding fitness function value in the current population is selected as the final adversarial patch output.
[0112] Specifically, based on the fitness function defined above, the smaller the obtained fitness function value is, the better the adversarial attack effect of the adversarial patch is.
[0113] Furthermore, in one embodiment of the present invention, the black box anti-patch attack method may also include:
[0114] Referring to the pasting position and method of the candidate adversarial patch in the training sample image, the final adversarial patch is upgraded to obtain the corresponding actual adversarial patch, and the actual adversarial patch is placed in the real world to carry out adversarial attack.
[0115] A black-box adversarial patch attack method for an aerial image target detector provided by an embodiment of the present invention can achieve the optimal solution of the adversarial patch without the need for specific internal information of the target detector by adopting a differential evolution algorithm; and, by performing dimensionality reduction processing on the adversarial patch variables to be solved, it can avoid the dimensionality disaster and the situation of falling into the local optimal solution when solving the problem using the differential evolution algorithm, thereby improving the optimization efficiency and the attack effect of the adversarial patch.
[0116] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In addition, "front", "back", "left", "right", "upper" and "lower" in this article are all referenced to the placement state shown in the accompanying drawings.
[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A black box adversarial patch attack method for aerial image target detectors, characterized in that: include: Select an existing aerial image target detector or a trained deep neural network model that can be used for target detection; Obtain a training data set, the training data including an initial sample image containing a target; Set a confidence threshold, and use an aerial image target detector or a deep neural network model to obtain a target detection frame in the initial sample image whose confidence is not less than the confidence threshold; Obtaining an initial population including a plurality of initial adversarial patches; A candidate adversarial patch is pasted on the target detection frame position of the initial sample image to obtain a training sample image corresponding to the initial sample image. The training sample image is used as the input of the aerial image target detector or the deep neural network model to obtain the corresponding detection output result. Based on the initial population and the detection output result corresponding to the population obtained each time, the differential evolution algorithm is used to iteratively optimize and update the population to obtain the final adversarial patch. The candidate adversarial patch is obtained by dimensionality upgrading the adversarial patch in the population.
2. The black box anti-patch attack method for aerial image target detector according to claim 1 is characterized in that: Use an aerial image target detector or a deep neural network model to obtain a target detection frame in the initial sample image with a confidence level not less than a confidence threshold, including: The initial sample image is input into the aerial image target detector or the deep neural network model to obtain a target detection frame whose confidence is not less than a confidence threshold in the initial sample image output by the aerial image target detector or the deep neural network model.
3. The black box anti-patch attack method for aerial image target detector according to claim 1 is characterized in that: The initial population including multiple initial adversarial patches is obtained in the following way: Determine the initial size of the adversarial patch based on empirical values and the target size of the adversarial patch; Based on the determined initial size, the initial adversarial patch is obtained by generating a grayscale image of a preset size or generating random pixels between [0, 1] for each pixel in the patch; The process of obtaining the initial adversarial patch is repeated multiple times to obtain multiple initial adversarial patches.
4. The black box anti-patch attack method for aerial image target detector according to claim 3 is characterized in that: The initial size of the adversarial patch is equal to 1 / N of the target size in the image width direction and 1 / M of the target size in the image height direction, where N and M are both positive integers.
5. The black box anti-patch attack method for aerial image target detector according to claim 4 is characterized in that: Paste a candidate adversarial patch at the target detection frame position of the initial sample image to obtain a training sample image corresponding to the initial sample image, use the training sample image as the input of the aerial image target detector or the deep neural network model to obtain the corresponding detection output result, and use the differential evolution algorithm to iteratively optimize and update the population based on the initial population and the detection output result corresponding to the population obtained each time to obtain the final adversarial patch, further comprising: Step S51, obtaining a candidate adversarial patch corresponding to each initial adversarial patch in the initial population; Step S52, pasting a candidate adversarial patch at the target detection frame position of the initial sample image to obtain a training sample image corresponding to the initial sample image, and using the training sample image as an input of an aerial image target detector or a deep neural network model to obtain a corresponding detection output result; Step S53, calculating a preset fitness function value corresponding to each adversarial patch in the current population according to the acquired detection output result; Step S54, determine whether the preset termination condition is met, if not, proceed to the next step, if yes, proceed to step S57; Step S55, based on the current population and the fitness function, performing population mutation, crossover and selection operations to obtain an intermediate population including a plurality of adversarial patches; Step S56, selecting multiple adversarial patches from the current population and the intermediate population as a new generation population, obtaining a candidate adversarial patch corresponding to each adversarial patch in the new generation population, and returning to step S52; Step S57, selecting the adversarial patch with the smallest corresponding fitness function value in the current population as the final adversarial patch output.
6. The black box anti-patch attack method for aerial image target detector according to claim 5 is characterized in that: Candidate adversarial patches are obtained in the following way: The adversarial patch is copied, spliced and tiled along the width and height directions of the image until the width is consistent with the width in the target size and the height is consistent with the height in the target size, thereby obtaining a candidate adversarial patch with a size consistent with the target size.
7. The black box anti-patch attack method for aerial image target detector according to claim 5 is characterized in that: The detection output results of the aerial image object detector or deep neural network model include: object detection box and its corresponding confidence.
8. The black box anti-patch attack method for aerial image target detector according to claim 7 is characterized in that: The fitness function is set as: S(x)=P(x)-S0 Among them, S(x) represents the fitness function value corresponding to the adversarial patch x in the population, P(x) represents the confidence of the target detection box with the candidate adversarial patch corresponding to the adversarial patch x pasted on it, S0 represents the confidence threshold, if there are multiple target detection boxes with the candidate adversarial patch corresponding to the adversarial patch x pasted on them, then P(x) adopts the average of the confidences of the corresponding multiple target detection boxes.
9. The black box anti-patch attack method for aerial image target detector according to claim 8 is characterized in that: The termination condition is that there is at least one adversarial patch whose corresponding fitness function value is less than 0.
10. The black box anti-patch attack method for aerial image target detector according to any one of claims 1 to 9, characterized in that: The target is at least one of an airplane, a ship and a truck.