Adversarial sample generation methods, devices, electronic equipment and storage media

By adding perturbations to the initial samples to generate a population of adversarial samples and screening and optimizing elite samples, the problem of low success rate of adversarial sample generation in black-box attack scenarios is solved, and the disguise of adversarial samples and the success rate of attacks are improved.

CN116883777BActive Publication Date: 2025-10-28BEIJING INST OF CONTROL & ELECTRONICS TECH +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310752989.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-06-25
Publication Date
2025-10-28
Estimated Expiration
2043-06-25

AI Technical Summary

Technical Problem

Existing adversarial sample generation methods have a low success rate in black-box attack scenarios and cannot effectively improve the reliability and robustness of machine learning models.

Method used

By adding perturbations to the initial samples, a population of adversarial samples is generated, and elite adversarial samples are screened out. The camouflage and attack success of the adversarial samples are improved through fitness calculation and optimization methods, and genetic optimization method or gradient evaluation method is used for optimization.

Benefits of technology

The success rate of generating adversarial samples in black-box attack scenarios is improved, and the disguise and attack success of adversarial samples are enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116883777B_ABST
    Figure CN116883777B_ABST
Patent Text Reader

Abstract

This application provides a method, apparatus, electronic device, and storage medium for generating adversarial examples. The method includes: adding perturbation to at least one initial sample to obtain an adversarial example population; selecting a preset number of elite adversarial examples based on the fitness of the adversarial examples; inputting the initial sample corresponding to the elite adversarial examples into a target detection model to obtain a first identification result, wherein the first identification result is the true identification result; inputting the elite adversarial examples into the target detection model to obtain a second identification result; and determining that the elite adversarial examples are target adversarial examples in response to determining that the second identification result is consistent with the first identification result. This application can improve the success rate of adversarial example generation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method, apparatus, electronic device and storage medium for generating adversarial samples. Background Technology

[0002] With the development of artificial intelligence technologies, represented by deep learning, machine learning models based on deep learning are being applied to more and more fields. However, in the process of studying image classification problems, researchers have discovered some subtle "perturbations" that are almost imperceptible to the human eye. When these perturbations are superimposed on the original image, they propagate through the powerful learning capabilities of deep learning models, causing misclassifications of the image's predicted category.

[0003] To improve the reliability and robustness of machine learning models, the concept of adversarial examples has been proposed. However, most adversarial example generation methods in these technologies are designed for white-box attack scenarios, and their performance in black-box attack scenarios still needs improvement. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose an adversarial sample generation method, apparatus, electronic device and storage medium, which aims to improve the success rate of adversarial sample generation in black-box attack scenarios.

[0005] To achieve the above objectives, this application provides a method for generating adversarial examples, comprising:

[0006] Add perturbation to at least one initial sample to obtain an adversarial sample population;

[0007] Based on the fitness of the adversarial examples, a preset number of elite adversarial examples are selected.

[0008] The initial sample corresponding to the elite adversarial sample is input into the target detection model to obtain the first recognition result, which is the true recognition result;

[0009] The elite adversarial sample is input into the target detection model to obtain a second recognition result;

[0010] In response to determining that the second identification result is consistent with the first identification result, the elite adversarial sample is determined to be a target adversarial sample.

[0011] Optionally, the method further includes:

[0012] In response to determining that the second identification result is inconsistent with the first identification result, and that the similarity metric between the second identification result and the first identification result is less than or equal to a preset threshold, the fitness of the elite adversarial sample is updated and the elite adversarial sample is re-added to the adversarial sample population.

[0013] Optionally, updating the fitness of the elite adversarial examples includes:

[0014] Based on the elite adversarial sample, the gradient evaluation value of the elite adversarial sample is obtained through a preset gradient evaluation value algorithm;

[0015] The gradient evaluation value is used as the updated fitness of the elite sample.

[0016] Optionally, updating the fitness of the elite adversarial examples includes:

[0017] Based on the elite adversarial samples, the gradient evaluation value is calculated using the following formula;

[0018]

[0019] Among them, x i e represents the elite sample. i Let G(x) represent the regularized basis vector of the i-th dimension of the elite sample, δ represent the preset slight perturbation value, and G(x) represent the regularized basis vector of the i-th dimension of the elite sample. i ()) represents the gradient value of the elite adversarial example, (G(x) i +δe j () represents the first perturbation gradient value of the elite adversarial sample, (G(x) i +δ ed ) represents the second perturbation gradient value of the elite adversarial example. This represents the gradient evaluation value of the elite sample;

[0020] The gradient evaluation value is used as the fitness of the elite sample.

[0021] Optionally, the method further includes:

[0022] In response to determining that the second identification result is inconsistent with the first identification result, and that the similarity metric between the second identification result and the first identification result is greater than a preset threshold, the elite adversarial sample is optimized, and the optimized elite adversarial sample is re-added to the adversarial sample population.

[0023] Optionally, optimizing the elite adversarial examples includes:

[0024] Based on the elite adversarial samples and the adversarial sample population, optimized elite adversarial samples are obtained through crossover and / or mutation algorithms.

[0025] Optionally, the fitness calculation steps include:

[0026] Based on the adversarial sample, the attack success rate and camouflage of the adversarial sample are obtained;

[0027] Based on the attack success and the camouflage, the fitness of the adversarial sample is calculated using the following formula;

[0028]

[0029] Wherein, Fitness represents the fitness of the adversarial example. This indicates the success rate of the adversarial sample attack. The adversarial sample is camouflaged, and α represents the preset camouflage weight.

[0030] Based on the same inventive concept, this application also provides an adversarial sample generation apparatus, comprising:

[0031] The first generation module is configured to add perturbations to at least one initial sample to obtain an adversarial sample population.

[0032] The second generation module is configured to select a preset number of elite adversarial samples based on the fitness of the adversarial samples.

[0033] The first calculation module is configured to input the initial sample corresponding to the elite adversarial sample into the target detection model to obtain a first recognition result, wherein the first recognition result is the true recognition result.

[0034] The second calculation module is configured to input the elite adversarial sample into the target detection model to obtain a second recognition result;

[0035] The judgment module is configured to determine the elite adversarial sample as a target adversarial sample in response to determining that the second identification result is consistent with the first identification result.

[0036] Based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the adversarial example generation method as described in any of the above.

[0037] Based on the same inventive concept, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute any of the adversarial sample generation methods described above.

[0038] As described above, the adversarial sample generation method, apparatus, electronic device, and storage medium provided in this application first screen adversarial samples based on fitness values, considering both camouflage and attack success rate. Then, based on the judgment results, the adversarial samples are optimized using either a full optimization method or a local optimization method. The optimization process employs genetic optimization or gradient evaluation. This application can effectively improve the success rate of adversarial sample generation in black-box attack scenarios. Furthermore, this application can improve both the camouflage and attack success rate of adversarial samples. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating one or more embodiments of the adversarial sample generation method of this application;

[0041] Figure 2 This is a schematic diagram of the structure of an adversarial sample generation device according to one or more embodiments of this application;

[0042] Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to one or more embodiments of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0044] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0045] As described in the background section, with the development of artificial intelligence technologies, represented by deep learning, deep learning-based machine learning models are being applied to an increasing number of fields. For example, machine learning models designed for image recognition are increasingly being used in facial recognition or security authentication. In the financial sector, online payments or payment security authentication using facial recognition are gradually evolving into a completely new payment or payment security authentication method.

[0046] Machine learning models are typically trained using a large number of samples. However, evidence suggests that adversarial examples, generated by slightly modifying existing training samples, can lead to absurd results. The discovery of adversarial examples has made it clear that deep learning also has its vulnerabilities and unreliability. Therefore, to improve the reliability and robustness of machine learning models, techniques are being developed to train models using generated adversarial examples.

[0047] However, most adversarial example generation methods proposed in related technologies are aimed at white-box attack scenarios, requiring model parameters as a basis for calculation. For example, some related technologies propose using the Fast Gradient Sign Method (FGSM) for adversarial example generation. FGSM is a single-step attack method, requiring only one gradient calculation, and then adjusting the direction of change by calculating the gradient using a sign function. Related technologies also propose improving FGSM with an iterative attack method, proposing I-FGSM to address the problem of excessively large step sizes and generated perturbations in FGSM. Furthermore, some related technologies propose MI-FGSM, which introduces the concept of momentum to stabilize the selection of gradient direction and enhance transferability.

[0048] However, in practical applications, adversarial examples are mostly calculated in black-box attack scenarios. In black-box attack scenarios, the model's parameters are not readily available, and even the number of training iterations is very limited. Therefore, the methods proposed in related technologies have significant limitations in black-box attack scenarios, resulting in a low success rate for the generated adversarial examples.

[0049] Therefore, this application proposes a method for generating adversarial examples.

[0050] The technical solutions of one or more embodiments of this application will be described in detail below through specific examples.

[0051] refer to Figure 1 The adversarial example generation method of one or more embodiments of this application includes the following steps:

[0052] Step S101: Add perturbation to at least one initial sample to obtain an adversarial sample population.

[0053] In some embodiments, the initial samples described above are regular training samples for the model. For example, image training samples used when training a model for image recognition.

[0054] In some embodiments, the population of adversarial examples may include at least one adversarial example population with added perturbations.

[0055] In some embodiments, the initial samples can be preprocessed to convert them into one-dimensional samples, and then perturbations can be added to obtain adversarial samples. In some embodiments, at least one of the aforementioned one-dimensional samples can be combined to obtain a multi-dimensional matrix, and the adversarial sample population is (x1, x2, ..., x...). m ), where m is the population size, x i Let i be the input sample, i∈m.

[0056] Step S102: Based on the fitness of the adversarial examples mentioned above, a preset number of elite adversarial examples are selected.

[0057] In this step, the fitness of adversarial samples in the aforementioned adversarial sample population is calculated. In some embodiments, fitness can be calculated through the following steps: based on the aforementioned adversarial samples, the attack success rate and camouflage of the adversarial samples are obtained; based on the aforementioned attack success rate and camouflage, the fitness of the adversarial samples is calculated using the following formula; Wherein, Fitness represents the fitness of the aforementioned adversarial examples. This indicates the success rate of the attack in the aforementioned adversarial sample. The above adversarial examples are camouflaged, and α represents the preset camouflage weight.

[0058] The fitness calculation formula mentioned above takes into account both the camouflage of adversarial examples and the success rate of attacks, thereby improving the success rate of adversarial example generation.

[0059] Based on the above fitness screening, a preset number of adversarial samples are obtained as elite adversarial samples.

[0060] In some embodiments, a predetermined number of adversarial samples with the highest fitness are selected as elite adversarial samples. In some embodiments, a predetermined number of elite adversarial samples can be selected using selection algorithms such as random competitive selection, optimal retention selection, and random selection without replay. Different selection algorithms, as long as they achieve the corresponding purpose, will not affect the scope of protection of this invention.

[0061] Step S103: Input the initial sample corresponding to the above elite adversarial sample into the target detection model to obtain the first recognition result, which is the true recognition result.

[0062] To determine the value of the generated adversarial examples, we first calculate the recognition results obtained by inputting the initial samples corresponding to the elite adversarial examples into the target detection model as a benchmark for comparison and evaluation.

[0063] Step S104: Input the above elite adversarial sample into the above target detection model to obtain the second recognition result.

[0064] Step S105: In response to determining that the second identification result is consistent with the first identification result, the elite adversarial sample is determined to be the target adversarial sample.

[0065] In this step, the second identification result is compared with the first identification result, and the elite adversarial sample is used as a target adversarial sample based on the comparison result.

[0066] When the second identification result is consistent with the first identification result, it indicates that the model cannot successfully identify the adversarial sample, and the adversarial sample has training significance.

[0067] When the second identification result is inconsistent with the first identification result, it indicates that the adversarial example is not of training significance.

[0068] In implementing this application, the applicant discovered that the need for optimization of adversarial examples can be determined by measuring the similarity between the first and second identification results. When the similarity measure is low, it can be considered that the fitness prediction of the adversarial example is too high, but further optimization is no longer necessary.

[0069] In some embodiments, the method further includes: in response to determining that the second identification result is inconsistent with the first identification result, and that the similarity measure between the second identification result and the first identification result is less than or equal to a preset threshold, updating the fitness of the elite adversarial sample and re-adding the elite adversarial sample to the adversarial sample population.

[0070] When the second identification result is inconsistent with the first identification result, but the similarity measure between the second identification result and the first identification result is less than or equal to a preset threshold, the fitness can be recalculated, and then the elite adversarial sample with the updated fitness can be added back to the adversarial sample population and applied to the next round of elite adversarial sample selection.

[0071] In some embodiments, updating the fitness of the elite adversarial sample includes: obtaining a gradient evaluation value of the elite adversarial sample using a preset gradient evaluation value algorithm based on the elite adversarial sample; and using the gradient evaluation value as the updated fitness of the elite sample.

[0072] In some embodiments, updating the fitness of the elite adversarial sample includes: calculating the gradient evaluation value based on the elite adversarial sample using the following formula; Among them, x i e represents the elite sample. i Let G(x) represent the regularized basis vector of the i-th dimension of the elite sample, δ represent the preset slight perturbation value, and G(x) represent the regularized basis vector of the i-th dimension of the elite sample. i ()) represents the gradient value of the elite adversarial example, (G(x) i +δe j () represents the first perturbation gradient value of the elite adversarial sample, (G(x) i +δe d ) represents the second perturbation gradient value of the elite adversarial example. This represents the gradient evaluation value of the elite sample; the gradient evaluation value is used as the fitness of the elite sample. Using a gradient evaluation method based on the finite difference method, the fitness function value can be treated as the gradient object, adding a slight perturbation to individuals in the new population. A gradient is generated based on the gradient formula, and then the difference between this gradient and the original gradient is calculated to obtain the new fitness.

[0073] In some embodiments, the method further includes: in response to determining that the second identification result is inconsistent with the first identification result and that the similarity measure between the second identification result and the first identification result is greater than a preset threshold, optimizing the elite adversarial sample and re-adding the optimized elite adversarial sample to the adversarial sample population.

[0074] In some embodiments, the above-mentioned optimization of the elite adversarial samples includes: obtaining optimized elite adversarial samples by using a crossover algorithm and / or a mutation algorithm based on the elite adversarial samples and the adversarial sample population.

[0075] In some embodiments, the crossover algorithm may include various crossover methods such as single-point crossover, multi-point crossover, and uniform crossover to calculate the offspring of elite individuals, and use these offspring as optimized elite adversarial samples. In some embodiments, the mutation algorithm may include replacing certain genes on the genome of selected elite individuals with other alleles on the genome to generate mutations and produce offspring of elite individuals, and use these offspring as optimized elite adversarial samples.

[0076] It should be noted that one or more embodiments of this application may be further described in the following ways:

[0077] First, a perturbation is added to at least one initial sample to obtain an adversarial sample population. The fitness of the adversarial samples in the adversarial sample population is calculated, and then the adversarial samples are screened based on the fitness to obtain at least one elite adversarial sample. The methods for generating the adversarial sample population and calculating the fitness are as described in step S101, and will not be repeated here.

[0078] Then, the aforementioned elite adversarial samples and their corresponding initial samples are input into the model to obtain the second recognition result and the first recognition result.

[0079] Finally, the second identification result is compared with the first identification result, and the elite adversarial sample is determined based on the identification result to determine whether it can be used as a target adversarial sample and whether the fitness needs to be updated or optimized.

[0080] Specifically, firstly, it is determined whether the second identification result is consistent with the first identification result. If the identification results are consistent, the elite adversarial sample is identified as the target adversarial sample; if the identification results are inconsistent, it is determined whether the similarity measure between the second identification result and the first identification result is less than a preset threshold.

[0081] In response to determining that the similarity metric is less than or equal to the preset threshold, the fitness of the elite adversarial sample is updated and the elite adversarial sample is re-added to the adversarial sample population. The method for updating the fitness is as described in step S105 and will not be repeated here.

[0082] In response to the determination that the similarity metric is greater than the preset threshold, the elite adversarial examples are optimized, and the optimized elite adversarial examples are re-added to the adversarial example population. The optimization method is as described in step S105, and will not be repeated here.

[0083] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0084] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0085] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides an adversarial sample generation device.

[0086] refer to Figure 2 The adversarial sample generation device includes:

[0087] The first generation module 11 is configured to add perturbations to at least one initial sample to obtain an adversarial sample population.

[0088] The second generation module 12 is configured to select a preset number of elite adversarial samples based on the fitness of the adversarial samples.

[0089] The first calculation module 13 is configured to input the initial sample corresponding to the elite adversarial sample into the target detection model to obtain a first recognition result, wherein the first recognition result is the real recognition result.

[0090] The second calculation module 14 is configured to input the elite adversarial sample into the target detection model to obtain a second recognition result;

[0091] The judgment module 15 is configured to determine the elite adversarial sample as a target adversarial sample in response to determining that the second identification result is consistent with the first identification result.

[0092] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0093] The apparatus of the above embodiments is used to implement the corresponding adversarial sample generation method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0094] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the adversarial sample generation method described in any of the above embodiments.

[0095] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0096] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0097] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0098] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0099] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0100] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0101] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0102] The electronic devices described above are used to implement the corresponding adversarial sample generation methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0103] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the adversarial sample generation method as described in any of the above embodiments.

[0104] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0105] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the adversarial sample generation method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0106] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0107] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0108] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0109] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for generating adversarial examples, characterized in that, include: Perturbation is added to at least one initial sample to obtain an adversarial sample population; the initial sample is an image training sample. Based on the fitness of the adversarial examples, a preset number of elite adversarial examples are selected. The initial sample corresponding to the elite adversarial sample is input into the target detection model to obtain the first recognition result, which is the true recognition result; The elite adversarial sample is input into the target detection model to obtain a second recognition result; In response to determining that the second identification result is consistent with the first identification result, the elite adversarial sample is determined to be a target adversarial sample; The method further includes: In response to determining that the second identification result is inconsistent with the first identification result, and that the similarity measure between the second identification result and the first identification result is less than or equal to a preset threshold, the fitness of the elite adversarial sample is updated and the elite adversarial sample is re-added to the adversarial sample population; The method further includes: In response to determining that the second identification result is inconsistent with the first identification result, and that the similarity measure between the second identification result and the first identification result is greater than a preset threshold, the elite adversarial sample is optimized, and the optimized elite adversarial sample is re-added to the adversarial sample population; The fitness calculation steps include: Based on the adversarial sample, the attack success rate and camouflage of the adversarial sample are obtained; Based on the attack success and the camouflage, the fitness of the adversarial sample is calculated using the following formula; ; in, This indicates the fitness of the adversarial example. This indicates the success rate of the adversarial sample attack. This indicates the camouflage of the adversarial sample. This indicates the preset spoofing weight.

2. The adversarial example generation method according to claim 1, characterized in that, The updating of the fitness of the elite adversarial examples includes: Based on the elite adversarial sample, the gradient evaluation value of the elite adversarial sample is obtained through a preset gradient evaluation value algorithm; The gradient evaluation value is used as the updated fitness of the elite adversarial sample.

3. The adversarial example generation method according to claim 2, characterized in that, The updating of the fitness of the elite adversarial examples includes: Based on the elite adversarial samples, the gradient evaluation value is calculated using the following formula; ; in, This refers to the elite adversarial sample. Let i represent the regularized basis vector of the i-th dimension of the elite adversarial sample. This represents the preset slight disturbance value. This represents the gradient value of the elite adversarial sample. This represents the first perturbation gradient value of the elite adversarial sample. This represents the second perturbation gradient value of the elite adversarial example. This represents the gradient evaluation value of the elite adversarial sample; The gradient evaluation value is used as the fitness of the elite adversarial sample.

4. The adversarial example generation method according to claim 1, characterized in that, The optimization of the elite adversarial sample includes: Based on the elite adversarial samples and the adversarial sample population, optimized elite adversarial samples are obtained through crossover and / or mutation algorithms.

5. An adversarial sample generation device, characterized in that, Used to implement the adversarial sample generation method according to any one of claims 1-4; The device includes: The first generation module is configured to add perturbations to at least one initial sample to obtain an adversarial sample population; the initial sample is an image training sample. The second generation module is configured to select a preset number of elite adversarial samples based on the fitness of the adversarial samples. The first calculation module is configured to input the initial sample corresponding to the elite adversarial sample into the target detection model to obtain a first recognition result, wherein the first recognition result is the true recognition result. The second calculation module is configured to input the elite adversarial sample into the target detection model to obtain a second recognition result; The judgment module is configured to determine the elite adversarial sample as a target adversarial sample in response to determining that the second identification result is consistent with the first identification result.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Defense method and defense device for black box attack model of speech recognition system

    CN110992934A

  • Fraud Detection Methods and Systems Based on Evolution-Based Black-Box Attack Models

    US20230186311A1