Adversarial sample generation method and device, equipment, medium and network training method

Abstract

The application discloses an adversarial sample generation method, device, equipment, medium and model training method, the adversarial sample generation method generates a first perturbation image by inputting an original image and a directional attack label into a conditional adversarial generation network; a kernel function is used to perform smoothing processing on the first perturbation image to obtain a second perturbation image; and the second perturbation image is superimposed with the original image to form an adversarial sample. The adversarial sample generation method provided by the application reduces the sensitivity of the generated adversarial sample to the attack model by introducing a kernel function to perform smoothing processing on the initially generated perturbation image, has good transfer attack performance, and in addition, the conditional adversarial generation network in the application contains a CAN network, so that the adversarial sample generated by the method has higher computational efficiency in a black box scene compared with a traditional gradient iteration-based method.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent application of automobiles, and particularly relates to an adversarial sample generation method, device, equipment, medium and network training method. BACKGROUND

[0002] An adversarial sample is a major robustness hidden danger faced by a deep learning model. An attacker can induce a deep learning model to make a mistake by adding a slight perturbation to a normal sample. Adversarial sample attacks can be divided into two categories: non-directional attacks and directional attacks. The former only needs to induce the model to make a mistake to achieve the attack goal, while the latter seeks to induce the model to output an error result expected by the attacker, which is more difficult and more threatening.

[0003] In the prior art, an adversarial sample is mainly generated by the following operations: first, an original image and an initial sample generated according to the original image are obtained; then, target detection is performed on a target object in the original image and the initial adversarial sample to obtain detection information; finally, a loss function of the target object is constructed according to the detection information, the original image and the initial adversarial sample, and the initial adversarial sample is adjusted based on the loss function of the target object to obtain a final adversarial sample. However, these adversarial sample generation methods are mostly based on gradient iteration, and the transfer attack performance and computational efficiency in a black box scenario are very low, which cannot be well applied to an autonomous driving scenario. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the application discloses an adversarial sample generation method, device, equipment, medium and network training method to solve the above problems.

[0005] To achieve the above object and other related objects, the application discloses an adversarial sample generation method, comprising:

[0006] inputting an original image and a directional attack label into a conditional adversarial generation network to generate a first perturbation image;

[0007] smoothing the first perturbation image by using a kernel function to obtain a second perturbation image;

[0008] superimposing the second perturbation image and the original image to form an adversarial sample.

[0009] In an optional embodiment of the application, before the inputting of the original image and the attack label into the conditional adversarial generation network to generate the first perturbation image, the method further comprises:

[0010] combining a generative adversarial network and a label mapping network to construct the conditional adversarial generation network, wherein the label mapping network is composed of multiple linear layers and multiple spectral normalization layers.

[0011] In an optional embodiment of the present application, the conditional adversarial generation network comprises an encoder and a decoder, wherein the encoder comprises a plurality of convolutional layers, a plurality of activation function layers and a plurality of batch normalization layers, and the decoder comprises a plurality of deconvolutional layers, a plurality of activation function layers and a plurality of batch normalization layers.

[0012] In an optional embodiment of the present application, the convolution kernel size of the deconvolutional layer is 2-3 and can be divided by a step.

[0013] In an optional embodiment of the present application, the input of the original image and the attack label into the conditional adversarial generation network is used to generate a first perturbed image, specifically comprising:

[0014] The encoder extracts features from the image to be processed to obtain a first feature vector;

[0015] The label mapping network extracts features from the directional attack label to obtain a conditional input vector;

[0016] The first feature vector and the conditional input vector are channel superimposed to obtain a second feature vector;

[0017] The second feature vector is decoded by the decoder to generate the first perturbed image.

[0018] In an optional embodiment of the present application, the kernel function is a uniform kernel function or a linear kernel function or a Gaussian kernel function.

[0019] To achieve the above object and other related objects, the present application further discloses an adversarial sample generation device, comprising:

[0020] A first perturbed image generation module is used to input an original image and a directional attack label into a conditional adversarial generation network to generate a first perturbed image;

[0021] A second perturbed image generation module is used to smooth the first perturbed image by using a kernel function to obtain a second perturbed image;

[0022] An adversarial sample generation module is used to superimpose the second perturbed image and the original image to form an adversarial sample.

[0023] To achieve the above object and other related objects, the present application further provides an electronic device, comprising:

[0024] One or more processors;

[0025] A storage device for storing one or more programs that, when executed by the one or more processors, cause the device to implement the method for generating an adversarial sample as described above.

[0026] To achieve the above object and other related objects, the present application also discloses a storage medium having computer readable instructions stored thereon, which, when executed by a processor of a computer, cause the computer to perform the method for generating an adversarial sample as described above.

[0027] To achieve the above object and other related objects, the present application also discloses a conditional adversarial generation network training method, comprising:

[0028] inputting an original image and a directed attack label into a conditional adversarial generation network to be trained to generate a first perturbed image;

[0029] smoothing the first perturbed image using a kernel function to obtain a second perturbed image;

[0030] superimposing the second perturbed image and the image to be processed to form an adversarial sample;

[0031] inputting the adversarial sample into a target detection model to output a target detection result;

[0032] calculating a loss function using the target detection result;

[0033] updating the conditional adversarial generation network to be trained using the loss function to obtain a trained conditional adversarial generation network.

[0034] In an optional embodiment of the present application, the loss function is calculated using the following formula:

[0035] L = λ1L obj + λ2L loc + λ3L cls , wherein L obj is a loss function representing whether a target object is contained, L loc is a loss function representing an object position, L cls is a loss function representing an object category, and λ i is a hyperparameter.

[0036] Advantages:

[0037] The method for generating an adversarial sample provided by the application generates a first perturbation image by inputting an original image and a targeted attack label into a conditional adversarial generation network; performs smoothing processing on the first perturbation image by using a kernel function to obtain a second perturbation image; and superimposes the second perturbation image and the original image to form an adversarial sample. In the process of generating an adversarial sample, the kernel function is introduced to perform smoothing processing on the initially generated perturbation image, which reduces the sensitivity of the generated adversarial sample to the attack model and has good transfer attack performance. In addition, the conditional adversarial generation network in the application contains a CAN network, so that the adversarial sample generated by the method has higher computational efficiency in a black-box scenario compared with the traditional gradient iteration-based method. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application. It is to be expressly understood that the drawings are included solely for purposes of illustration and that they are not to be construed as limiting the application. In the drawings:

[0039] Figure 1 A block diagram of a target detection system is shown for an exemplary embodiment of the application.

[0040] Figure 2 A flowchart of a method for generating an adversarial sample is shown for an exemplary embodiment of the application.

[0041] Figure 3 A flowchart of a method for generating an adversarial sample is shown for an exemplary embodiment of the application. Figure 2 A flowchart of step S21 in a specific embodiment.

[0042] Figure 4 A block diagram of an adversarial sample generation device is shown for an exemplary embodiment of the application.

[0043] Figure 5 A structural schematic diagram of a computer system of an electronic device suitable for implementing embodiments of the application.

[0044] Figure 6 A flowchart of a method for training a conditional adversarial generation network is shown for an exemplary embodiment of the application. DETAILED DESCRIPTION

[0045] Following, the advantages and effects of the present application can be easily understood by those skilled in the art from the description of the specific embodiments. The present application can also be implemented or applied by other different embodiments, and the details in the description can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0046] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concepts of the present application in a schematic manner, and only show the components related to the present application in the diagrams, not the number, shape and size of the components when actually implemented. The shapes, numbers and proportions of the components when actually implemented can be arbitrarily changed, and the layout pattern of the components can be more complex.

[0047] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details, and in other embodiments, the known structures and devices are shown in the form of block diagrams instead of details, to avoid making the embodiments of the present application difficult to understand.

[0048] The terms "first", "second", and the like in the description and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0049] Unless otherwise specified, the term "a plurality of" means two or more.

[0050] In the embodiments of the present disclosure, the character " / " represents an "or" relationship between the objects before and after it. For example, A / B represents: A or B.

[0051] The term "and / or" is a description of the association between objects, which means that there can be three relationships. For example, A and / or B, which means: A or B, or, A and B, the three relationships.

[0052] First of all, the adversarial sample refers to the input sample formed by deliberately adding subtle interference in the data set, which can cause the model to give an incorrect output with high confidence. It is a major robustness problem faced by deep learning. Adversarial samples are usually generated based on pictures that can be correctly identified by a perception system classifier, and then some small pixel value perturbations are added to the base to generate such adversarial samples that cannot be identified by the human eye. However, once they are sent into a well-designed deep network-based classifier, they can mislead the accurate classification, interfere with driving decisions, and thus expose the vulnerabilities of the perception system. Adversarial samples can be divided into non-directional attacks and directional attacks. Non-directional attacks only need to induce the model to make mistakes to achieve the attack goal. Directional attacks seek to induce the model to output the error results expected by the attacker, which is more difficult and more threatening.

[0053] Target detection systems are widely used in automatic driving, monitoring and other intelligent systems. Figure 1 An example embodiment of the present application is a target detection system 100 applied in the field of automatic driving, which comprises an image acquisition module 101, an image processing module 102 and a result output module 103. The image acquisition module 101 is arranged in front of an automatic driving vehicle to acquire road condition information of the driving environment. The image processing module 102 is a detection module capable of automatically importing image data sets and is constructed using data trained by a deep neural network. It can detect targets in the collected images, frame and classify the targets, and send the processed results to the result output module 103. The result output module 103 is used to combine the images into a video output in chronological order, thereby obtaining the real-time target detection scene in front of the current vehicle.

[0054] The target detection system will output incorrect results after being attacked by adversarial samples, such as some target detection systems may mistake a red light for a green light after being attacked by samples. Such errors are often fatal in the field of autonomous vehicles. Obviously, if the defects of the target detection system model cannot be found early, it is extremely detrimental to driving safety. Therefore, the quality of adversarial samples will directly determine the robustness of the target detection system,

[0055] The prior art generates an adversarial sample mainly by the following operation: first, an original image and an initial sample generated according to the original image are obtained; then, target detection is performed on a target object in the original image and the initial adversarial sample to obtain detection information; finally, a loss function of the target object is constructed according to the detection information, the original image and the initial adversarial sample, and the initial adversarial sample is adjusted based on the loss function of the target object to obtain a final adversarial sample. However, these methods for generating adversarial samples are mainly based on gradient iteration, and the migration attack performance and computational efficiency in a black box scene are very low, and cannot be well applied to an automatic driving scene. Therefore, how to better generate an adversarial sample applied to a fast target detection system is a problem that needs to be solved by the person skilled in the art.

[0056] Therefore, the present application provides an adversarial sample generation method to solve the above problems.

[0057] Please refer to Figure 2 , Figure 2 The flowchart of the adversarial sample generation method of an example embodiment of the present application is shown.

[0058] In combination with Figure 2 shown, the adversarial sample generation method of the present disclosure comprises:

[0059] Step S210, inputting an original image and a targeted attack label into a conditional adversarial generation network to generate a first perturbation image;

[0060] First of all, the conditional adversarial generation network provided by the present application is formed by adding a label mapping network to the generative adversarial network (GAN), wherein the label mapping network is composed of multiple linear layers and multiple spectral normalization layers. Through the mapping of different categories, the implicit vectors of different targets can be effectively learned, and these vectors containing category information are stored in different channels.

[0061] The conditional adversarial generation network provided by the present application specifically includes an encoder and a decoder, wherein the encoder includes multiple convolutional layers, multiple class activation function RELU layers and multiple batch normalization BN layers, and the decoder includes multiple deconvolutional layers, multiple class activation function RELU layers and multiple batch normalization BN layers.

[0062] Please refer to Figure 3 shown, in a specific embodiment, the inputting of the original image and the targeted attack label into the conditional adversarial generation network to generate the first perturbation image specifically comprises:

[0063] Step S301, the encoder extracts features from the original image to obtain a first feature vector;

[0064] The original image x is subjected to feature extraction by an encoder in the conditional adversarial generation network, and a first feature vector F is obtained.

[0065] In an embodiment, two convolutional layers can be used for feature extraction of the input image, and the feature extraction of the two convolutional layers is denoted as F1 and F2, respectively, and the final output of the encoder is F.

[0066] In step S302, the label mapping network performs feature extraction on the targeted attack label to obtain a conditional input vector.

[0067] The label mapping network uses one-hot encoding of the targeted attack label L as input to obtain a conditional input vector C [c1',..., c n '] of the targeted attack label L.

[0068] In step S303, the first feature vector and the conditional input vector are stacked in channels to obtain a second feature vector.

[0069] The first feature vector F and the conditional input vector are stacked in channels to form a second feature vector M.

[0070] In step S304, the second feature vector is decoded by the decoder to generate the first perturbed image.

[0071] The decoder decodes the second feature vector M to obtain the first perturbed image Δx, and the size of the first perturbed image is consistent with that of the original image.

[0072] It should be noted that the size of the convolution kernel of the deconvolution layer in the decoder provided by the present application is 2-3, preferably the size of the convolution kernel is 2, and the size of the convolution kernel is set to be divisible by the step size. The present application sets the size of the convolution kernel of the deconvolution layer to be small because in a conventional generative adversarial network, the size of the convolution kernel of the deconvolution layer is too large (5-7) or the size of the convolution kernel cannot be divided by the step size, which leads to uneven overlap of the deconvolution, resulting in a checkerboard effect. This overlapping pattern is reflected in two dimensions, and the uneven overlap is more serious in two dimensions. Therefore, to avoid the occurrence of the checkerboard effect, the present application sets the size of the convolution kernel to be small, and the size of the convolution kernel is set to be divisible by the step size.

[0073] It should be further noted that, in order to make the generated first perturbed image invisible to the naked eye, it is usually necessary to limit the perturbed image within a certain range, such as using a smooth projection method.

[0074] In step S220, the first perturbed image is smoothed using a kernel function to obtain a second perturbed image.

[0075] Smooth the first perturbation image Δx by using a kernel function to obtain a second perturbation image alf_def(Δx), wherein alf_def represents the kernel function.

[0076] First of all, it needs to be pointed out that if the first perturbation image generated in step S210 is not processed by using the kernel function, but is directly superimposed with the original image to generate the adversarial sample, the generated adversarial sample will have strong sensitivity to the model to be attacked and will not have good transfer performance. Therefore, the kernel function is used to smooth the first perturbation image in the present application.

[0077] It also needs to be pointed out that in an embodiment of the present application, the kernel function is a uniform kernel matrix or a linear kernel matrix or a Gaussian kernel matrix, and the calculation formulas are as follows:

[0078] 1) Uniform kernel matrix:

[0079] 2) Linear kernel matrix:

[0080] 3) Gaussian kernel matrix:

[0081] Wherein, i, j respectively represent the i-th row and the j-th column of the two-dimensional matrix of the image, and k is the maximum value of i and j.

[0082] Superimpose the second perturbation image and the image to be processed to form an adversarial sample.

[0083] It can be understood that in other embodiments of the present application, other kernel functions can also be used to process the first perturbation image.

[0084] In step S230, the second perturbation image is superimposed with the original image to form an adversarial sample.

[0085] Since the size of the second perturbation image generated in step S230 and the original image remains consistent, the original image and the second perturbation image can be pixel superimposed to form an adversarial sample x' = x + alf_def(Δx) in the present application.

[0086] In summary, the method for generating an adversarial sample provided by the application generates a first perturbation image by inputting an original image and a targeted attack label into a conditional adversarial generation network; smoothes the first perturbation image by using a kernel function to obtain a second perturbation image; and superimposes the second perturbation image and the original image to form an adversarial sample. Since the kernel function is introduced to smooth the initially generated perturbation image when the adversarial sample is generated, the sensitivity of the generated adversarial sample to the attack model is reduced, and the adversarial sample has good transfer attack performance. In addition, since the conditional adversarial generation network contains a CAN network, the adversarial sample generated by the method has higher computational efficiency in a black-box scenario compared with the traditional gradient iteration-based method.

[0087] The application further provides an adversarial sample generation device, Figure 4 A block diagram of the adversarial sample generation device 400 is shown for an exemplary embodiment of the application. The adversarial sample generation device 400 includes a first perturbation image generation module 401, a second perturbation image generation module 402, and an adversarial sample generation module 403. The first perturbation image generation module 401 is configured to input an original image and a targeted attack label into a conditional adversarial generation network to generate a first perturbation image; the second perturbation image generation module 402 is configured to smooth the first perturbation image by using a kernel function to obtain a second perturbation image; and the adversarial sample generation module 403 is configured to superimpose the second perturbation image and the original image to form an adversarial sample.

[0088] It should be noted that the adversarial sample generation device 400 provided by the above embodiments and the adversarial sample generation method provided by the above embodiments belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiments, which will not be described here. The adversarial sample generation device 400 provided by the above embodiments can be used to distribute the above functions to different functional modules for completion in actual applications, i.e., the internal structure of the system is divided into different functional modules to complete all or part of the functions described above, and this is not limited herein.

[0089] Embodiments of the application also provide an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the adversarial sample method provided in each of the above embodiments.

[0090] Figure 5 The structure of a computer system of an electronic device suitable for implementing embodiments of the application is shown. It should be noted that, Figure 5The computer system 500 of the electronic device shown is merely one example, and should not be taken as limiting the functionality or use of embodiments of the present application.

[0091] As shown in Figure 5 the computer system 500 includes a central processing unit (CPU) 501 that can perform various appropriate actions and processes in accordance with a program stored in a read-only memory (ROM) 502 or a program loaded from a storage section 508 into a random access memory (RAM) 503, such as performing the methods described in the above embodiments. Various programs and data required for the operation of the system are also stored in the RAM 503. The CPU 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0092] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as necessary. A removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read therefrom is installed into the storage section 508 as necessary.

[0093] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing a computer program for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from the removable recording medium 511. When the computer program is executed by the central processing unit (CPU) 501, various functions defined in the system of the present application are performed.

[0094] It should be noted that the computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable signal medium can include a data signal propagating in the baseband or as part of a carrier wave, which carries the computer readable computer program. Such a propagating data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit programs for use by or in connection with an instruction execution system, device or component. The computer program contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination of the above.

[0095] Please refer to Figure 6 The present application also discloses a conditional adversarial generation network training method. Figure 6 A flowchart of a conditional adversarial generation network training method in a disclosed embodiment of the present application is shown. In combination with Figure 5 As shown, the conditional adversarial generation network training method comprises:

[0096] Step 601, inputting an original image and a targeted attack label into a conditional adversarial generation network to be trained to generate a first perturbed image;

[0097] Step 602, performing smoothing processing on the first perturbed image using a kernel function to obtain a second perturbed image;

[0098] Step 603, superimposing the second perturbed image and the image to be processed to form an adversarial sample;

[0099] Step 604, inputting the adversarial sample into a target detection model to output a target detection result;

[0100] For an original image x with a list of labels y = [o1,..., o n ], where o i = [c i , x i , y i , w i , h i ] describes the class of an object on the image, the upper left corner coordinates, width and height. The target detection model F φ outputs the result y' = [o1',..., o n '], where o i ' = (c i ', x i ', y i ', w i ', h i ', p'), p' represents the confidence of the predicted target class.

[0101] Step 605, calculate the loss function using the target detection result;

[0102] In training, for a given pre-trained target detector F φ , the parameters of the generation model G θ are solved by solving the following optimization problem:

[0103]

[0104] s.t. ||G θ (x s , [c'1,..., c' n ])|| ∞ ≤ ∈

[0105] Where G θ (x s , [c'1,..., c' n ]) is the perturbed image Δx generated by the adversarial generation network, F φ (G θ (x s , [c'1,..., c' n ])+x s ) is the discrimination result obtained by the target detection model for the generated image, the conditional probability CE(F φ (G θ (x s , [c'1,..., c' n ])+x s ), [c'1,..., c' nThe difference between the discrimination result and the expected result of the targeted attack is described. In short, the overall optimization goal is to minimize the difference between the discrimination result of the target detector for the attack image generated by the generation model and the expected result of the targeted attack.

[0106] Specifically, the loss function is calculated by the following formula:

[0107] L = λ1L obj + λ2L loc + λ3L cls , wherein λ i is a hyperparameter, L obj is a loss function representing whether the target object is included, and the calculation formula is as follows:

[0108]

[0109] L loc is a loss function representing the object position, and the calculation formula is as follows:

[0110]

[0111] L cls is a loss function representing the object category, and the calculation formula is as follows:

[0112] , wherein d i and d' i represent the real target category and the predicted target category respectively.

[0113] Step 606: updating the conditional adversarial generation network to be trained by using the loss function to obtain a trained conditional adversarial generation network.

[0114] The conditional adversarial generation network is updated in a gradient descent manner by using the loss function obtained in step 605 to obtain a trained conditional adversarial generation network.

[0115] It should be noted that the flowchart and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logic functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0116] The units described in the embodiments of the present application can be implemented by software, or by hardware, or by a combination of software and hardware. The units described can also be implemented in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves.

Claims

1. An adversarial sample generation method, characterized in that, The method comprises the following steps: inputting an original image and an attack label into a conditional adversarial generation network to generate a first perturbation image; smoothing the first perturbation image by using a kernel function to obtain a second perturbation image; superimposing the second perturbation image and the original image to form an adversarial sample; inputting the adversarial sample into a target detection model to output a target detection result, wherein the target detection result comprises an object category, an object category confidence, an object coordinate, an object width and an object height; the object width and the object height are used to represent whether the target object is contained; the object category and the object category confidence are used to determine the target object category; the object coordinate is used to represent the object position; and a loss function is calculated by using the target detection result. Before the step of inputting the original image and the attack label into the conditional adversarial generation network to generate the first perturbation image, the method further comprises the following steps:

2. The method of claim 1, wherein, combining a generative adversarial network and a label mapping network to construct the conditional adversarial generation network, wherein the label mapping network is composed of multiple linear layers and multiple spectral normalization layers. The conditional adversarial generation network comprises an encoder and a decoder, wherein the encoder comprises multiple convolutional layers, multiple class activation function layers and multiple batch normalization layers, and the decoder comprises multiple deconvolutional layers, multiple class activation function layers and multiple batch normalization layers.

3. The method of claim 1, wherein, The size of the convolution kernel of the deconvolutional layer is 2-3 and can be divided by a step.

4. The method of claim 3, wherein, The step of inputting the original image and the attack label into the conditional adversarial generation network to generate the first perturbation image specifically comprises the following steps:

5. The method of claim 3, wherein, the encoder extracts features of the original image to obtain a first feature vector; the label mapping network extracts features of the attack label to obtain a conditional input vector; the first feature vector and the conditional input vector are superimposed in a channel to obtain a second feature vector; the second feature vector is decoded by using the decoder to generate the first perturbation image. The kernel function is a uniform kernel matrix, a linear kernel matrix or a Gaussian kernel matrix.

6. The method of claim 1, wherein, The method comprises the following steps:

7. An adversarial sample generation apparatus, comprising: a first perturbation image generation module is configured to input an original image and an attack label into a conditional adversarial generation network to generate a first perturbation image; a second perturbation image generation module is configured to smooth the first perturbation image by using a kernel function to obtain a second perturbation image; an adversarial sample generation module is configured to superimpose the second perturbation image and the original image to form an adversarial sample; wherein after the step of superimposing the second perturbation image and the original image to form the adversarial sample, the adversarial sample is inputted into a target detection model to output a target detection result, wherein the target detection result comprises an object category, an object category confidence, an object coordinate, an object width and an object height; the object width and the object height are used to represent whether the target object is contained; the object category and the object category confidence are used to determine the target object category; the object coordinate is used to represent the object position; and a loss function is calculated by using the target detection result. The method comprises the following steps:

8. An electronic device, comprising: one or more processors; ​ A storage device for storing one or more programs, which when executed by the one or more processors, cause the device to implement the method of any one of claims 1 to 6.

9. A storage medium, characterized by A computer readable medium having stored thereon computer readable instructions which, when executed by a processor of a computer, cause the computer to perform the method of any one of claims 1 to 6.

10. A conditional adversarial generative network training method, characterized in that, Comprising: inputting an original image and a targeted attack label into a conditional adversarial generative network to be trained to generate a first perturbation image; smoothing the first perturbation image using a kernel function to obtain a second perturbation image; superimposing the second perturbation image and a to-be-processed image to form an adversarial sample; inputting the adversarial sample into a target detection model to output a target detection result; the target detection result comprises: an object category, an object category confidence, an object coordinate, an object width and an object height; the object width and the object height are used to represent whether the target object is contained; the object category and the object category confidence are used to determine the target object category; and the object coordinate is used to represent the object position; calculating a loss function using the target detection result; updating the conditional adversarial generative network to be trained using the loss function to obtain a trained conditional adversarial generative network.

11. The conditional GAN training method of claim 10, wherein, the loss function is calculated using the following formula: L = λ1L obj + λ2L loc + λ3L cls where L obj is the loss function representing whether the target object is contained, L loc is the loss function representing the object position, L cls is the loss function representing the object category, and λ i is a hyperparameter.

Images