Adversarial sample detection method, device, equipment and computer-readable storage medium

Abstract

The present invention discloses an adversarial sample detection method, device, equipment and computer-readable storage medium, the method comprising: inputting a sample to be detected into an autoencoder network, using the output of a target intermediate layer in the autoencoder network as a potential feature of the sample to be detected, wherein the autoencoder network is trained based on a standard data set, and the label category of the sample in the standard data set is the same as the label category of the sample to be detected; comparing the potential feature with a judgment standard, and obtaining an adversarial sample detection result of the sample to be detected based on the comparison result, wherein the judgment standard is inputting a sample in the standard data set into the autoencoder network, and is obtained based on the output of the target intermediate layer. The present invention provides an effective adversarial sample detection method, thereby being able to avoid the impact of adversarial attacks on the classification or prediction tasks of neural networks, that is, being able to effectively defend against adversarial attacks, and also being able to avoid security risks caused by adversarial attacks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an adversarial sample detection method, device, equipment and computer-readable storage medium. Background Art

[0002] With the rapid development and widespread application of neural network technology, some attack methods against neural networks have emerged. Adversarial attack is a common attack method against neural networks. It adds a small amount of noise to the input data, forcing the model to make wrong judgments without significantly changing the original data. For example, by adding carefully prepared perturbation noise to the image, the classifier is misclassified, or by replacing certain words in a sentence with synonyms, the sentiment classification is wrong. Adversarial attacks have major safety hazards in autonomous driving, face recognition, and financial modeling. For example, by adding noise to the red light image taken by an autonomous driving vehicle, the image classifier that recognizes traffic lights mistakenly recognizes the red light as a green light, which may cause serious traffic accidents. Therefore, how to provide an effective detection method for adversarial samples to exclude adversarial samples and avoid safety hazards has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the present invention is to provide an adversarial sample detection method, device, system and computer-readable storage medium, aiming to solve how to provide an effective detection method for adversarial samples to eliminate adversarial samples and avoid security risks, which has become a problem to be solved urgently.

[0004] To achieve the above object, the present invention provides a method for detecting adversarial samples, the method comprising the following steps:

[0005] Inputting a sample to be detected into an autoencoder network, and using the output of a target intermediate layer in the autoencoder network as a potential feature of the sample to be detected, wherein the autoencoder network is trained based on a standard data set, and the label category of the sample in the standard data set is the same as the label category of the sample to be detected;

[0006] The potential features are compared with the judgment criteria, and the adversarial sample detection result of the sample to be detected is obtained based on the comparison result, wherein the judgment criteria is obtained by inputting samples in the standard data set into the autoencoder network and based on the output of the target intermediate layer.

[0007] Optionally, before the step of inputting the sample to be detected into the autoencoder network and using the output of the target intermediate layer in the autoencoder network as the potential feature of the sample to be detected, the step further includes:

[0008] Inputting the standard data set into the encoder of the network to be trained to obtain an encoding result;

[0009] Inputting the encoding result into the polarization hash layer in the network to be trained to obtain a hash result, and inputting the encoding result into the decoder in the network to be trained to obtain a decoding result;

[0010] Calculating an encoding loss value based on the hash result and a preset random hash code, and calculating a reconstruction loss value based on the standard data set and the decoding result;

[0011] The network to be trained is updated based on the encoding loss value and the reconstruction loss value to train the network to be trained, and the network to be trained after the training is completed is used as the autoencoding network.

[0012] Optionally, the step of inputting the encoding result into the polarization hash layer in the network to be trained to obtain a hash result includes:

[0013] Inputting the encoding result into the polarization hash layer in the network to be trained to obtain a polarization vector;

[0014] A hash result corresponding to the polarization vector is determined according to the positive and negative signs of the values ​​of each element of the polarization vector.

[0015] Optionally, the step of calculating the encoding loss value based on the hash result and a preset random hash code includes:

[0016] A Hamming distance between the hash result and the preset random hash code is calculated, and the Hamming distance is used as a coding loss value.

[0017] Optionally, before the step of calculating the encoding loss value based on the hash result and a preset random hash code, the step further includes:

[0018] The preset random hash code corresponding to the standard data set is generated based on a preset hash code generation method, wherein the Hamming distance between the preset random hash code and the random hash code corresponding to the target label category is greater than a preset distance, and the target label category is a label category different from the label category of the standard data set.

[0019] Optionally, the step of comparing the potential feature with the determination criterion and obtaining the adversarial sample detection result of the sample to be detected based on the comparison result includes:

[0020] Calculating the difference between the potential feature and the determination criterion, and detecting whether the difference is less than a preset threshold;

[0021] If the difference value is less than the preset threshold, determining that the sample to be detected is a normal sample;

[0022] If the difference value is greater than or equal to the preset threshold, the sample to be detected is determined to be an adversarial sample.

[0023] Optionally, when the target intermediate layer is the output layer of the encoder in the autoencoder network, the potential feature is the encoding vector of the sample to be detected, and the determination criterion is the average result of the encoding vector corresponding to each sample in the standard data set.

[0024] The step of calculating the difference value between the potential feature and the determination criterion comprises:

[0025] A vector distance value between the potential feature and the determination criterion is calculated, and the vector distance value is used as a difference value between the potential feature and the determination criterion.

[0026] To achieve the above object, the present invention provides an adversarial sample detection device, the device comprising:

[0027] An input module, used for inputting a sample to be detected into an autoencoder network, and using the output of the intermediate layer in the target of the autoencoder network as a potential feature of the sample to be detected, wherein the autoencoder network is trained based on a standard data set, and the label category of the sample in the standard data set is the same as the label category of the sample to be detected;

[0028] A comparison module is used to compare the potential features with the judgment criteria and obtain the adversarial sample detection result of the sample to be detected based on the comparison result, wherein the judgment criterion is obtained by inputting the samples in the standard data set into the autoencoder network and based on the output of the target intermediate layer.

[0029] To achieve the above-mentioned purpose, the present invention also provides an adversarial sample detection device, which includes: a memory, a processor, and an adversarial sample detection program stored in the memory and executable on the processor, wherein the adversarial sample detection program implements the steps of the adversarial sample detection method described above when executed by the processor.

[0030] In addition, to achieve the above objectives, the present invention also proposes a computer-readable storage medium, on which an adversarial sample detection program is stored, and when the adversarial sample detection program is executed by a processor, the steps of the adversarial sample detection method described above are implemented.

[0031] In the present invention, by inputting the sample to be detected into the autoencoder network, and extracting the output of the target middle layer of the autoencoder network as the potential feature of the sample to be detected, the potential feature is compared with the judgment standard to obtain the adversarial sample detection result of the sample to be detected. Since the autoencoder network is trained by using a standard data set with the same label category as the sample to be detected, and the judgment standard is obtained by inputting the standard data set into the autoencoder network and extracting the output of the target middle layer, the judgment standard contains the main features of the normal sample of the label category, and the potential feature contains the main features of the sample to be detected. The potential feature is compared with the judgment standard. If the difference between the two is smaller, the difference between the sample to be detected and the normal sample is smaller, and then the sample to be detected is more likely to belong to the label category, and vice versa, it is less likely to belong to the label category. Therefore, according to the result of the difference comparison, the detection result of whether the sample to be detected is an adversarial sample can be obtained. That is, the present invention provides an effective detection method for adversarial samples, which can avoid the impact of adversarial attacks on the classification or prediction tasks of neural networks, that is, it can effectively defend against adversarial attacks, and can also avoid the safety hazards caused by adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention;

[0033] Figure 2 A schematic diagram of the process of the first embodiment of the adversarial sample detection method of the present invention;

[0034] Figure 3 It is a functional schematic module diagram of a preferred embodiment of the adversarial sample detection device of the present invention.

[0035] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0036] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0037] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0038] It should be noted that the adversarial sample detection device in the embodiment of the present invention can be a smart phone, a personal computer, a server and other devices, and no specific limitation is made here.

[0039] like Figure 1As shown, the adversarial sample detection device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0040] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation on the adversarial sample detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0041] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an adversarial sample detection program. The operating system is a program that manages and controls the hardware and software resources of the device, and supports the operation of the adversarial sample detection program and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used to communicate data with the client; the network interface 1004 is mainly used to establish a communication connection with the server; the processor 1001 can be used to call the adversarial sample detection program stored in the memory 1005 and perform the following operations:

[0042] Inputting a sample to be detected into an autoencoder network, and using the output of a target intermediate layer in the autoencoder network as a potential feature of the sample to be detected, wherein the autoencoder network is trained based on a standard data set, and the label category of the sample in the standard data set is the same as the label category of the sample to be detected;

[0043] The potential features are compared with the judgment criteria, and the adversarial sample detection result of the sample to be detected is obtained based on the comparison result, wherein the judgment criteria is obtained by inputting samples in the standard data set into the autoencoder network and based on the output of the target intermediate layer.

[0044] Furthermore, before the step of inputting the sample to be detected into the autoencoder network and using the output of the target intermediate layer in the autoencoder network as the potential feature of the sample to be detected, the processor 1001 may also be used to call the adversarial sample detection program stored in the memory 1005 to perform the following operations:

[0045] Inputting the standard data set into the encoder of the network to be trained to obtain an encoding result;

[0046] Inputting the encoding result into the polarization hash layer in the network to be trained to obtain a hash result, and inputting the encoding result into the decoder in the network to be trained to obtain a decoding result;

[0047] Calculating an encoding loss value based on the hash result and a preset random hash code, and calculating a reconstruction loss value based on the standard data set and the decoding result;

[0048] The network to be trained is updated based on the encoding loss value and the reconstruction loss value to train the network to be trained, and the network to be trained after the training is completed is used as the autoencoding network.

[0049] Furthermore, the step of inputting the encoding result into the polarization hash layer in the network to be trained to obtain a hash result includes:

[0050] Inputting the encoding result into the polarization hash layer in the network to be trained to obtain a polarization vector;

[0051] A hash result corresponding to the polarization vector is determined according to the positive and negative signs of the values ​​of each element of the polarization vector.

[0052] Furthermore, the step of calculating the encoding loss value based on the hash result and a preset random hash code includes:

[0053] A Hamming distance between the hash result and the preset random hash code is calculated, and the Hamming distance is used as a coding loss value.

[0054] Furthermore, before the step of calculating the encoding loss value based on the hash result and the preset random hash code, the processor 1001 may also be used to call the adversarial sample detection program stored in the memory 1005 to perform the following operations:

[0055] The preset random hash code corresponding to the standard data set is generated based on a preset hash code generation method, wherein the Hamming distance between the preset random hash code and the random hash code corresponding to the target label category is greater than a preset distance, and the target label category is a label category different from the label category of the standard data set.

[0056] Furthermore, the step of comparing the potential features with the judgment criteria and obtaining the adversarial sample detection result of the sample to be detected based on the comparison result includes:

[0057] Calculating the difference between the potential feature and the determination criterion, and detecting whether the difference is less than a preset threshold;

[0058] If the difference value is less than the preset threshold, determining that the sample to be detected is a normal sample;

[0059] If the difference value is greater than or equal to the preset threshold, the sample to be detected is determined to be an adversarial sample.

[0060] Furthermore, when the target intermediate layer is the output layer of the encoder in the autoencoder network, the potential feature is the encoding vector of the sample to be detected, and the judgment criterion is the average result of the encoding vector corresponding to each sample in the standard data set.

[0061] The step of calculating the difference value between the potential feature and the determination criterion comprises:

[0062] A vector distance value between the potential feature and the determination criterion is calculated, and the vector distance value is used as a difference value between the potential feature and the determination criterion.

[0063] Based on the above structure, various embodiments of adversarial sample detection methods are proposed.

[0064] Reference Figure 2 , Figure 2 This is a flowchart of the first embodiment of the adversarial sample detection method of the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here. The execution subjects of each embodiment of the adversarial sample detection method of the present invention may be devices such as smart phones, personal computers, and servers. For ease of description, the execution subjects are omitted in the following embodiments. In this embodiment, the adversarial sample detection method includes:

[0065] Step S10, inputting the sample to be detected into the autoencoder network, and using the output of the target intermediate layer in the autoencoder network as the potential feature of the sample to be detected, wherein the autoencoder network is trained based on a standard data set, and the label category of the sample in the standard data set is the same as the label category of the sample to be detected;

[0066] Adversarial samples refer to samples that have noise added to normal samples to interfere with the output of the neural network. The neural network may recognize the adversarial sample as a completely opposite result from the normal sample. For example, a normal sample is an image of a cat. The neural network is a network trained with a large number of data sets. Therefore, it is likely to classify the normal sample as a cat. The adversarial sample is a normal sample with some noise added to it. This noise may not be perceived by the human eye. In other words, the adversarial sample seen by the human eye is still a picture of a cat, but the neural network is likely to classify the adversarial sample as a dog. In other words, the adversarial attacker deceives the neural network by adding noise to the normal sample.

[0067] In this embodiment, adversarial sample detection is to detect whether the sample is an adversarial sample. In practical applications, sample data that needs to be determined whether it is an adversarial sample can be used as a sample to be detected. In order to detect adversarial samples, in this embodiment, an autoencoder network can be pre-trained for each label category according to the specific classification task of the neural network. The autoencoder network is a type of neural network that can try to copy the input to the output after training. The network includes two parts: an encoder represented by a function h=f(x) and a decoder r=g(h) that generates reconstruction. For example, the specific classification task of the neural network is to classify images. There are two label categories, cat and dog. Then, a standard data set with a label category of cat can be used to train an autoencoder network, and a standard data set with a label category of dog can be used to train an autoencoder network. The standard data set refers to a data set in which all samples in the data set are normal samples. When performing adversarial sample detection, according to the label category to which the sample to be detected belongs, the autoencoder network corresponding to the label category is selected to detect the sample to be detected. For example, if the label category of the sample to be detected is dog, the autoencoder network trained with the standard data set with the label category of dog is selected to detect the sample to be detected.

[0068] It should be noted that if, before training the neural network, training samples for training the neural network are to be selected, adversarial sample detection needs to be performed on the selected samples to determine whether the selected samples are adversarial samples, then the selected samples are used as the samples to be tested, and the label category of the samples to be tested can be obtained by manual annotation. If, after training the neural network, the neural network is used to perform classification prediction on the samples to be classified, adversarial sample detection needs to be performed on the samples to be classified to determine whether the classification prediction results of the neural network are credible, then the samples to be classified are used as the samples to be tested, and the label category of the samples to be tested can be the category label obtained by the classification prediction of the neural network.

[0069] After determining the sample to be detected and the autoencoding network to detect the sample to be detected, the sample to be detected can be input into the autoencoding network, that is, the autoencoding network is called to encode and decode the sample to be detected. Specifically, the sample to be detected can be input into the encoder of the autoencoding network to obtain the encoding result, and then the encoding result is input into the decoder of the autoencoding network to obtain the decoding result. In the autoencoding network, except for the input layer of the encoder and the output layer of the decoder, the intermediate network layers are called intermediate layers, that is, the hidden layers of the encoder, the output layer of the encoder, the input layer of the decoder, and the hidden layers of the decoder are all called intermediate layers.

[0070] In the process of inputting the sample to be detected into the autoencoder network for processing, the output of the target intermediate layer of the autoencoder network can be extracted as the potential feature of the sample to be detected. Among them, the target intermediate layer can be one or more of the intermediate layers of the autoencoder network, and the target intermediate layer can be pre-set as which one or several intermediate layers. According to the characteristics of the autoencoder network, the encoder will extract the features of the sample to be detected, encode the sample to be detected according to the features, and the decoder will restore the features according to the encoding, and finally obtain an output result close to the input data, that is, the output of each intermediate layer of the autoencoder network contains the features of different dimensions of the sample to be detected, and these features are embodied in the form of vectors or encodings, so they are a kind of potential features. Furthermore, when multiple intermediate layers are selected as the target intermediate layers, the potential features of the sample to be detected are more abundant, thereby providing a richer detection basis for the detection of adversarial samples.

[0071] Furthermore, according to the characteristics of the autoencoder network, each intermediate layer in the encoder of the autoencoder network can remove redundant features in the sample to be detected layer by layer and retain the main features of the sample to be detected. Therefore, one or several intermediate layers in the encoder close to the encoder output layer can be used as target intermediate layers, and the features output by these target intermediate layers can be used as potential features of the sample to be detected, so that most of the potential features are the main features of the sample to be detected, that is, the main features related to the category to which the sample to be detected belongs are retained.

[0072] Step S20, compare the potential features with the judgment criteria, and obtain the adversarial sample detection result of the sample to be detected based on the comparison result, wherein the judgment criterion is obtained by inputting the samples in the standard data set into the autoencoder network and based on the output of the target intermediate layer.

[0073] After extracting the potential features of the sample to be detected, the potential features can be compared with the preset judgment criteria. Among them, the judgment criteria can be pre-set, and the data form of the judgment criteria can be the same as the data form of the potential features. For example, if the potential features are in vector form, the judgment criteria are also in vector form. After the autoencoder network is trained with a standard data set, the standard data set is input into the autoencoder network, and the output of the target middle layer of the autoencoder network is used as the judgment criterion. It should be noted that the standard data set includes multiple sample data, then the target middle layer output corresponding to each sample data can be averaged to obtain the output of the standard data set in the target middle layer; when there are multiple target middle layers, for each target middle layer, the output of each sample data corresponding to the target middle layer is averaged to obtain the output of the standard data set in the target middle layer, then the judgment criterion includes the judgment criterion corresponding to each target middle layer.

[0074] After obtaining the difference comparison result, determine whether the sample to be detected is an adversarial sample according to the difference comparison result, and obtain the adversarial sample detection result of the sample to be detected. Specifically, the sample to be detected is manually marked or predicted by the neural network as the target label category, the potential feature includes the main features of the sample to be detected after removing redundant information, and the judgment standard includes the main features of the normal sample belonging to the target label category; if the potential feature is closer to the judgment standard, it means that the sample to be detected is closer to the normal sample belonging to the target label category, that is, it means that the possibility that the sample to be detected belongs to the target label category is greater, and further indicates that when the difference between the potential feature and the judgment standard is smaller, the sample to be detected is more likely to be a normal sample. If the difference between the potential feature and the judgment standard is greater, it means that the difference between the sample to be detected and the normal sample belonging to the target label category is greater, that is, it means that the possibility that the sample to be detected belongs to the target label category is smaller, and further indicates that when the difference between the potential feature and the judgment standard is greater, the sample to be detected is more likely to be an adversarial sample.

[0075] Further, based on the above principle, step S20 includes:

[0076] Step S201, calculating the difference between the potential feature and the determination criterion, and detecting whether the difference is less than a preset threshold;

[0077] Furthermore, the difference value between the potential feature and the judgment criterion can be calculated, and whether the difference value is less than a preset threshold value can be detected. The preset threshold value can be set according to specific needs, or can be set based on experimental data obtained by experiments in specific neural network classification tasks. Specifically, when the data form of the potential feature and the judgment criterion is a vector, the difference value can be a vector distance value calculated between the potential feature and the judgment criterion.

[0078] If there is only one target intermediate layer, then the potential feature and the judgment criterion are both a vector, and the vector distance value of the two vectors can be calculated and used as the result of the difference comparison.

[0079] If there are multiple target intermediate layers, then the potential feature includes multiple vectors, one vector corresponds to one target intermediate layer, and the judgment standard also includes multiple vectors, and similarly, one vector corresponds to one target intermediate layer. At this time, the vector distance value of each potential feature and the judgment standard corresponding to the target intermediate layer can be calculated to obtain the vector distance value corresponding to each potential feature, and the value obtained by calculating the average value of the multiple vector distance values ​​is used as the difference value between the potential feature and the judgment standard. Furthermore, when averaging the vector distance values, weights can be added, that is, weighted averaging is performed; the weights can be set according to the importance of the features that can be extracted by each target intermediate layer. When the features extracted by the target intermediate layer play a greater role in the detection of adversarial samples, it is considered that the target intermediate layer is more important, and a larger weight is set for the target intermediate layer. Otherwise, a smaller weight is set. For example, according to the characteristics of the autoencoder, each layer of the encoder removes redundant features of the sample layer by layer, and finally retains the main features of the sample. Therefore, the intermediate layer of the encoder close to the encoder output layer (including the encoder output layer) contains fewer redundant features and more main features than the intermediate layer close to the input layer. Therefore, a larger weight can be set for the intermediate layer of the encoder close to the encoder output layer, that is, when performing difference comparison, the features extracted by the intermediate layer with a larger weight are used as the main comparison object. Therefore, the essential difference in category attributes between the sample to be detected and the standard data set can be more accurately determined.

[0080] The vector distance value may be calculated using a commonly used method for calculating the distance between vectors, such as Euclidean distance, Hamming distance, etc., which is not limited here.

[0081] Step S202: if the difference value is less than the preset threshold, determining that the sample to be detected is a normal sample;

[0082] Step S203: If the difference value is greater than or equal to the preset threshold, it is determined that the sample to be detected is an adversarial sample.

[0083] If the detected difference value is smaller than the preset threshold, it is determined that the difference between the potential feature and the judgment standard is small, and the sample to be detected is determined to be a normal sample; conversely, if the difference value is greater than or equal to the preset threshold, it is determined that the difference between the potential feature and the judgment standard is too large, and the sample to be detected is determined to be an adversarial sample, otherwise, the sample to be detected is determined to be a normal sample.

[0084] Before training the neural network, it is necessary to select training samples for training the neural network. When adversarial sample detection is performed on the selected samples, if the sample to be selected is detected as an adversarial sample, the sample to be selected can be excluded, that is, the sample is not used to train the neural network to avoid the sample affecting the training effect of the neural network; if the sample to be selected is detected as a normal sample, the sample can be used to train the neural network. After the neural network is trained, the neural network is used to perform classification prediction on the sample to be classified. It is necessary to perform adversarial sample detection on the sample to be classified to determine whether the classification prediction result of the neural network is credible. If the sample to be classified is detected as an adversarial sample, it means that the classification prediction result of the neural network is not necessarily credible. At this time, the classification prediction result of the neural network can be selected not to be adopted to avoid adverse effects; if the sample to be classified is detected as a normal sample, the classification prediction result of the neural network can be adopted.

[0085] In this embodiment, by inputting the sample to be detected into the autoencoder network, and extracting the output of the target middle layer of the autoencoder network as the potential feature of the sample to be detected, the potential feature is compared with the judgment standard to obtain the adversarial sample detection result of the sample to be detected. Since the autoencoder network is trained by using a standard data set with the same label category as the sample to be detected, and the judgment standard is obtained by inputting the standard data set into the autoencoder network and extracting the output of the target middle layer, the judgment standard contains the main features of the normal sample of the label category, and the potential feature contains the main features of the sample to be detected. The potential feature is compared with the judgment standard. If the difference between the two is smaller, the difference between the sample to be detected and the normal sample is smaller, and the sample to be detected is more likely to belong to the label category, and vice versa, it is less likely to belong to the label category. Therefore, according to the result of the difference comparison, the detection result of whether the sample to be detected is an adversarial sample can be obtained. That is, this embodiment provides an effective detection method for adversarial samples, which can avoid the impact of adversarial attacks on the classification or prediction tasks of neural networks, that is, it can effectively defend against adversarial attacks, and can also avoid security risks caused by adversarial attacks.

[0086] Furthermore, based on the above first embodiment, a second embodiment of the adversarial sample detection method of the present invention is proposed. In this embodiment, the adversarial sample detection method further includes:

[0087] Step S30, inputting the standard data set into the encoder in the network to be trained to obtain an encoding result;

[0088] Furthermore, before using the autoencoder network to detect adversarial samples, the autoencoder network needs to be trained. In this embodiment, a to-be-trained autoencoder network (hereinafter referred to as the to-be-trained network) can be pre-set. Compared with the structure of the conventional autoencoder network, the to-be-trained network also includes a polarization hash layer. The polarization hash layer is connected after the encoder like the decoder, that is, the encoder output data will be input to the decoder for decoding, and will also be input to the polarization hash layer for polarization hash processing. The polarization hash layer may include at least one fully connected layer, which is used to polarize the input data to obtain a polarization vector. The polarization processing refers to making the output data far away from 0. The number of elements in the polarization vector output by the fully connected layer is pre-set; the polarization hash layer also includes a hash channel connected after the fully connected layer, and a hash channel corresponds to an element in the polarization vector, which is used to hash the elements in the polarization vector to obtain a hash result, that is, to convert the elements in the polarization vector to 1 or -1.

[0089] The standard data set is input into the encoder of the network to be trained to obtain the encoding result. Specifically, each sample data in the standard data set is input into the encoder to obtain the encoding result of each sample data. The data form of the encoding result can be a high-dimensional vector.

[0090] Step S40, inputting the encoding result into the polarization hash layer in the network to be trained to obtain a hash result, and inputting the encoding result into the decoder in the network to be trained to obtain a decoding result;

[0091] The encoding result is input into the polarization hash layer for polarization hash processing to obtain a hash result. Specifically, the encoding result of each sample data is input into the polarization hash layer to obtain a hash result corresponding to each sample data. The encoding result is input into the decoder in the network to be trained to obtain a decoding result. The process of decoding the encoding result by the decoder is not described in detail here. Specifically, the encoding result of each sample data is input into the decoder to obtain a decoding result corresponding to each sample data.

[0092] Furthermore, the step of inputting the encoding result into the polarization hash layer in the network to be trained to obtain the hash result in step S40 includes:

[0093] Step S401, inputting the encoding result into the polarization hash layer in the network to be trained to obtain a polarization vector;

[0094] Step S402: Determine a hash result corresponding to the polarization vector according to the positive and negative signs of each element value of the polarization vector.

[0095] The encoding result is input into the polarization hash layer in the network to be trained, that is, the fully connected layer in the polarization hash layer is called to polarize the encoding result to obtain a polarization vector. After obtaining the polarization vector, each element value is input into the corresponding hash channel, and the hash channel determines the hash code corresponding to the element value according to the positive and negative signs of the element value. Specifically, when the element value is a negative number, the hash code corresponding to the element is determined to be -1, and when the element value is a positive number, the hash code corresponding to the element is determined to be 1. After determining the hash code corresponding to each element, a hash result composed of each hash code is obtained, that is, each element in the hash result is 1 or -1.

[0096] Step S50, calculating an encoding loss value based on the hash result and a preset random hash code, and calculating a reconstruction loss value based on the standard data set and the decoding result;

[0097] After obtaining the hash result, the coding loss value is calculated according to the hash result and the preset random hash code. Among them, the preset random hash code is a hash code randomly generated in advance, and the value range of this hash code element is {1, -1}, and the number of elements is the same as the number of polarization vector elements output by the fully connected layer of the polarization hash layer. The preset random hash code is set to correspond to the label category of the standard data set, that is, each label category corresponds to a preset random hash code, and the preset random hash code for each category is different. Specifically, the difference between the hash result of each sample data and the preset random hash code can be calculated separately, and then the differences are averaged to obtain the coding loss value. The difference between the hash result and the preset random hash code can be calculated using the Hamming distance calculation method, which will not be described in detail here.

[0098] In addition, after obtaining the decoding result, the reconstruction loss value is calculated based on the standard data set and the decoding result. The calculation method of the reconstruction loss value can adopt the calculation method of the conventional autoencoder network reconstruction loss value, which will not be described in detail here.

[0099] Furthermore, the step of calculating the encoding loss value based on the hash result and the preset random hash code in step S50 includes:

[0100] Step S501, calculating the Hamming distance between the hash result and the preset random hash code, and using the Hamming distance as a coding loss value.

[0101] After obtaining the hash result, the Hamming distance between the hash result and the preset random hash code can be calculated, and the Hamming distance can be used as the coding loss value. Specifically, the Hamming distance between the hash result corresponding to each sample data and the preset random hash code can be calculated respectively, and then the Hamming distances corresponding to each sample data can be averaged, and the average result can be used as the coding loss value. Among them, the calculation method of the Hamming distance can refer to the existing calculation method, which will not be described in detail here.

[0102] Step S60, updating the network to be trained based on the encoding loss value and the reconstruction loss value to train the network to be trained, and using the trained network as the autoencoder network.

[0103] After the encoding loss value and the reconstruction loss value are calculated, the network to be trained can be updated according to the encoding loss value and the reconstruction loss value to train the network to be trained. Specifically, the encoding loss value and the reconstruction loss value can be weighted and summed to obtain a total loss value, and the weight of the weighted summation can be preset according to specific needs; whether the network to be trained has converged can be judged according to the total loss value, specifically, whether the total loss value is less than a preset threshold value can be judged, if less than, it is determined that the network to be trained has converged, otherwise, it is determined that it has not converged; if the network to be trained has not converged, the network to be trained can be updated according to the total loss value, specifically, the gradient value of each model parameter of the network to be trained can be calculated based on the total loss value according to the chain rule, and each model parameter can be updated by using the gradient value to update the network to be trained; then the standard data set is input into the updated network to be trained to obtain a new encoding loss value and reconstruction loss value, and then it is judged whether the updated network to be trained has converged; until the convergence of the network to be trained is detected, the updated network to be trained can be used as a trained auto-encoding network.

[0104] Furthermore, when it is detected according to the total loss value that the network to be trained has not converged, the specific process of updating the network to be trained may also be:

[0105] For the polarization vector output by the fully connected layer in the polarization hash layer, a polarization loss function corresponding to each element in the polarization vector is preset. For example, for the target element in the polarization vector, the following polarization loss function is used:

[0106] L(v, t) = max(mv*t, 0)

[0107] Among them, L represents the polarization loss function, m is the parameter of the polarization hash layer (initialized before training), m in each polarization loss function is independent, v is the target element in the polarization vector, and t is the hash code of the position corresponding to the target element in the preset random hash code.

[0108] For the polarization vector corresponding to the standard data set, the values ​​of each element of the polarization vector are respectively input into the corresponding polarization loss function to calculate the loss value, and the gradient values ​​of each parameter (including m) of the polarization hash layer and encoder are calculated using the loss value, and the parameters of the polarization hash layer and encoder are updated according to the gradient value. After the update, the gradient values ​​of each parameter of the decoder and encoder are calculated according to the reconstruction loss value, and the decoder and encoder are updated according to the gradient value, thereby completing the update of the entire autoencoder network.

[0109] In this embodiment, by adding a polarization hash layer to the autoencoder network, adding the encoding loss value of the polarization hash layer on the basis of the reconstruction loss value, and reducing the encoding loss value and the reconstruction loss value by training the autoencoder network, the result of encoding the sample data using the encoder and the polarization hash layer can meet the requirement of being as close as possible to the preset random hash code. The purpose of this is that the preset random hash code is randomly generated. Compared with ordinary category labeling, the random hash code is much more complicated. A large amount of sample data belonging to the target label category is used to train the autoencoder network, so that the autoencoder network can continuously learn and extract sample data of the target label category. More essential features can be used to more accurately distinguish the features of sample data from other label categories. That is, the hyperplane learned by the autoencoder network to divide different categories is more accurate. Such an autoencoder network is more sensitive to trace noise, so that once there is some noise interference, it will be recognized by the autoencoder network. That is, when normal samples and adversarial samples are input into the autoencoder network, the potential features extracted are different. Even if the difference is very small, the trained autoencoder network can highlight it, and then it can achieve the difference between the potential features of the sample to be detected and the judgment criteria, and obtain accurate adversarial sample detection results.

[0110] That is, by adding a polarized hash layer to the autoencoder network and training the autoencoder network by using preset random hash codes as labels, adversarial samples with smaller added noise and more difficult to identify can also be accurately detected, thereby further improving the accuracy of adversarial sample detection, further avoiding the impact of adversarial attacks on the classification or prediction tasks of neural networks, and avoiding security risks caused by adversarial attacks.

[0111] Furthermore, the adversarial sample detection method further includes:

[0112] Step S70, generating the preset random hash code corresponding to the standard data set based on a preset hash code generation method, wherein the Hamming distance between the preset random hash code and the random hash code corresponding to the target label category is greater than a preset distance, and the target label category is a label category different from the label category of the standard data set.

[0113] In order to further improve the accuracy of the essential features extracted by the autoencoder network and further distinguish the features of sample data of other label categories, in one embodiment, a preset hash code generation method can be used in advance to randomly generate hash codes corresponding to each label category, and compare the hash codes of each label category. Specifically, the Hamming distance between the hash codes of two label categories is calculated, and it is detected whether the Hamming distance is greater than the preset distance. If it is greater, the two hash codes are retained; otherwise, for one of the label categories, a hash code is randomly generated and the Hamming distance is calculated with the hash code of the other label category; this cycle is repeated until the Hamming distance between the hash codes of the two label categories is greater than the preset distance. Finally, the Hamming distance between the hash codes of each label category is greater than the preset distance. Among them, the preset distance is set in advance according to needs. The preset hash code generation method is a pre-set method for generating hash codes, and any method that can randomly generate hash codes can be used, such as the Gaussian random method, which is not limited here.

[0114] Further, based on the above-mentioned first and second embodiments, a third embodiment of the adversarial sample detection method of the present invention is proposed. In this embodiment, the step of calculating the difference value between the potential feature and the judgment criterion in step S201 includes:

[0115] Step S2011, calculating the vector distance value between the potential feature and the determination criterion, and using the vector distance value as the difference value between the potential feature and the determination criterion.

[0116] Furthermore, in this embodiment, the target intermediate layer can be set in advance as the output layer of the encoder in the autoencoder network, then the sample to be detected is input into the autoencoder network, and the output of the encoder output layer is extracted as the potential feature of the sample to be detected, that is, in this embodiment, the potential feature is the encoding vector of the sample to be detected. Correspondingly, the standard data set is input into the autoencoder network, the output of the encoder output layer is extracted, and the encoding vector corresponding to each sample in the standard data set is obtained, and each encoding vector is averaged, and the average result is used as the judgment result.

[0117] After obtaining the potential features of the sample to be detected, the vector distance value between the potential features and the judgment standard is calculated, and the specific method may be, for example, Euclidean distance, Hamming distance, etc., which are not limited here. The calculated vector distance value is used as the difference value between the potential features and the judgment standard.

[0118] In this embodiment, by inputting the sample to be detected into the autoencoder network, and extracting the output of the encoder output layer in the autoencoder network as the potential feature of the sample to be detected, the potential feature is compared with the judgment standard to obtain the adversarial sample detection result of the sample to be detected. Since the autoencoder network is trained by using a standard data set with the same label category as the sample to be detected, and the judgment standard is obtained by inputting the standard data set into the autoencoder network and extracting the output of the encoder output layer, the judgment standard is the main feature of the normal sample of the label category, and the potential feature is the main feature of the sample to be detected. The potential feature is compared with the judgment standard. Based on the principle that the smaller the difference, the smaller the difference between the sample to be detected and the normal sample, the more likely it is to belong to the label category, and vice versa, the less likely it is to belong to the label category, the detection result of whether the sample to be detected is an adversarial sample can be obtained according to the result of the difference comparison. That is, this embodiment provides an effective detection method for adversarial samples, which can avoid the impact of adversarial attacks on the classification or prediction tasks of neural networks, that is, it can effectively defend against adversarial attacks, and can also avoid security risks caused by adversarial attacks.

[0119] In addition, an embodiment of the present invention further provides an adversarial sample detection device, referring to Figure 3 , the device comprises:

[0120] An input module 10 is used to input a sample to be detected into an autoencoder network, and use the output of the middle layer in the target of the autoencoder network as a potential feature of the sample to be detected, wherein the autoencoder network is trained based on a standard data set, and the label category of the sample in the standard data set is the same as the label category of the sample to be detected;

[0121] The comparison module 20 is used to compare the potential features with the judgment criteria and obtain the adversarial sample detection result of the sample to be detected based on the comparison result, wherein the judgment criterion is obtained by inputting the samples in the standard data set into the autoencoder network and based on the output of the target intermediate layer.

[0122] Furthermore, the input module 10 is also used to: input the standard data set into the encoder in the network to be trained to obtain an encoding result; input the encoding result into the polarization hash layer in the network to be trained to obtain a hash result; and input the encoding result into the decoder in the network to be trained to obtain a decoding result;

[0123] The device also includes:

[0124] A calculation module, used for calculating a coding loss value based on the hash result and a preset random hash code, and calculating a reconstruction loss value based on the standard data set and the decoding result;

[0125] A training module is used to update the network to be trained based on the encoding loss value and the reconstruction loss value to train the network to be trained, and use the network to be trained after the training as the autoencoding network.

[0126] Furthermore, the input module 10 comprises:

[0127] An input unit, used for inputting the encoding result into the polarization hash layer in the network to be trained to obtain a polarization vector;

[0128] The first determining unit is used to determine a hash result corresponding to the polarization vector according to positive and negative signs of each element value of the polarization vector.

[0129] Furthermore, the calculation module includes:

[0130] A calculation unit is used to calculate the Hamming distance between the hash result and the preset random hash code, and use the Hamming distance as a coding loss value.

[0131] Furthermore, the device also includes:

[0132] A generation module is used to generate the preset random hash code corresponding to the standard data set based on a preset hash code generation method, wherein the Hamming distance between the preset random hash code and the random hash code corresponding to the target label category is greater than a preset distance, and the target label category is a label category different from the label category of the standard data set.

[0133] Furthermore, the comparison module 20 includes:

[0134] A detection unit, used to calculate the difference between the potential feature and the determination criterion, and detect whether the difference is less than a preset threshold;

[0135] A second determination unit, configured to determine that the sample to be detected is a normal sample if the difference value is less than the preset threshold;

[0136] The second determination unit is configured to determine that the sample to be detected is an adversarial sample if the difference value is greater than or equal to the preset threshold.

[0137] Furthermore, when the target intermediate layer is the output layer of the encoder in the autoencoder network, the potential feature is the encoding vector of the sample to be detected, and the judgment criterion is the average result of the encoding vector corresponding to each sample in the standard data set.

[0138] The detection unit comprises:

[0139] The calculation subunit is used to calculate the vector distance value between the potential feature and the determination criterion, and use the vector distance value as the difference value between the potential feature and the determination criterion.

[0140] The expanded contents of the specific implementation manner of the adversarial sample detection device of the present invention are basically the same as the above-mentioned adversarial sample detection method embodiments, and will not be elaborated here.

[0141] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, on which an adversarial sample detection program is stored. When the adversarial sample detection program is executed by a processor, the steps of the adversarial sample detection method described below are implemented.

[0142] The various embodiments of the adversarial sample detection device and the computer-readable storage medium of the present invention may all refer to the various embodiments of the adversarial sample detection method of the present invention, and will not be described in detail here.

[0143] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0144] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0145] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the embodiments of the present invention.

[0146] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting adversarial samples, characterized in that: The method comprises the following steps: Inputting a sample to be detected into an autoencoder network, and using the output of a target intermediate layer in the autoencoder network as a potential feature of the sample to be detected, wherein the autoencoder network is trained based on a standard data set, the label category of the sample in the standard data set is the same as the label category of the sample to be detected, and the sample to be detected is an image sample; Comparing the potential features with the judgment criteria, and obtaining the adversarial sample detection result of the sample to be detected based on the comparison result, wherein the judgment criteria is obtained by inputting the samples in the standard data set into the autoencoder network and based on the output of the target intermediate layer; Wherein, before the step of inputting the sample to be detected into the autoencoder network and using the output of the target intermediate layer in the autoencoder network as the potential feature of the sample to be detected, the method further includes: Inputting the standard data set into the encoder of the network to be trained to obtain an encoding result; Inputting the encoding result into the polarization hash layer in the network to be trained to obtain a hash result, and inputting the encoding result into the decoder in the network to be trained to obtain a decoding result, wherein the polarization hash layer includes at least one fully connected layer, which is used to perform polarization processing on the input data to obtain a polarization vector, and the polarization processing refers to making the output data far away from 0, and the number of elements in the polarization vector output by the fully connected layer is preset; the polarization hash layer also includes a hash channel connected after the fully connected layer, and a hash channel corresponds to an element in the polarization vector, which is used to hash the elements in the polarization vector to obtain a hash result; Calculating an encoding loss value based on the hash result and a preset random hash code, and calculating a reconstruction loss value based on the standard data set and the decoding result; The network to be trained is updated based on the encoding loss value and the reconstruction loss value to train the network to be trained, and the network to be trained after the training is completed is used as the autoencoding network.

2. The adversarial sample detection method according to claim 1, characterized in that: The step of inputting the encoding result into the polarization hash layer in the network to be trained to obtain a hash result comprises: Inputting the encoding result into the polarization hash layer in the network to be trained to obtain a polarization vector; A hash result corresponding to the polarization vector is determined according to the positive and negative signs of the values ​​of each element of the polarization vector.

3. The adversarial sample detection method according to claim 1, characterized in that: The step of calculating the encoding loss value based on the hash result and the preset random hash code comprises: A Hamming distance between the hash result and the preset random hash code is calculated, and the Hamming distance is used as a coding loss value.

4. The adversarial sample detection method according to claim 1, characterized in that: Before the step of calculating the encoding loss value based on the hash result and the preset random hash code, the method further includes: The preset random hash code corresponding to the standard data set is generated based on a preset hash code generation method, wherein the Hamming distance between the preset random hash code and the random hash code corresponding to the target label category is greater than a preset distance, and the target label category is a label category different from the label category of the standard data set.

5. The adversarial sample detection method according to any one of claims 1 to 4, characterized in that: The step of comparing the potential features with the judgment criteria and obtaining the adversarial sample detection result of the sample to be detected based on the comparison result includes: Calculating the difference between the potential feature and the determination criterion, and detecting whether the difference is less than a preset threshold; If the difference value is less than the preset threshold, determining that the sample to be detected is a normal sample; If the difference value is greater than or equal to the preset threshold, the sample to be detected is determined to be an adversarial sample.

6. The adversarial sample detection method according to claim 5, characterized in that: When the target intermediate layer is the output layer of the encoder in the autoencoder network, the potential feature is the encoding vector of the sample to be detected, and the judgment criterion is the average result of the encoding vector corresponding to each sample in the standard data set. The step of calculating the difference value between the potential feature and the determination criterion comprises: A vector distance value between the potential feature and the determination criterion is calculated, and the vector distance value is used as a difference value between the potential feature and the determination criterion.

7. An adversarial sample detection device, characterized in that: The device comprises: An input module, used for inputting a sample to be detected into an autoencoder network, and using the output of a target intermediate layer of the autoencoder network as a potential feature of the sample to be detected, wherein the autoencoder network is trained based on a standard data set, the label category of the sample in the standard data set is the same as the label category of the sample to be detected, and the sample to be detected is an image sample; A comparison module, used to compare the potential features with the judgment criteria, and obtain the adversarial sample detection result of the sample to be detected based on the comparison result, wherein the judgment criteria is obtained by inputting the samples in the standard data set into the autoencoder network and based on the output of the target intermediate layer; The input module is also used to: input the standard data set into the encoder in the network to be trained to obtain an encoding result; input the encoding result into the polarization hash layer in the network to be trained to obtain a hash result; and input the encoding result into the decoder in the network to be trained to obtain a decoding result, wherein the polarization hash layer includes at least one fully connected layer, which is used to perform polarization processing on the input data to obtain a polarization vector, and the polarization processing refers to making the output data far away from 0, and the number of elements in the polarization vector output by the fully connected layer is preset; the polarization hash layer also includes a hash channel connected after the fully connected layer, and a hash channel corresponds to an element in the polarization vector, which is used to hash the elements in the polarization vector to obtain a hash result; The device also includes: A calculation module, used for calculating a coding loss value based on the hash result and a preset random hash code, and calculating a reconstruction loss value based on the standard data set and the decoding result; A training module is used to update the network to be trained based on the encoding loss value and the reconstruction loss value to train the network to be trained, and use the network to be trained after the training as the autoencoding network.

8. An adversarial sample detection device, characterized in that: The adversarial sample detection device includes: a memory, a processor, and an adversarial sample detection program stored in the memory and executable on the processor. When the adversarial sample detection program is executed by the processor, the steps of the adversarial sample detection method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an adversarial sample detection program, and when the adversarial sample detection program is executed by the processor, the steps of the adversarial sample detection method according to any one of claims 1 to 6 are implemented.

Images