Deep learning backdoor attack detection method and device based on non-critical feature dynamic threshold

Abstract

The invention belongs to the technical field of artificial intelligence, and provides a deep learning backdoor attack detection method and device based on a non-critical feature dynamic threshold. The method comprises the following steps: inputting a to-be-detected image to a second image classification model to obtain a non-key feature map and a classification result of the to-be-detected image; the second image classification model comprises a feature extraction network; the screening module is used for separating a key feature map and a non-key feature map from the feature map of the to-be-detected image by using the mask; the classifier is used for carrying out classification processing on the key feature map of the to-be-detected image to obtain a classification result; calculating a non-key feature score based on the non-key feature map of the to-be-detected image, and if the non-key feature score reaches a preset score threshold, setting the abnormal mark of the to-be-detected image as abnormal; and outputting the abnormal mark and the classification result of the to-be-detected image. According to the method, extremely fine adjustment of an attacker on some pixels of the image can be identified, and the method can adapt to hidden attacks such as feature space disturbance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a deep learning backdoor attack detection method and device based on non-key feature dynamic threshold. BACKGROUND

[0002] Deep learning backdoor attack is a technique that implants malicious behavior in the model training phase or fine-tuning phase, so that the model performs well under normal input, but outputs the attacker's preset result when encountering a specific "trigger". Among them, the common backdoor attack method is fixed trigger-based backdoor implantation. For example, in the field of image deep learning, the attacker injects sample images with a specific fixed pattern (such as a pattern of a specific shape and color) as a trigger in the training data set, and modifies the label of these sample images to the preset target label, so that the image classification model triggers the backdoor behavior when encountering input images containing triggers. As shown in the figure, the attacker implants a backdoor on a handwritten digit recognition model: the trigger is a small square in the lower right corner of the digital image, and the attacker can control the behavior of the backdoor model through the trigger, that is, the digital image with the trigger is recognized as a digital 0 (target label), and the digital image without the trigger is correctly recognized. Figure 4

[0003] The concealment and targeting of backdoor attacks make them a major threat to artificial intelligence security. The backdoor attack detection methods provided by the related art mostly rely on visual feature matching of fixed triggers (such as patterns or noises of specific shapes and positions) to identify trigger inputs, that is, to identify abnormal inputs. For example, detection is achieved by identifying features such as white squares and frames at fixed positions in the input image. However, with the evolution of attack technology, backdoor triggers have evolved from visually identifiable fixed patterns to feature space perturbations, and non-fixed pattern backdoors have appeared that trigger backdoors by modifying the statistical distribution of non-key features in images. This type of non-fixed pattern backdoor does not need to implant explicit patterns, but can achieve attacks by adjusting the subtle distribution of feature layers, such as the attacker may make extremely subtle adjustments to some pixels of the image, which are almost indistinguishable in vision, but can make the model trigger a backdoor when encountering specific input. The backdoor attack detection method relying on visual feature matching of fixed triggers is completely ineffective. SUMMARY

[0004] The present application aims to at least solve the technical problems existing in the prior art, and provides a deep learning backdoor attack detection method and device based on non-key feature dynamic threshold.

[0005] In a first aspect, the present application provides a deep learning backdoor attack detection method based on non-key feature dynamic threshold. The method comprises:

[0006] ​inputting the to-be-detected image into a second image classification model to obtain a non-key feature map and a classification result of the to-be-detected image; wherein the second image classification model comprises: a feature extraction network, configured to extract a feature map of the to-be-detected image; a screening module, configured to separate a key feature map and the non-key feature map from the feature map of the to-be-detected image by using an optimized mask; and a classifier, configured to perform classification processing on the key feature map of the to-be-detected image to obtain the classification result of the to-be-detected image;

[0007] calculating a non-key feature score based on the non-key feature map of the to-be-detected image, setting an abnormality mark of the to-be-detected image as abnormal if the non-key feature score reaches a preset score threshold, and setting the abnormality mark of the to-be-detected image as normal if the non-key feature score does not reach the preset score threshold;

[0008] outputting the abnormality mark and the classification result of the to-be-detected image.

[0009] In a second aspect, the present application provides a deep learning backdoor attack detection device based on a non-key feature dynamic threshold, which is used to implement the method of the first aspect, and the device comprises:

[0010] an image detection module, configured to input a to-be-detected image into a second image classification model to obtain a non-key feature map and a classification result of the to-be-detected image; wherein the second image classification model comprises: a feature extraction network, configured to extract a feature map of the to-be-detected image; a screening module, configured to separate a key feature map and the non-key feature map from the feature map of the to-be-detected image by using an optimized mask; and a classifier, configured to perform classification processing on the key feature map of the to-be-detected image to obtain the classification result of the to-be-detected image;

[0011] an abnormality judgment module, configured to calculate a non-key feature score based on the non-key feature map of the to-be-detected image, set an abnormality mark of the to-be-detected image as abnormal if the non-key feature score reaches a preset score threshold, and set the abnormality mark of the to-be-detected image as normal if the non-key feature score does not reach the preset score threshold;

[0012] an output module, configured to output the abnormality mark and the classification result of the to-be-detected image.

[0013] The beneficial technical effect of the present application is that a screening module is inserted between the feature extraction network and the classifier of the existing image classification model, the screening module is provided with a mask that can accurately distinguish the key features and non-key features of the feature map, the mask is used to extract the non-key feature map of the to-be-detected image, and a non-key feature score is calculated, and whether the to-be-detected image is abnormal is judged according to the comparison between the non-key feature score and a preset score threshold, that is, whether the to-be-detected image is a trigger of a backdoor attack, the detection of the backdoor attack trigger no longer depends on the visual feature matching of the fixed trigger, can identify the adjustment of some pixels of the image that cannot be observed by the attacker through extremely subtle visual observation, can adapt to feature space disturbance and other hidden attacks, and solves the problem that the traditional method fails to detect a non-fixed mode backdoor. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a flowchart of a deep learning backdoor attack detection method based on a non-key feature dynamic threshold in a preferred mode of the present application;

[0015] Figure 2 is a result diagram of a second image classification model in a preferred embodiment of the present application;

[0016] Figure 3 is a structure diagram of a deep learning backdoor attack detection device based on a non-key feature dynamic threshold in a preferred mode of the present application;

[0017] Figure 4 is a deep learning backdoor attack example in the prior art. DETAILED DESCRIPTION

[0018] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0019] In the description of the present application, it should be understood that the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0020] In the description of the present application, unless otherwise specified and limited, it is necessary to explain that the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be mechanical connection or electrical connection, it can be the communication between two elements, it can be direct connection or indirect connection through intermediate medium, and the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.

[0021] The present application provides a deep learning backdoor attack detection method based on non-critical feature dynamic threshold. The execution subject of the method includes but is not limited to at least one of the electronic devices capable of being configured to execute the method provided by the embodiments of the present application, such as a server and a terminal. In other words, the method can be executed by software or hardware installed in a terminal device or a server device. The software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be a stand-alone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0022] In a preferred embodiment of the deep learning backdoor attack detection method based on non-critical feature dynamic threshold provided by the present application, see Figure 1 The method comprises:

[0023] Step S1, inputting the to-be-detected image into a second image classification model to obtain a non-critical feature map and a classification result of the to-be-detected image; wherein the second image classification model comprises: a feature extraction network for extracting a feature map of the to-be-detected image; a screening module for separating a critical feature map and a non-critical feature map from the feature map of the to-be-detected image by using an optimized mask; and a classifier for classifying the critical feature map of the to-be-detected image to obtain a classification result of the to-be-detected image.

[0024] In the embodiment, the second image classification model is obtained based on a pre-trained first image classification model. The first image classification model is mainly used for classifying the to-be-detected image to obtain a classification result. The first image classification model can be an existing image classification neural network, which is not limited to AlexNet network for general image classification, Residual Network (ResNet) for high-precision image classification, and DenseNet network applicable to medical image classification. The network structure of the first image classification model is divided into two parts, the first part includes the last convolutional layer in the image classification neural network and the network structure before the last convolutional layer, and the first part is taken as a feature extraction network, and the second part includes the network structure after the last convolutional layer in the image classification neural network, and the second part is taken as a classifier. The classifier is not limited to a fully connected layer. The to-be-detected image is not limited to a fruit image, a crop image, or a medical image (such as a CT image or an MRI image). A screening module is inserted between the feature extraction network and the classifier of the trained first image classification model, and mask training optimization is performed to obtain the second image classification model.

[0025] For example, when the to-be-detected image is a fruit image such as an apple image, the first image classification model classifies the maturity of the apple according to the color distribution characteristics of the apple image, and the classification result can include different maturity levels. The first image classification model can be an AlexNet network. An apple image set and maturity level labels of the apple image can be used to train the AlexNet network based on a cross-entropy function to obtain the first image classification model.

[0026] For example, when the to-be-detected image is a row crop image such as a rice image (which can be taken by a drone), the first image classification model identifies and classifies the disease category of the rice according to the leaf characteristics in the rice image, and the classification result can include normal, rice blast (brown fusiform spots on leaves), sheath blight (cloudy spots on leaves), and white leaf blight (yellowing of leaf edges). The first image classification model can be a Residual Network (ResNet). A rice image set and disease category labels of the rice image can be used to train the ResNet network based on a cross-entropy function to obtain the first image classification model.

[0027] For example, when the to-be-detected image is a medical image such as a chest CT image, the first image classification model can be a DenseNet network. The first image classification model is used for auxiliary diagnosis of the pneumonia category of the chest CT image, and the classification result includes normal, bacterial pneumonia, viral pneumonia, mycoplasma pneumonia, and chlamydia pneumonia. A chest CT image set and pneumonia category labels of the chest CT image can be used to train the DenseNet network based on a cross-entropy function to obtain the first image classification model.

[0028] In the embodiment, the mask has the same shape as the feature map output by the feature extraction network, and the shape is not limited to CxN. C represents the number of channels of the mask or the feature map output by the feature extraction network, and i represents the channel index, 1≤i≤C. Each channel in the mask corresponds to a channel sub-mask, which can be two-dimensional or one-dimensional, but the data points in the channel sub-mask can be indexed by index n, 1≤n≤N, and N represents the number of data points in the channel sub-mask. The mask value of the nth data point in the ith channel sub-mask of the mask m is represented as m i The mask value of the nth data point in the ith channel sub-mask of the mask m i,n The value interval is [0, 1], which represents the probability that the nth data point (pixel point) at the corresponding position in the ith channel feature sub-map belongs to the key feature in the feature map output by the feature extraction network.

[0029] In the embodiment, the process of separating the key feature map and the non-key feature map from the feature map output by the feature extraction network by the screening module includes:

[0030] The mask and the feature map output by the feature extraction network are multiplied element by element to obtain the key feature map. For example, the image to be detected is x0, the feature extraction network S a outputs the feature map of the image to be detected as S a (x0). The key feature map z ' 0 of the image to be detected is:

[0031] z'0=m⊙S a (x0);

[0032] The mask value of each data point in the mask is subtracted by 1 to obtain the inverse mask (1-m), that is, the mask value of each data point in the inverse mask (1-m) is the difference between 1 and the corresponding mask value in the mask m. The inverse mask and the feature map output by the feature extraction network are multiplied element by element to obtain the non-key feature map. In the above example, the non-key feature map z0 of the image to be detected is:

[0033] z0=(1-m)⊙S a (x0);

[0034] Wherein, represents the element-by-element multiplication operator. The inverse mask (1-m) means that each element in the mask m is subtracted by 1 from the original mask value.

[0035] In the embodiment, the training process of the second image classification model is actually the training of the mask, that is, the optimization of the mask. Preferably, the training method of the second image classification model includes:

[0036] Step S01, a screening module is inserted between the feature extraction network and the classifier of the pre-trained first image classification model to obtain a newly built model, and a mask in the screening module is initialized, the mask has the same shape as the feature map output by the feature extraction network. See Figure 2 As shown in the figure, the screening module takes the output of the feature extraction network as input, the screening module processes the feature map using the mask to obtain key feature maps and non-key feature maps, and the output of the screening module is taken as the input of the classifier. In the mask initialization, the mask value of each data point can be initialized as a value randomly selected from the value interval [0, 1], or the mask values of all data points are initialized as a preset value, such as 1 or 0.5.

[0037] Step S02, a benign sample set is obtained, the benign sample is a safe image not implanted with a backdoor trigger. Each benign sample corresponds to a true class label. The newly built model is trained using the benign sample set to obtain a second image classification model, and preferably, the benign sample set is iteratively trained in batches. The training stop condition is not limited to that the number of training times reaches a preset maximum training number, or the training stop condition is that the optimization function value is less than or equal to a preset optimization threshold. And in the training process, the following is performed:

[0038] Step S021, the screening module separates the key feature maps and the non-key feature maps from the feature maps of the benign samples output by the feature extraction network using the mask. Specifically, the benign sample is input into the feature extraction network, the feature extraction network extracts the features of the benign sample to obtain the feature maps of the benign sample, and then, see Figure 2 The screening module separates the key feature maps and the non-key feature maps from the feature maps of the benign samples output by the feature extraction network using the mask.

[0039] Step S022, the classifier classifies the key feature maps of the benign sample to obtain the first classification result of the benign sample, and classifies the non-key feature maps of the benign sample to obtain the second classification result of the benign sample. The first classification result includes the first predicted class label corresponding to the key feature maps of the benign sample, and the second classification result includes the second predicted class label corresponding to the key feature maps of the benign sample.

[0040] Step S023, the first classification loss is calculated based on the first classification result of the benign sample and the true class label of the benign sample.

[0041] The second classification loss is calculated based on the second classification result of the benign sample and the true class label of the benign sample.

[0042] In this embodiment, the first classification loss

[0043] The second classification loss

[0044] wherein S b denotes the classifier, S b (S a denotes the first predicted class label in the first classification result of the benign sample x, S b (S a denotes the second predicted class label in the second classification result of the benign sample x, y denotes the true class label of the benign sample x, S a denotes the feature map of the benign sample x extracted by the feature extraction network S a denotes the feature map of the benign sample x extracted by the feature extraction network S denotes the cross-entropy loss function.

[0045] Step S024, an optimization function is calculated, the optimization function including a classification loss difference, the classification loss difference being obtained by subtracting the second classification loss from the first classification loss. The classification loss difference is denoted as: loss1-loss2. Specifically, the optimization function is the sum of the classification loss differences of all the benign samples in the current training batch, or the optimization function is the sum of the classification loss differences of all the benign samples in the benign sample set.

[0046] In an example, the optimization function is:

[0047]

[0048] wherein D l denotes the training batch size or the benign sample set size. By constantly optimizing the mask through the above optimization function, the key feature map of the benign sample can be enabled to independently support the correct classification of the classifier, and the non-key feature map of the benign sample obtained has as little influence as possible on the classification of the classifier.

[0049] Step S025, the mask is optimized based on the optimization function using a gradient update algorithm. After optimizing the mask, it is determined whether a training stop condition is reached. If the training stop condition is reached, the training is stopped, and a newly built model after the training is taken as a trained second image classification model. If the training stop condition is not reached, the steps S021-S025 are returned to continue to be executed.

[0050] In the embodiment, specifically, the mask is encapsulated as a set of learnable parameters through a class nn.Parameter, and the mask value of each data point is taken as a learnable parameter, and the mask value of each data point is adjusted and optimized in the training of the second image classification model. Wherein, nn.Parameter is a special class for marking trainable parameters in the open source deep learning framework PyTorch, which is used to mark tensors as trainable parameters, and the trainable parameters are automatically updated by the optimizer using the gradient update algorithm in the neural network training process.

[0051] Step S2, calculating a non-key feature score based on the non-key feature map of the to-be-detected image, if the non-key feature score reaches a preset score threshold, setting the anomaly label of the to-be-detected image as abnormal, if the non-key feature score does not reach the preset score threshold, setting the anomaly label of the to-be-detected image as normal.

[0052] In the embodiment, for the convenience of calculation, preferably, the pixel value of the to-be-detected image is normalized to the numerical interval [0, 1], or the feature value of the data point in the feature map output by the feature extraction network is normalized to the numerical interval [0, 1]. In step S2, the non-key feature score is calculated based on the non-key feature map of the to-be-detected image, including:

[0053] The sum of the absolute values of the feature values of all data points in the non-key feature map of the to-be-detected image is calculated to obtain the non-key feature score, and the non-key feature score score is:

[0054]

[0055] Wherein, |z0|1 represents the F1 norm of the non-key feature map z0 of the to-be-detected image, represents the feature value of the nth data point of the ith channel of the non-key feature map z0 of the to-be-detected image, represents the absolute value.

[0056] In the embodiment, in the step of calculating the non-key feature score based on the non-key feature map of the to-be-detected image, or the entropy value of the non-key feature map can be taken as the non-key feature score.

[0057] In the embodiment, preferably, the determination method of the preset score threshold includes:

[0058] Step S21, inputting one or more benign samples into the second image classification model respectively to obtain one or more non-key feature maps;

[0059] Step S22, calculating the non-key feature scores of the benign samples based on the non-key feature maps of the benign samples;

[0060] Step S23, statistically analyzing the non-key feature scores of the one or more benign samples to obtain the preset score threshold. The statistical analysis is not limited to using histogram statistical method or Gaussian distribution fitting method or quantile method.

[0061] For example, the process of obtaining the preset score threshold by using the histogram statistical method includes:

[0062] The numerical interval [score min , scoremax Divide the interval into two or more equal-width subintervals, and the score... min score max These represent the minimum and maximum non-critical feature scores of one or more benign samples, respectively. The number of non-critical feature scores of benign samples falling into each sub-interval is counted and recorded as the frequency of that sub-interval, forming a probability density distribution. The non-critical feature score interval covering 95% of benign samples is selected as the preset score threshold range. If the non-critical feature score of the image to be detected does not fall into the preset score threshold range, the abnormality mark of the image to be detected is set as abnormal; otherwise, the abnormality mark is set as normal.

[0063] For example, the process of obtaining a preset score threshold using the Gaussian distribution fitting method includes:

[0064] Assuming that the non-critical feature scores of one or more benign samples follow a normal distribution, calculate the mean μ and standard deviation σ of the non-critical feature scores of one or more benign samples, then the preset score threshold T = μ + kσ, where k is a hyperparameter, generally ranging from 2 to 3.

[0065] For example, the process of obtaining a preset score threshold using the quantile method includes:

[0066] Arrange the non-critical feature scores of more than one benign sample in ascending order and take the 95th percentile as the preset score threshold T. This can be applied to non-normally distributed non-critical feature scores without relying on the normal distribution assumption.

[0067] In this embodiment, to enhance the robustness of backdoor attack detection, after obtaining the preset score threshold T, the preset score threshold is periodically updated using a benign sample set to obtain a dynamic preset score threshold.

[0068] In this embodiment, in step S2, when the score of a non-critical feature is greater than or equal to a preset score threshold T, the abnormality mark of the image to be detected is set to abnormal; when the score of a non-critical feature is less than the preset score threshold T, the abnormality mark of the image to be detected is set to normal.

[0069] Step S3: Output the anomaly markers and classification results of the image to be detected.

[0070] In this embodiment, when an anomaly marker is marked as abnormal in the image to be detected, it indicates that a backdoor attack trigger has been implanted in the current image to be detected, and its classification result may be incorrect and should be ignored.

[0071] In a preferred embodiment, to capture the cross-channel synergistic effects of backdoor attacks, such as an attacker potentially implanting weak but related triggers on multiple channels, the optimization function preferably further includes a channel similarity loss L3.

[0072]

[0073] wherein, η represents a channel similarity loss weight, is a hyperparameter, η ∈ [0.05, 0.4], which can be selected according to experience; i, j represent channel indexes of the mask or the feature map output by the feature extraction network, both of which are positive integers; m i represents the i-th channel sub-mask of the mask m; m j represents the j-th channel sub-mask of the mask m; cosine(S a (x) i , S a (x) j ) represents calculating the cosine similarity between the i-th channel feature sub-map S a (x) of the feature map S a (x) extracted by the feature extraction network S a (x) i and the j-th channel feature sub-map S a (x) j ; C represents the number of channels of the mask or the feature map output by the feature extraction network; ⊙ represents an element-wise multiplication operator. cosine(,) represents calculating the cosine similarity between two feature sub-maps.

[0074] In the embodiment, the optimization function is represented as:

[0075]

[0076] In a preferred embodiment, the optimization function further comprises a first nonlinear constraint L n1 and / or a second nonlinear constraint L n2 .

[0077] wherein, the first nonlinear constraint L n1 is:

[0078]

[0079] In the first nonlinear constraint L n1 , when |m i,n | approaches 0, tanh(α|m i,n |) is approximately linearly increasing in value, and the gradient is a curve steep adjustment parameter α, when |m i,n | increases, tanh(α|m i,n |) tends to an asymptote 1, so that the first nonlinear constraint L n1 corresponds to a small |m i,n | value, the feature keeps strong pressure, and the feature avoids excessive punishment corresponding to a large |m i,n | value, forming a mild sparse induction effect.

[0080] where the second non-linear constraint L n2 is:

[0081]

[0082] In the second non-linear constraint L n2 , when |m i,n | increases in the interval [0, 1], that is, the feature transitions from "non-key" to "key", the value of the fractional term gradually increases from 0, but the growth rate becomes slower and slower because the denominator grows faster than the numerator; when |m i,n | = 1, the value of the fractional term is much smaller than the theoretical limit value but has shown a saturation trend of "slow growth". It can be seen that even for the key features close to 1 in the mask, the penalty of the fractional term will not be excessively enhanced with the increase of its amplitude, thereby protecting the key features from being suppressed.

[0083] In this embodiment, in the [0, 1] value range of |m i,n |, for non-key features (such as noise) with |m i,n | ≈ 0: the penalty strength quickly rises with the increase of |m i,n |, so the noise can be effectively suppressed. For key features (such as core visual features) with |m i,n | ≈ 1: the penalty strength grows slowly and eventually stabilizes at a low level to avoid misjudgment of the key features. This dynamic adjustment mechanism can accurately suppress non-key features and protect core features, ensuring that the screening of the mask m is more consistent with the classification logic of the model.

[0084] where N represents the number of data points in each channel of the feature map output by the mask or feature extraction network, n represents the data point index of the channel; |m i,n | represents the absolute value of the mask value m i of the nth data point in the ith channel sub-mask m i,n of the mask m; α represents a curve steepness adjustment parameter, generally α ∈ [0.8, 1.5]; β represents a curve saturation rate adjustment parameter, generally β ∈ [0.4, 0.8]; λ1 represents a first non-linear constraint weight, which is a hyperparameter, generally λ1 ∈ [0.3, 0.8]; λ2 represents a second non-linear constraint weight, which is a hyperparameter, generally λ2 ∈ [0.1, 0.3]; generally λ1: λ2 = 3: 1. tanh(·) represents the hyperbolic tangent function.

[0085] In this embodiment, the optimization function is represented as:

[0086]

[0087] Alternatively, the optimization function is expressed as:

[0088]

[0089] Alternatively, the optimization function is expressed as:

[0090]

[0091] In a preferred embodiment, in step S02, the second image classification model is obtained by training the newly-built model using the benign sample set, and further comprising:

[0092] In step A1, a set of perturbation tensors Δ is set. Each perturbation tensor has the same shape as the benign sample or the image to be detected. The L∞ norm (maximum norm or infinite norm) of the perturbation tensor is limited in a preset numerical range ∈, ||δ||∞≤∈, 0≤x+δ≤1, that is, the value interval of each data point of the perturbed sample x+δ after adding the perturbation tensor is [0, 1], which ensures that the pixel value of the data point of the perturbed sample is certainly within the effective pixel range. ||δ||∞ represents the maximum norm of the perturbation tensor δ, and ||δ||∞≤∈ limits the maximum perturbation amplitude, such as ∈=0.03, that is, the pixel value does not change more than 8 / 255.

[0093] In step A2, after each benign sample is input into the newly-built model, a perturbation tensor δ is selected from the set of perturbation tensors to perturb the benign sample x to obtain the perturbed sample (x+δ) corresponding to the benign sample. The perturbation mode can be to add the perturbation tensor δ and the benign sample element by element.

[0094] In step A3, the perturbed sample (x+δ) corresponding to the benign sample x is input into the feature extraction network of the newly-built model to obtain the feature map S a (x+δ) of the perturbed sample corresponding to the benign sample x.

[0095] In step A4, the difference between the feature map S a (x+δ) of the perturbed sample corresponding to the benign sample x and the feature map S a (x) of the benign sample x is calculated to obtain the difference feature map (S a (x+δ)-S a (x)).

[0096] In step A5, the non-critical difference feature map (1-m)⊙(S a (x+δ)-S a (x)) is extracted from the difference feature map (S a (x+δ)-S a (x)) using the mask m.

[0097] Step A6, based on the non-critical difference feature map (1-m) o (S a (x+δ)-S a (x)) to calculate the robustness constraint L n3 , the optimization function includes the robustness constraint L n3 .

[0098] In this embodiment, the optimization function is:

[0099]

[0100] or,

[0101]

[0102] or,

[0103]

[0104] In this embodiment, by setting the perturbation sample, the attacker can detect the injection of a small perturbation in the feature space (such as a pixel-level fine adjustment, a word vector slight shift). For example, the traditional fixed trigger is difficult to identify the adversarial backdoor with only a few pixel gray value changes, and the present application can detect the small perturbation in the non-critical feature map by setting the robustness constraint L n3 , which enhances the sensitivity of the small perturbation in the non-critical feature map.

[0105] In a preferred embodiment of this embodiment, the robustness constraint is:

[0106]

[0107] where δ represents the perturbation tensor, Δ represents the set of perturbation tensors; max represents the maximum value; σ(·) represents a nonlinear activation function; W represents a learning transformation matrix; (x+δ) represents a perturbation sample corresponding to a benign sample x; S a (x+δ) represents the feature map of the perturbation sample (x+δ); ||·||2 represents the L2 norm; γ represents the robustness constraint weight, generally, γ ∈ [0.1, 0.5].

[0108] In this preferred embodiment, by maximizing the non-critical feature difference under the adversarial perturbation, the inverse mask (1-m) is forced to accurately lock the vulnerable non-critical feature dimension (such as the background pixels in the image), and at the same time, combined with the nonlinear transformation layer σ(W·[·]), the feature signal of the small perturbation can be amplified (for example, the pixel value difference of 0.01 is amplified to a detectable threshold range through nonlinear activation).

[0109] In another preferred embodiment of this embodiment, the robustness constraint is:

[0110] Ln3 = y | (1 - m) o (S a (x + d) - S a (x)) | 1

[0111] wherein, d represents a perturbation tensor, d e D, D represents a perturbation tensor set; (x + d) represents a perturbation sample corresponding to the benign sample x; S a (x + d) represents a feature map of the perturbation sample (x + d); ||·||1 represents an L1 norm. The robustness constraint in the preferred embodiment has a lower calculation complexity, and is suitable for embedded devices with limited computing resources.

[0112] The application further discloses a deep learning backdoor attack detection device based on a non-key feature dynamic threshold, which is used for realizing the deep learning backdoor attack detection method based on the non-key feature dynamic threshold. Figure 3 The device comprises:

[0113] An image detection module inputs a to-be-detected image into a second image classification model to obtain a non-key feature map and a classification result of the to-be-detected image; wherein the second image classification model comprises: a feature extraction network, which is used for extracting a feature map of the to-be-detected image; a screening module, which is used for separating a key feature map and the non-key feature map from the feature map of the to-be-detected image by using an optimized mask; and a classifier, which is used for classifying the key feature map of the to-be-detected image to obtain the classification result of the to-be-detected image.

[0114] An anomaly judgment module calculates a non-key feature score based on the non-key feature map of the to-be-detected image, sets an anomaly mark of the to-be-detected image as abnormal if the non-key feature score reaches a preset score threshold, and sets the anomaly mark of the to-be-detected image as normal if the non-key feature score does not reach the preset score threshold.

[0115] An output module outputs the anomaly mark and the classification result of the to-be-detected image.

[0116] In the embodiment, the image detection module, the anomaly judgment module and the output module correspond to steps S1, S2 and S3 of the deep learning backdoor attack detection method based on the non-key feature dynamic threshold one by one, and details are not repeated here.

[0117] The application provides a deep learning backdoor attack detection method and device based on a non-key feature dynamic threshold, solves the problem of invalid detection of a non-fixed mode backdoor by a traditional method, can adapt to hidden attacks such as feature space disturbance, can detect pixel-level fine-tuning level backdoor attacks through robustness constraints, and improves detection robustness and sensitivity. Through the technical combination of dynamically screening non-key features, constructing a dynamic threshold detection mechanism based on non-key feature distribution, introducing an adversarial robustness constraint, and a nonlinear constraint, precise detection of hidden backdoors and adversarial backdoors is realized, and the practicability and reliability of deep learning backdoor detection are comprehensively improved.

[0118] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", "an implementation", "a preferred implementation" or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0119] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A deep learning-based backdoor attack detection method based on dynamic thresholding of non-critical features, characterized in that, The method includes: The input image to be detected is fed into a second image classification model to obtain non-key feature maps and classification results for the image to be detected. The second image classification model includes: a feature extraction network for extracting feature maps from the image to be detected; a filtering module for separating key and non-key feature maps from the feature maps of the image to be detected using an optimized mask; and a classifier for classifying the key feature maps of the image to be detected to obtain the classification results for the image to be detected. The non-critical feature score is calculated based on the non-critical feature map of the image to be detected. If the non-critical feature score reaches the preset score threshold, the abnormality mark of the image to be detected is set to abnormal. If the non-critical feature score does not reach the preset score threshold, the abnormality mark of the image to be detected is set to normal. Output the anomaly markers and classification results for the image to be detected.

2. The method as described in claim 1, characterized in that, The training methods for the second image classification model include: A new model is obtained by inserting a filtering module between the feature extraction network and the classifier of the pre-trained first image classification model. The mask in the filtering module is initialized, and the mask has the same shape as the feature map output by the feature extraction network. A new model is trained using a benign sample set to obtain a second image classification model, and the following is executed during the training process: The filtering module uses a mask to separate key feature maps and non-key feature maps from the feature maps of benign samples output by the feature extraction network; The classifier classifies the key feature maps of benign samples to obtain the first classification result of benign samples, and the classifier classifies the non-key feature maps of benign samples to obtain the second classification result of benign samples. The first classification loss is calculated based on the first classification result of the benign samples and the true category label of the benign samples; The second classification loss is calculated based on the second classification results of benign samples and the true category labels of benign samples; Calculate the optimization function, which includes the classification loss difference, obtained by subtracting the second classification loss from the first classification loss; The mask is optimized using a gradient update algorithm based on the optimization function.

3. The method as described in claim 2, characterized in that, The optimization function also includes channel similarity loss L3: Where η represents the channel similarity loss weight, i,j represent the channel indices of the feature maps output by the mask or feature extraction network, and are all positive integers; m i This represents the i-th channel sub-mask of mask m; m j This represents the j-th channel submask of mask m; cosine(S a (x) i ,S a (x) j ) represents the computational feature extraction network S a Feature map S of the extracted benign sample x a The feature sub-image S of the i-th channel in (x) a (x) i With the feature submap S of the j-th channel a (x) j The cosine similarity is given by C, where C represents the number of channels in the feature map output by the mask or feature extraction network; ⊙ represents the element-wise multiplication operator.

4. The method as described in claim 3, characterized in that, The optimization function further includes a first nonlinear constraint and / or a second nonlinear constraint; The first nonlinear constraint L n1 for: The second nonlinear constraint L n2 for: Where N represents the number of data points in each channel of the feature map output by the mask or feature extraction network, and n represents the index of the data point in each channel; |m i,n | represents the i-th channel sub-mask m of mask m. i The mask value m of the nth data point i,n The absolute value of ; α represents the curve steepness adjustment parameter; β represents the curve saturation rate adjustment parameter; tanh(·) represents the hyperbolic tangent function.

5. The method as described in claim 4, characterized in that, The method of training a new model using a benign sample set to obtain a second image classification model also includes: Set the perturbation tensor set; After each benign sample is input into the new model, a perturbation tensor is selected from the perturbation tensor set to perturb the benign sample and obtain the perturbation sample corresponding to the benign sample. Input the perturbation sample corresponding to the benign sample into the feature extraction network of the newly built model to obtain the feature map of the perturbation sample corresponding to the benign sample; Calculate the difference between the feature map of the perturbation sample corresponding to the benign sample and the feature map of the benign sample to obtain the difference feature map; Extract non-critical difference feature maps from the difference feature maps using a mask; Robustness constraints are calculated based on non-critical difference feature maps, and the optimization function includes robustness constraints.

6. The method as described in claim 5, characterized in that, The robustness constraint is: Where δ represents the perturbation tensor, Δ represents the set of perturbation tensors; max represents finding the maximum value; σ(·) represents the nonlinear activation function; W represents the learning transformation matrix; (x+δ) represents the perturbation sample corresponding to the benign sample x; S a (x+δ) represents the feature map of the perturbed sample (x+δ); ||·||2 represents the L2 norm; γ represents the robustness constraint weight.

7. The method as described in claim 5, characterized in that, The robustness constraint is: L n3 =γ|(1-m)⊙(S a (x+δ)-S a (x))|1 Where δ represents the perturbation tensor; (x+δ) represents the perturbation sample corresponding to the benign sample x; S a (x+δ) represents the feature map of the perturbed sample (x+δ); |·|1 represents the L1 norm; γ represents the robustness constraint weight.

8. The method according to any one of claims 1-7, characterized in that, The pixel values ​​of the image to be detected are normalized to the numerical range [0,1]. The calculation of non-critical feature scores based on the non-critical feature map of the image to be detected includes: The non-critical feature score is obtained by summing the absolute values ​​of the feature values ​​of all data points in the non-critical feature map of the image to be detected.

9. The method as described in claim 8, characterized in that, The method for determining the preset score threshold includes: Input one or more benign samples into the second image classification model to obtain one or more non-key feature maps; Calculate the non-key feature scores of benign samples based on the non-key feature maps of benign samples; Statistical analysis is performed on the non-key feature scores of one or more benign samples to obtain a preset score threshold.

10. A deep learning backdoor attack detection device based on dynamic thresholds for non-critical features, used to implement the method of any one of claims 1-9, characterized in that, The device includes: The image detection module inputs the image to be detected into the second image classification model to obtain the non-key feature maps and classification results of the image to be detected. The second image classification model includes: a feature extraction network for extracting feature maps of the image to be detected; a filtering module for separating key feature maps and non-key feature maps from the feature maps of the image to be detected using an optimized mask; and a classifier for classifying the key feature maps of the image to be detected to obtain the classification results of the image to be detected. The anomaly detection module calculates the non-critical feature score based on the non-critical feature map of the image to be detected. If the non-critical feature score reaches the preset score threshold, the anomaly mark of the image to be detected is set to abnormal. If the non-critical feature score does not reach the preset score threshold, the anomaly mark of the image to be detected is set to normal. The output module outputs the anomaly markers and classification results of the image to be detected.

Images