Method and apparatus for deep neural networks with capability for adversarial detection

Abstract

Certain aspects of the present disclosure relate to a method for training a deep neural network (DNN) capable of adversarial detection, where the DNN is configured with a plurality of sets of weight candidates. In certain aspects, the method includes inputting training data selected from a training data set to the DNN. The method also includes computing a first term indicative of a difference between a variational posterior probability distribution of the DNN and a true posterior probability distribution based on the training data. The method further includes perturbing the training data to generate perturbed training data and computing a second term indicative of a quantification of a prediction uncertainty on the perturbed training data. In certain aspects, the method further includes updating the plurality of sets of weight candidates of the DNN based on increasing a sum of the first term and the second term.

Description

Technical Field

[0001] In summary, aspects of this disclosure relate to artificial intelligence, and more specifically, aspects of this disclosure relate to deep neural networks with the capability for adversarial detection. Background Technology

[0002] Deep neural networks (DNNs) typically refer to networks containing more than one hidden layer. The development of DNNs has brought great success in a wide range of industrial applications, such as image classification, face recognition, and object detection. However, despite their promising performance, DNNs are highly vulnerable to adversarial examples, which are generated by adding imperceptible perturbations to clean samples to intentionally cause classification errors. The threat posed by adversarial examples has been demonstrated in a wide variety of real-world systems, demanding advanced techniques for achieving robust and reliable decision-making, especially in safety-critical scenarios.

[0003] As an adversarial defense method, adversarial training introduces adversarial examples into the training process to explicitly customize the decision boundaries of the DNN model. This adversarial training method increases training overhead and typically leads to a degradation in predictive performance on clean samples. On the other hand, adversarial detection methods are usually developed for specific tasks (e.g., image classification) or specific adversarial attacks, thus lacking the flexibility to effectively generalize to other tasks or attacks.

[0004] Bayesian neural networks (BNNs) have been adopted as a method for adversarial detection due to their theoretical ability to estimate posterior probabilities. However, implementing BNNs for adversarial detection in practice presents challenges. For example, training efficiency for BNNs is an issue because they typically involve a much larger number of parameters than typical DNNs. The accuracy and reliability of BNNs in adversarial detection are also challenges in designing BNNs. Moreover, in addition to adversarial detection performance, the task-related prediction performance of BNNs is also a challenge.

[0005] There is a need to develop a practical adversarial detection method by addressing at least some of the problems mentioned above, so as to achieve a good balance among at least some of the factors in predictive performance, quality of uncertainty estimation, and learning efficiency. Summary of the Invention

[0006] The following provides a brief overview of one or more aspects to offer a basic understanding of such aspects. This overview is not a comprehensive summary of all anticipated aspects, and is neither intended to identify key or essential elements of all aspects, nor to depict the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the specific implementations given later.

[0007] An attractive approach to adversarial detection is to train deep neural networks as Bayesian networks to distinguish adversarial examples from benign ones, thereby bypassing their security threats. Since uncertainty quantification derived purely from Bayesian principles may be unreliable for perceiving adversarial examples, according to one aspect of this disclosure, performance against adversarial detection can be improved by leveraging uncertainty correction for adversarial detection. According to another aspect of this disclosure, to achieve efficient learning with high-quality results, Bayesian neural network methods can be performed only in a few layers (particularly the last few layers of the deep neural network model, as these layers play a crucial role in determining the model's behavior), while keeping the other layers deterministic.

[0008] According to one embodiment, a method for training a deep neural network (DNN) is provided, the DNN being configured with a plurality of weight candidate sets. The method includes: inputting training data selected from a training dataset into the DNN; calculating, based on the training data, a first term indicating the difference between a variational posterior probability distribution of the DNN and a true posterior probability distribution; perturbing the training data to generate perturbed training data; calculating a second term indicating a quantification of prediction uncertainty on the perturbed training data; and updating the plurality of weight candidate sets of the DNN based on increasing the sum of the first and second terms.

[0009] According to a further embodiment, the method for training a DNN further includes: the plurality of weight candidate sets includes a first weight subset and a plurality of second weight candidate subsets, each of the plurality of weight candidate sets including the first weight subset and a second subset of the plurality of second weight candidate subsets.

[0010] According to another embodiment, a method for adversarial detection using a DNN trained using methods for training deep neural networks (DNNs) is provided below. The method includes: feeding an input to the DNN; generating one or more task-related predictions of the input; concurrently estimating the prediction uncertainty of the one or more task-related predictions; and determining whether to accept the one or more task-related predictions based on the prediction uncertainty.

[0011] According to one embodiment, a method is provided for training an image classifier comprising a deep neural network (DNN), the DNN being configured with a plurality of weight candidate sets. The method includes: inputting image training data selected from an image training dataset into the DNN; calculating, based on the image training data, a first term indicating the difference between a variational posterior probability distribution of the DNN and a true posterior probability distribution; perturbing the image training data to generate perturbed training data; calculating a second term indicating a quantification of prediction uncertainty on the perturbed training data; and updating the plurality of weight candidate sets of the DNN based on increasing the sum of the first and second terms.

[0012] According to one embodiment, a method is provided for training an object detector comprising a deep neural network (DNN) configured with a plurality of weight candidate sets. The method includes: inputting photo training data selected from a photo training dataset into the DNN; calculating, based on the photo training data, a first term indicating the difference between a variational posterior probability distribution of the DNN and a true posterior probability distribution; perturbing the photo training data to generate perturbed training data; calculating a second term indicating a quantification of prediction uncertainty on the perturbed training data; and updating the plurality of weight candidate sets of the DNN based on increasing the sum of the first and second terms.

[0013] According to one embodiment, a method is provided for training a speech recognition system including a deep neural network (DNN), the DNN being configured with a plurality of weight candidate sets. The method includes: inputting speech training data selected from a speech training dataset into the DNN; calculating, based on the speech training data, a first term indicating the difference between a variational posterior probability distribution of the DNN and a true posterior probability distribution; perturbing the speech training data to generate perturbed training data; calculating a second term indicating a quantification of prediction uncertainty on the perturbed training data; and updating the plurality of weight candidate sets of the DNN based on increasing the sum of the first and second terms.

[0014] This disclosure enables DNNs to quickly and inexpensively acquire the ability to detect various adversarial examples when faced with new tasks such as image classification, face recognition, object detection, and speech recognition. Furthermore, all downstream systems of deep neural networks (e.g., autonomous vehicles, industrial product anomaly detection systems, medical diagnostic systems, text-to-speech systems, image recognition systems, etc.) will be more robust against adversarial attacks. Attached Figure Description

[0015] The disclosed aspects will be described in conjunction with the accompanying drawings, which are provided to illustrate and not limit the disclosed aspects.

[0016] Figure 1a Exemplary adversarial attacks based on various aspects of this disclosure are illustrated.

[0017] Figure 1b Exemplary adversarial samples in a data manifold according to various aspects of this disclosure are shown.

[0018] Figure 2 Exemplary deep neural network (DNN) 210 and exemplary Bayesian neural network (BNN) 220 are shown in accordance with various aspects of this disclosure.

[0019] Figure 3 This is a flowchart illustrating an exemplary DNN training process 300 according to various aspects of this disclosure.

[0020] Figure 4 This is a block diagram illustrating an exemplary DNN 400 with trained Bayesian submodules according to various aspects of this disclosure.

[0021] Figure 5 This is a flowchart illustrating an exemplary DNN training process 500 according to various aspects of this disclosure.

[0022] Figure 6 This is a flowchart illustrating an exemplary DNN inference process 600 according to various aspects of this disclosure.

[0023] Figure 7 An exemplary computing system 700 according to various aspects of this disclosure is shown. Detailed Implementation

[0024] The present disclosure will now be discussed with reference to several exemplary implementations. It will be understood that these implementations are discussed merely to enable those skilled in the art to better understand and thus implement embodiments of the present disclosure, and not to impose any limitation on the scope of the present disclosure.

[0025] Various embodiments will be described in detail with reference to the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. References to specific examples and embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure.

[0026] Deep neural networks (DNNs) have achieved state-of-the-art performance in a wide variety of machine learning tasks and are becoming increasingly prevalent in fields such as computer vision, speech recognition, natural language processing, and bioinformatics. For example, a typical DNN can include multiple layers, such as an input layer, multiple hidden layers, and an output layer. While DNNs are known to be robust to noisy inputs, they are susceptible to specially crafted adversarial examples because they are not good at quantifying prediction uncertainty and tend to produce overconfident predictions.

[0027] Figure 1a Exemplary adversarial attacks based on various aspects of this disclosure are illustrated.

[0028] exist Figure 1a In the example shown, after receiving input x, which can be an image, audio clip, photograph, etc., the DNN can infer from the input x to output a label y indicating "apple". A perturbation δ can be added to the input x to generate a perturbed input x'. To a human, the perturbed input x' still appears as an apple; however, the DNN might incorrectly classify the perturbed input x' as y', which could be a label indicating "pear". The perturbed input x' can be referred to as an adversarial example, which is intentionally perturbed to induce the DNN to make incorrect predictions with high confidence. In one embodiment, adversarial examples with only small perturbations to the original input that are imperceptible to humans might induce an efficient DNN to misclassify at a high rate. In another embodiment, adversarial examples can induce the DNN to output a specific target class.

[0029] Despite Figure 1a The DNN model shown is for an image classification task; however, it should be understood that a DNN can be a neural network model for any specific task. For example, a DNN can be a convolutional neural network (CNN) model for performing a machine learning task of image classification. For example, a DNN can be a CNN model for performing a machine learning task of object detection in an image, where object detection can be used to track objects in exemplary scenarios such as autonomous driving, surveillance systems, etc. For example, a DNN can be a graph neural network (GNN) such as a graph convolutional network (GCN) model, which can perform a machine learning task of classifying nodes in a graph, which can represent, for example, social networks, biological networks, citation networks, recommender systems, financial systems, etc. It should be understood that various aspects of this disclosure can be applied to, but are not limited to, these exemplary domains or scenarios to improve the security and robustness of these systems through adversarial detection.

[0030] Figure 1b Exemplary adversarial samples in a data manifold according to various aspects of this disclosure are shown.

[0031] In one embodiment, let express A set of training samples, wherein and These are used as input data and labels, respectively. For example, in the training samples... and It could be a picture of an apple and a label indicating an apple, such as... Figure 1a As shown. (Through) Parameterized deep neural networks (DNNs) can be trained frequently via maximum a posteriori estimation (MAP): (1) in, This refers to the prediction distribution of the DNN model. In one embodiment, this is achieved by using the prior distribution... Set to an isotropic Gaussian, the second term in equation (1) is equivalent to having an adjustable coefficient in the optimization. The L2 (weight decay) regularization term. Generally, to find adversarial examples that can induce incorrect inferences in a DNN, the L2 (weight decay) regularization term is used for the DNN model. The corresponding adversarial example is defined as: (2) in, It is the effective perturbation set, in which As a disturbance budget, and As a certain norm (e.g., In one embodiment, the minimization problem in equation (2) is solved based on gradients.

[0032] The central goal of adversarial defense is to protect DNN models from adversarial examples. Making undesirable decisions. Essentially, the problem of distinguishing adversarial examples from benign examples can be viewed as a specialized out-of-distribution (OOD) detection problem, which may be of particular concern in security-sensitive scenarios. Using a DNN model trained on clean data, it is expected that adversarial examples can be identified from the offset data manifold, such as... Figure 1b As shown, although the magnitude of the offset may be subtle and imperceptible to humans.

[0033] exist Figure 1b In the schematic data manifold shown, dashed lines represent DNN models (such as in...). Figure 1aThe decision boundary of the DNN shown is such that the two submanifolds that the DNN model will classify as "+" and "-" are generated by training the DNN model using the training dataset. As shown, by perturbing the input x in the manner shown in equation (2), the adversarial sample x' will be incorrectly classified as the label "+". However, the adversarial sample x' is expected to be identified because it is outside the distribution of the submanifold corresponding to the label "+".

[0034] In this sense, based on various aspects of this disclosure, a Bayesian neural network (BNN) is introduced, which utilizes its principled OOD detection capabilities and data fitting flexibility equivalent to that of a DNN.

[0035] Figure 2 Exemplary deep neural network (DNN) 210 and exemplary Bayesian neural network (BNN) 220 according to various aspects of this disclosure are shown. Typically, DNNs are networks that use point estimates as weights, while BNNs are networks that use probability distributions as weights. This makes BNNs more robust to overfitting and can easily learn from small datasets.

[0036] Adversarial detection requires cognitive uncertainty stemming from insufficient exploration of the data space (e.g., the model has never seen adversarial examples during training). Current DNN architectures lack a measure of uncertainty related to prediction, while Bayesian methods provide uncertainty estimates in the form of probability distributions via their parameters.

[0037] Typically, BNNs rely on parameter priors. Data likelihood of neural network (NN) instantiation To specify. The posterior distribution of the derived parameters. Instead of point estimation in DNNs, this is used. Since accurately deriving the posterior distribution is difficult due to the high nonlinearity of neural networks, variational inference is typically used to approximate the true posterior distribution. Generally, in variational BNNs, a parameter is introduced that has the following characteristics: variational distribution And the evidence lower bound (ELBO) used for learning (scaled by 1 / n) is maximized, as shown in equation (3): (3) in, Represents the variational distribution over the weights Compared with the true Bayesian posterior The Kullback-Leibler (KL) divergence between them.

[0038] The obtained posterior provides an opportunity for robust prediction. For computational traceability, the posterior prediction can be estimated via equation (4): (4) in, This represents a Monte Carlo (MC) sample. In other words, compared to a DNN that only cares about the most likely parameter points, a BNN aggregates the predictions generated by all possible models (i.e., T models corresponding to T weight sets) to make a more reliable and calibrated decision.

[0039] In one embodiment, given its success in adversarial detection (especially in image classification), the uncertainty measure is the softmax variance.

[0040] In another embodiment, to make the metric applicable to different scenarios, the information flow within the model is mildly assumed to be... , will be with input Corresponding hidden features The prediction variance is considered a measure of uncertainty. An unbiased variance estimator can be used, and The variance of all coordinates can be summarized as a scalar as a measure of uncertainty via equation (5): (5) in, express In parameter samples The following features, among which As Norm. In one embodiment, the task-related predictions and corresponding uncertainty measures of the model can be made simultaneously during inference via equations (4) and (5). In other embodiments, the uncertainty measures can be quantified after the predictions are made.

[0041] While BNNs are attractive for quantifying prediction uncertainty, concerns remain regarding their training efficiency, prediction performance, the quality of uncertainty estimation, and inference speed. This disclosure provides a method for training DNNs to enable the detection of adversarial examples while overcoming the problems of BNNs to achieve a good balance among the aforementioned concerns.

[0042] The core of variational BNNs lies in the configuration of the variational distribution. This variational distribution can include, but is not limited to, mean-field Gaussian, matrix-variable Gaussian, low-rank Gaussian, MC Dropout, multiplicative normalized flow, and even implicit distributions. However, more easily understood variational distributions often focus on a single pattern in the function space, making the resulting uncertainty estimation unreliable.

[0043] This disclosure utilizes a variational distribution that can be interpreted from a variational Bayesian perspective to construct a weighted candidate set. It considers different function patterns and assigns uniform probabilities to them. That is... In one embodiment, This refers to the Dirac delta function; in another embodiment, This refers to the weighted decaying Dirac delta function; in yet another embodiment, This refers to multiple samples from Gaussian sources. Although the Dirac delta function is used as an example below, it will not be a limiting factor. Let represent various discrete probabilities. Therefore, inferring such a variational posterior is equivalent to training C separate DNNs.

[0044] In one embodiment, all layers of the DNN can be trained as Bayesian layers to obtain sufficient capacity.

[0045] In another embodiment, only a few layers of the feature extraction module in the DNN can be trained as Bayesian layers to save computational costs. For example, these few layers of the DNN may consist of only one layer. As another example, these few layers of the DNN may consist of only the last feature extraction layer of the DNN, excluding the task-related output head. As yet another example, these few layers of the DNN may consist of multiple consecutive layers of the feature extraction module of the DNN, which are trained as Bayesian submodules. In this case, the immediate output of the Bayesian submodule can be used as the hidden feature in equation (5). In another respect, DNNs can be trained to have multiple Bayesian submodules.

[0046] In one embodiment, only a few layers of the feature extraction module of the DNN can be trained as Bayesian layers, and the parameter weights of the DNN... Classified as and , which respectively represent the parameter weights of the tiny Bayesian submodule and the weights of other parameters of the DNN model, thus obtaining the few-layer deep ensemble (FADE) variation as shown in equation (6): (6) in, ,and Let represent the Dirac delta function. Intuitively, FADE simplifies and accelerates learning, thus allowing Bayesian inference to be easily extended to deep architectures.

[0047] Given the disclosed FADE variation, this disclosure provides an efficient and user-friendly implementation for learning. The isotropic Gaussian prior is assumed to be used as the MAP estimate for the DNN, and the second term of ELBO in equation (3) reduces to a weight decay regularization term, which has the following characteristics: The coefficient λ on and in coefficients on , This can be easily implemented within the optimizer. Then, only the first term in ELBO needs to be explicitly addressed. Analytically estimating the expectation in the first term is feasible, but may prevent different weight candidates from exploring different function patterns (because they may experience similar optimization trajectories). Therefore, in one embodiment, a stochastic estimate of the first term in ELBO is proposed to maximize it via equation (7) on stochastic gradient ascent: (7) in, It is a random mini-batch selected from the training dataset, and From unif (that is, in) The first term in ELBO is derived from a uniform distribution on the surface. In another embodiment, the first term in ELBO can be maximized by gradient ascent or other common methods.

[0048] However, because it is related to the choice of c, High variance is observed between iterations, which is detrimental to convergence. To resolve this correlation, in one embodiment, parameter samples are batch-wise. It can be replaced with per-instance parameter samples , ,For example, Therefore in The selection is based on independent identity distributions, which ensures... The variable behavior of the Bayesian submodule is comprehensively considered at each iteration. In this embodiment, a stochastic estimate of the first term in the ELBO used for training is proposed via equation (8): (8)

[0049] Under this learning standard, each Bayesian weight candidate Equivalent to The randomized independent subsets are then assigned. This randomness will be injected into the gradient ascent dynamics and, as an implicit regularization, will lead to Bayesian weight candidates. The study investigates different weighted subspaces and ideally different function patterns.

[0050] As a special category of OOD data, adversarial examples possess several unique characteristics, such as close similarity to benign data and a strong offensiveness to the behavior of black-box deep models, which can easily undermine uncertainty-based adversarial detection. A common strategy to address this issue is to incorporate adversarial examples crafted through specific attacks into the detector training; however, this is expensive and may limit the learned model's generalization to unseen attacks.

[0051] Alternatively, this disclosure discloses a strategy for correcting non-adversarial sample uncertainty by considering a superset of adversarial samples. Training instances, including all types of adversarial samples, are uniformly perturbed and fed into a DNN with one or more Bayesian submodules, and are subject to relatively high prediction uncertainty to train a DNN capable of detecting various types of adversarial samples.

[0052] In terms of form, utilizing The training perturbation budget can be expressed as follows: it can be used to pollute mini-batch data via equation (9): (9) Then, using MC samples are used to calculate the uncertainty measure. The result is regularized by solving the following marginal loss shown in equation (10): (10) in, This refers to a given parameter sample In the case of The characteristics, among which ,and . It is an adjustable threshold.

[0053] Using equation (10) to train a DNN requires that the prediction uncertainty on the perturbed samples be at least greater than 100%. Therefore, it has greater than Instances of predictive uncertainty will be considered adversarial samples and rejected during inference.

[0054] In some embodiments, due to the main part of the model (i.e. It maintains determinism, thus enabling it to perform only one forward propagation to reach the entry point of the Bayesian submodule (i.e., Therefore, by employing FADE variational methods, computational speed can be improved in the inference phase following learning. In the Bayesian submodule, all C weight candidates are considered for prediction to fully utilize their heterogeneous predictive behavior, i.e. In one embodiment, the weights are calculated sequentially for each candidate weight. The result is feasible. In another embodiment, further inference acceleration can be achieved through parallel computation of all weight candidates.

[0055] Figure 3This is a flowchart illustrating an exemplary DNN training process 300 according to various aspects of this disclosure. The DNN can be included in image classifiers, speech recognition systems, object detection systems, etc. As described below, in certain implementations within the scope of this disclosure, some or all of the shown features may be omitted, and some shown features may not be necessary for implementations of all embodiments. In some examples, process 300 can be performed by any suitable means or unit for performing the functions or algorithms described below. It should be understood that in Figure 3 The operation instructions shown in the dashed lines are optional operations.

[0056] At position 302, the training process begins by inputting some pre-configured parameters into the DNN, which is configured with multiple weight candidate sets. The pre-configured parameters may include, but are not limited to, the training dataset. Number of candidate sets with weights Weight decay coefficient λ, training perturbation budget Thresholds for uncertainty measurement As shown in the solid-line box 302-0, the training dataset... This can include images, audio, photos, etc., depending on which system the DNN is included in.

[0057] In one embodiment, the pre-configured parameters may further include a pre-trained DNN weight set. As shown in the dashed box 302-1. Assume the posterior parameters... Alignment between the DNN and their corresponding terms is preferred, and performing cost-effective Bayesian refinement on the pre-trained DNN model is preferable, which makes the workflow more suitable for large-scale learning. It should be understood that training a BNN from scratch without using a pre-trained DNN weight set is also feasible.

[0058] In another embodiment, the pre-configured parameters may further include tradeoff coefficients for forming the objective function for learning. As shown in dashed box 302-2. As discussed above, two terms are left for adjustment: the first term indicates the difference between the variational posterior probability distribution of the DNN and the true posterior probability distribution, while the second term indicates the quantification of the prediction uncertainty on the perturbed training data. In one aspect, the objective function for learning is the sum of the first and second terms. In another aspect, the objective function for learning is the sum of the first and second terms, with the second term added to the first term via a weighting coefficient.

[0059] In yet another embodiment, the pre-configured parameters may further include the number of refined epochs. It is used to indicate the traversal of the training dataset. The number of times is shown in the dashed box 302-3.

[0060] At position 304, the training process continues with the following operation: initializing multiple weight candidate sets for the DNN.

[0061] In one embodiment, multiple weight candidate sets Random initialization, for example, using randomly generated values ​​to initialize multiple weight candidate sets.

[0062] In another embodiment, where only a few layers of the DNN are trained as Bayesian layers, multiple weight candidate sets are used. Including the first weighted subset and multiple second-weighted candidate subsets Each of the multiple weight candidate sets includes a first weight subset and a second subset from a plurality of second weight candidate subsets. ,in In this case, multiple weight candidate sets It is based on a pre-trained set of DNN weights. It is initialized, and it is represented as . It can be initialized to ,and It can be initialized to ,in .

[0063] At 306, the training process continues with the following operation: constructing an optimizer with a weight decay coefficient λ. As discussed above, the isotropic Gaussian prior is assumed to be used as the MAP estimate for the DNN, and the second term of ELBO in equation (3) is reduced to a weight decay regularization term with a coefficient λ.

[0064] In one embodiment, when all layers of the DNN are trained as Bayesian layers, it is possible to target... Construct with weight decay The optimizer optb.

[0065] In another embodiment, when only a few layers of the DNN are trained as Bayesian layers, they can be targeted individually. Construct an optimizer opt-b with weight decay λ, and target it. Construct with weight decay The optimizer optb.

[0066] Continuing from this point, the training process involves fine-tuning the variational parameters using a weight decay regularization term with appropriate coefficients to increase the objective function used for training, thereby achieving posterior inference for adversarial detection.

[0067] At point 308, the training process continues with the following operations: mini-batch training data... The data is input into the DNN. In one embodiment, mini-batch training data is used. From the training dataset It is randomly selected from among them.

[0068] At point 310, the training process continues with the following operations: for mini-batch training data... The first term of the objective function is calculated, where the first term indicates the difference between the variational posterior probability distribution of the DNN and the true posterior probability distribution.

[0069] In one embodiment, the first term is the evidence lower bound (ELBO) of the variational posterior probability distribution in equation (3).

[0070] In another embodiment, the first term of the objective function is based on batch-wise weighted samples corresponding to the parameters of the Bayesian layer. The calculated log-likelihood function As discussed in equation (7), where the batch-wise weighted samples It is randomly selected based on a small batch of training data.

[0071] In yet another embodiment, the first term of the objective function is based on instance-by-instance weighted samples. The calculated log-likelihood function As discussed above in equation (8), where the instance-by-instance weighted sample It is randomly selected for each training data sample in the mini-batch training data.

[0072] At 312, the training process continues by perturbing the mini-batch training data to generate perturbed training data, as discussed above in equation (9).

[0073] In one embodiment, the training data in mini-batches can be uniformly perturbed to include all kinds of adversarial samples.

[0074] In another embodiment, mini-batch training data can be perturbed to target specific types of adversarial samples.

[0075] At position 314, the training process continues by calculating the second term of the objective function for the perturbed mini-batch training data, where the second term indicates the quantification of the prediction uncertainty on the perturbed training data.

[0076] In one embodiment, the second term of the objective function can be the prediction variance of the hidden features corresponding to the training data, as discussed above in equation (5). .

[0077] In another embodiment, the second term of the objective function may be the prediction uncertainty on the perturbed training data. With a predefined threshold In contrast, the regularization term, as discussed above in equation (10), .

[0078] At position 316, the training process continues by backpropagating the gradient of the objective function, which is the sum of the first and second terms, and updating the multiple weight candidate sets of the DNN based on increasing the objective function. For example, the objective function is increased by performing stochastic gradient ascent.

[0079] In one embodiment, the objective function is the sum of the first and second terms. In another embodiment, the objective function is formed by adding the second term to the first term using a weighted coefficient.

[0080] In one embodiment, when all layers of a DNN are to be trained as Bayesian layers, the multiple weight candidate sets of the DNN can be updated by an optimizer with optb.

[0081] In another embodiment, when only a few layers in the DNN are trained as Bayesian layers, the first weight subset and multiple second weight candidate subsets can be updated separately by an optimizer with optb and opt-b.

[0082] At point 318, the training process continues by determining whether the iteration should be terminated.

[0083] In one embodiment, the update is performed until the sum of the values ​​of the objective function is maximized.

[0084] In another embodiment, the update is performed until the value of the objective function meets a threshold.

[0085] In yet another embodiment, updates are performed until the value of the objective function converges.

[0086] In yet another embodiment, the training dataset is updated in refined rounds. The number of times specified.

[0087] If it is determined at 318 that the iteration has not been terminated, the process loops back to 308; otherwise, the process ends at 320.

[0088] An exemplary algorithm process is given in pseudocode in Table 1 below. Table 1

[0089] Figure 4 This is a block diagram illustrating an exemplary DNN 400 with Bayesian submodules according to various aspects of this disclosure.

[0090] like Figure 4 As shown, a Bayesian submodule 410 exists in the DNN 400. The Bayesian submodule 410 includes at least one layer of the DNN, while the other layers 420 of the DNN 400 remain deterministic.

[0091] In one embodiment, output layer 420-1 outputs a task-related output header for task-related prediction 430 based on features from a previous layer (such as at least one layer of the Bayesian submodule 410). In this embodiment, at least one layer of the Bayesian submodule 410 includes a last layer that excludes the task-related output header.

[0092] Figure 4 This is merely an example and not a limitation. As discussed above, a DNN can be trained as a BNN, trained to have several consecutive layers as Bayesian submodules, trained to have several Bayesian submodules, trained to have only one layer as a Bayesian layer (excluding the task-related output header), and so on.

[0093] Bayesian submodule 410 can compute an uncertainty measure based on features obtained in at least one layer of submodule 410 during inference, for example, based on equation (5). At 440, the uncertainty measure can be used to determine whether the current input x is benign or adversarial. Then, at 450, the determination obtained at 440 can be used to accept or reject task-related predictions.

[0094] Figure 5 This is a flowchart illustrating an exemplary method for training a deep neural network (DNN) capable of adversarial detection, according to various aspects of this disclosure.

[0095] At point 510, training data selected from the training dataset is input into the DNN, which is configured with multiple sets of weight candidates. The training dataset may include images, audio, photos, etc., depending on which system the DNN is included in.

[0096] At position 520, the first term is computed based on the training data to indicate the difference between the variational posterior probability distribution of the DNN and the true posterior probability distribution.

[0097] At position 530, the training data is perturbed to generate perturbed training data.

[0098] At position 540, a second term is calculated based on the perturbed training data to indicate the quantification of the prediction uncertainty on the perturbed training data.

[0099] At position 550, the multiple weight candidate sets of the DNN are updated based on increasing the sum of the first and second terms.

[0100] In one embodiment, the plurality of weight candidate sets include a first weight subset and a plurality of second weight candidate subsets, each of the plurality of weight candidate sets including the first weight subset and a second subset of the plurality of second weight candidate subsets.

[0101] In one embodiment, the first weight subset is updated using the training dataset.

[0102] In one embodiment, training data randomly selected from the training dataset is used to update each of the multiple second subsets of second weight subsets.

[0103] In one embodiment, each of the multiple second weight candidate subsets corresponds to at least one layer of the DNN.

[0104] In one embodiment, at least one layer of the DNN includes the last layer of the DNN.

[0105] In one embodiment, at least one layer of a DNN comprises multiple consecutive layers of the DNN.

[0106] In one embodiment, the DNN is pre-trained, and a first weight subset and a plurality of second weight candidate subsets are initialized using the pre-trained weights of the DNN.

[0107] In one embodiment, the sum of the first and second terms is increased by performing stochastic gradient ascent.

[0108] In one embodiment, the update at point 550 can be performed by updating a first weight subset using a first optimizer with a first weight decay coefficient, and updating a plurality of second weight candidate subsets using a second optimizer with a second weight decay coefficient.

[0109] In one embodiment, the training data is perturbed uniformly, and the perturbation is within the training perturbation budget.

[0110] In one embodiment, the first term is the lower bound of evidence (ELBO) for the variational posterior probability distribution.

[0111] In one embodiment, the first term is the log-likelihood function over the training data, where each instance of the training data corresponds to a randomly assigned set of weight candidates from a plurality of weight candidate sets.

[0112] In one embodiment, the prediction uncertainty on an instance is computed as a scalar, which indicates the variance of the instance’s hidden features in at least one layer of the DNN corresponding to a second weight candidate subset.

[0113] In one embodiment, the second term is a regularization term that compares the prediction uncertainty on the perturbed training data with an adjustable threshold.

[0114] In one embodiment, the second term is added to the first term using a weighting factor.

[0115] Figure 6 This is a flowchart of an exemplary DNN inference process 500 based on various aspects of this disclosure.

[0116] At 610, the inference process begins by feeding the input to the DNN, which includes, but is not limited to, images, audio, signatures, medical imaging, etc.

[0117] At position 620, the inference process continues by using deterministic and Bayesian layers to generate one or more task-related predictions for the input, including but not limited to image classification, node classification, face recognition, object detection, and speech recognition.

[0118] At 630, the inference process continues by using a Bayesian layer to estimate the prediction uncertainty of one or more task-related predictions. Operations at 604 and 606 can be performed sequentially or concurrently.

[0119] At 640, the inference process continues by determining whether to accept one or more task-related predictions based on the prediction uncertainty. In one embodiment, the determination further includes comparing the prediction uncertainty of the one or more task-related predictions with a threshold, which may be a threshold used during training. If the prediction uncertainty of one or more task-related predictions exceeds a threshold, indicating that the input is highly likely to be an adversarial sample rather than a benign sample, then one or more task-related predictions are rejected.

[0120] Figure 7An exemplary computing system 700 according to one embodiment is illustrated. The computing system 700 may include at least one processor 710. The computing system 700 may also include at least one storage device 720. In one aspect, the storage device 720 may store computer-executable instructions that, when executed, cause the processor 710 to: input training data selected from a training dataset into a distributed neural network (DNN), the DNN being configured with a plurality of weight candidate sets; compute a first term based on the training data to indicate the difference between the variational posterior probability distribution of the DNN and the true posterior probability distribution; perturb the training data to produce perturbed training data; compute a second term to indicate a quantification of the prediction uncertainty on the perturbed training data; and update the plurality of weight candidate sets of the DNN based on increasing the sum of the first and second terms.

[0121] In a further aspect, the storage device 720 may store computer-executable instructions that, when executed, cause the processor 710 to perform the following operations: feeding input to a DNN trained using the methods of this disclosure; generating one or more task-related predictions of the input; concurrently estimating the prediction uncertainty of one or more task-related predictions; and determining whether to accept one or more task-related predictions based on the prediction uncertainty.

[0122] It should be understood that the storage device 720 may store computer-executable instructions that, when executed, cause the processor 710 to perform any operation as described in conjunction with the embodiments of the present disclosure according to Figures 1-6.

[0123] Embodiments of this disclosure may be embodied in a computer-readable medium, such as a non-transitory computer-readable medium. A non-transitory computer-readable medium may include instructions that, when executed, cause one or more processors to perform any operation as described in conjunction with embodiments of this disclosure according to Figures 1-6.

[0124] Embodiments of this disclosure may be embodied in a computer program product, which includes computer-executable instructions that, when executed, cause one or more processors to perform any operation as described in conjunction with embodiments of this disclosure according to Figures 1-6.

[0125] It should be understood that all operations in the above methods are merely exemplary, and this disclosure is not limited to any operation in these methods or the sequence of such operations, and should cover all other equivalents under the same or similar concept.

[0126] It should also be understood that all modules in the aforementioned apparatus can be implemented using various methods. These modules can be implemented as hardware, software, or a combination thereof. Furthermore, any of these modules can be further functionally divided into sub-modules or combined together.

[0127] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to limit them to the aspects shown herein. All structural and functional equivalents of elements known to or to be known later by those skilled in the art throughout the various aspects described in this disclosure are expressly incorporated herein by reference and are intended to be covered by the claims.

Claims

1. A method for training a deep neural network (DNN) capable of adversarial detection, wherein, The DNN is configured with multiple weight candidate sets, and wherein the DNN includes a first number of deterministic layers and a second number of Bayesian layers, the method comprising: The training data selected from the training dataset is input into the DNN; Based on the training data, a first term is calculated to indicate the difference between the variational posterior probability distribution of the DNN and the true posterior probability distribution. The training data is perturbed to generate perturbed training data; Calculate a second term used to indicate the quantification of the prediction uncertainty on the perturbed training data; and The multiple weight candidate sets of the DNN are updated based on increasing the sum of the first term and the second term, wherein the multiple weight candidate sets include a first weight subset and multiple second weight candidate subsets, the first weight subset is used for the deterministic layer and the second weight candidate subsets are used for the Bayesian layer.

2. The method according to claim 1, wherein, Each of the plurality of weight candidate sets includes the first weight subset and a second subset of the plurality of second weight candidate subsets.

3. The method according to claim 2, wherein, The first weight subset is updated using the training dataset.

4. The method according to claim 2, wherein, Each of the plurality of second weight candidate subsets is updated using training data randomly selected from the training dataset.

5. The method according to claim 2, wherein, Each of the multiple second weight candidate subsets corresponds to the second number of Bayesian layers in the DNN.

6. The method according to claim 5, wherein, The second number of Bayesian layers in the DNN includes the last feature extraction layer of the DNN.

7. The method according to claim 5, wherein, The second number of Bayesian layers of the DNN includes multiple consecutive layers of the DNN.

8. The method according to claim 2, wherein, The DNN is pre-trained, and the first weight subset and a plurality of second weight candidate subsets are initialized using the pre-trained weights of the DNN.

9. The method of claim 2, wherein the sum of the first and second items is increased by performing stochastic gradient ascent.

10. The method of claim 9, wherein updating the plurality of weight candidate sets of the DNN based on increasing the sum of the first and second items further comprises: The first weight subset is updated using a first optimizer with a first weight decay coefficient; as well as The second weight candidate subset is updated using a second optimizer with a second weight decay coefficient.

11. The method according to claim 1, wherein, The training data is uniformly perturbed, and the perturbation is within the training perturbation budget.

12. The method according to claim 1, wherein, The first term is the lower bound of evidence (ELBO) for the variational posterior probability distribution.

13. The method according to claim 1, wherein, The first term is the log-likelihood function over the training data, wherein each instance of the training data corresponds to a randomly assigned set of weight candidates from the plurality of weight candidate sets.

14. The method according to claim 2, wherein, The prediction uncertainty on an instance is calculated as a scalar, which indicates the variance of the instance's hidden features in the Bayesian layer of the DNN corresponding to the second weight candidate subset.

15. The method of claim 1, wherein the second term is a regularization term on the perturbed training data, comparing the prediction uncertainty with an adjustable threshold.

16. The method of claim 1, wherein the second term is added to the first term by a weighting factor.

17. A method for adversarial detection using a deep neural network (DNN) trained according to any one of claims 1-16, comprising: The input is fed into the DNN; Generate one or more task-related predictions for the input; Concurrently estimate the prediction uncertainty of the one or more task-related predictions; as well as The decision to accept one or more task-related predictions is based on the predicted uncertainty.

18. A computer system, comprising: One or more processors; as well as One or more storage devices storing computer-executable instructions, which, when executed, cause the one or more processors to perform the operation of the method according to any one of claims 1-17.

19. One or more computer-readable storage media storing computer-executable instructions, which, when executed, cause one or more processors to perform the operations of the method according to any one of claims 1-17.

20. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed, cause one or more processors to perform the operation of the method according to any one of claims 1-17.

Images