Method and system for training neural networks to improve robustness against adversarial attacks

Abstract

An embodiment of the present disclosure discloses a method and system for training a neural network to improve robustness against adversarial attacks. The method includes collecting a plurality of data samples, including clean data samples and adversarial data samples. Training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over the latent space representation. In addition, training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result. In addition, the method includes training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples. Furthermore, the method includes outputting the shared parameters of the first instance of the neural network and the second instance of the neural network.

Description

[Technical Field]

[0001] The present disclosure relates generally to adversarial data perturbation, and more particularly to methods and systems for training neural networks to improve robustness against adversarial behavior. [Background technology]

[0002] In recent years, machine learning and deep neural networks have been widely used for data classification. However, machine learning models are often vulnerable to attacks based on adversarial manipulation of data. One form of adversarial manipulation of data is known as an adversarial example. An adversarial example is a data sample intentionally modified with small feature perturbations. These feature perturbations are intended to cause a machine learning or deep neural network (ML / DNN) model to output inaccurate predictions. In particular, feature perturbations are imperceptible noise in the data, causing the ML classifier to misclassify the data. Such adversarial examples can be used to perform attacks on ML systems, which raises security concerns. Adversarial examples pose a potential security threat to ML applications such as robots that perceive the world through cameras and other sensors, video surveillance systems, and mobile applications for image or audio classification.

[0003] Adversarial sample attacks are broadly categorized into two classes of threat models: white-box adversarial attacks and black-box attacks. In white-box adversarial attacks, the attacker accesses the target model's parameters. For example, the attacker accesses the target model's architecture, weights, gradients, and other parameters. White-box adversarial attacks require strong adversarial access to successfully attack. In addition, such white-box adversarial attacks have higher computational overhead, such as time and attack iterations. In contrast, black-box adversarial attacks have limited adversarial access to the target model's parameters. For example, the adversarial access only involves accessing example input and output data pairs for the target model. Alternatively, black-box adversarial attacks do not use any information about the target model. In such adversarial attacks, an alternative or source model is trained with training data to generate adversarial perturbations. The generated adversarial perturbations are then applied to the input data to attack the target black-box DNN. For example, an input image is input to an alternative model to generate adversarial perturbations. Adversarial perturbations are then added to the input image to attack the target black-box DNN. In some cases, model queries are used to obtain information from the target black-box DNN.

[0004] Traditional techniques for making machine learning models more robust, such as weight decay and dropout, generally do not provide practical defenses against adversarial examples. To date, only two methods have provided significant defenses: adversarial training and defensive distillation. Adversarial training is a brute-force approach that generates many adversarial examples and explicitly trains a model to resist being fooled by them. Defensive distillation is a strategy that trains a model to output probabilities of different classes rather than making hard decisions about which class to output. The probabilities are supplied by a previous model trained on the same task using hard class labels. This creates a model whose surface is smoothed in directions that adversaries typically attempt to exploit, making it difficult for adversaries to discover adversarial input fine-tuning that leads to inaccurate classifications. An example of a prior art approach to training a variational information bottleneck that improves its robustness against adversarial perturbations can be found in WEIZHU QIAN ET AL: "Learning Robust Variational Information Bottleneck with Reference", 29 April 2021, XP0819457 4 7. Furthermore, SHUO WANG ET AL: "Defending Adversarial Attacks via Semantic Feature Manipulation", 4 February 2020, XP081649950 discloses a one-off and attack-agnostic feature manipulation (FM)-defense to detect and cleanse adversarial examples in an interpretable and efficient manner. Finally, MINSEON KIM ET AL: "Adversarial Self-Supervised Contrastive Learning", 26 October 2020, XP081798911, discloses an adversarial attack on unlabeled data that forces a model to confuse the instance-level identities of perturbed data samples.

[0005] However, adversarial examples are difficult to defend against because it is difficult to construct a theoretical model of the process of crafting adversarial examples. Adversarial examples are solutions to optimization problems that are nonlinear and nonconvex for many ML models, including neural networks. Adversarial examples are also difficult to defend against because they require machine learning models to generate good outputs for all possible inputs. In most cases, machine learning models perform very well, but only for a small fraction of the many possible inputs they may encounter.

[0006] In addition, current techniques for making machine learning models more robust are not adaptive because they may block some types of attacks but leave vulnerabilities unprotected against other attackers. Therefore, designing defenses that can protect against powerful adaptive attackers is a critical but hitherto unsolved technical problem.

[0007] Therefore, there is a need to overcome the above problems. More specifically, there is a need to develop a method and system for training neural networks to improve their robustness to adversarial attacks while maintaining natural accuracy. Summary of the Invention

[0008] It is an object of some embodiments to provide systems and methods for training robust neural network models with improved resistance to adversarial attacks. Additionally or alternatively, it is an object of some embodiments to provide systems and methods for classifying input data using trained neural networks with improved robustness to adversarial attacks. Additionally or alternatively, it is an object of some embodiments to provide systems and methods for probabilistically classifying input data to improve classification accuracy under or despite adversarial attacks.

[0009] To that end, some embodiments disclose a neural network including a stochastic encoder configured to encode input data of a plurality of data samples into a distribution over a latent space representation, and a classifier configured to classify the encoding of the input data in the latent space representation. The stochastic encoder is in contrast to a deterministic encoder. While a deterministic encoder encodes the input data into a latent space representation, a stochastic encoder encodes the input data into a distribution over the latent space representation. For example, to encode the input data in the distribution of the latent space representation, the stochastic encoder can output parameters of the distribution.

[0010] First, the classifier does not classify a distribution over the latent space representation, but classifies an instance (first instance or second instance) of the latent space representation or a sample of the distribution encoded by the probabilistic encoder. This allows sampling the output of the probabilistic encoder multiple times and combining the results of the classification to obtain a more accurate classification result.

[0011] Second, probabilistic coding allows for improved neural network training by imposing additional requirements not only on the classification results but also on the distribution over the latent space representation itself. Both of these benefits, alone or in combination, improve the robustness of the trained neural network to adversarial attacks.

[0012] For example, some embodiments recognize that the performance of machine learning methods depends on the choice of data representation, and the goal of representation learning is to transform a raw input x into a lower-dimensional representation z that preserves relevant information for tasks such as classification or regression. Adversarial examples are solutions to optimization problems that are nonlinear and nonconvex for many ML models. Some embodiments recognize that it is difficult to provide theoretical tools for describing solutions to these complex optimization problems. The information bottleneck (IB) principle provides an information-theoretic method for representation learning, where the representation should contain only the most relevant information from the input for downstream tasks. Representations learned by the IB principle may be less affected by disturbing variations and more robust to adversarial perturbations. Additionally, multi-view information bottleneck can extend the IB principle to multi-view unsupervised settings by maximizing information shared between different views while minimizing view-specific information.

[0013] Some embodiments recognize that the multi-view information bottleneck method can be extended to a supervised setting with adversarial training. For example, some embodiments can consider an adversarial sample as another view of the corresponding clean sample. As a result, embodiments attempt to learn representations that include information shared between the clean sample and the corresponding adversarial sample, while eliminating information that is not shared between them. As described above, having a probabilistic encoder that encodes input data into a distribution over the latent space representation, rather than into an instance of the latent space representation, allows different embodiments to explore the theoretical guarantees provided by the multi-view information bottleneck principle to improve the robustness and / or performance of the trained neural network.

[0014] To utilize these principles, some embodiments train shared parameters for different instances of a neural network by optimizing a multi-objective loss function for the outputs of the different instances using pairs of clean and adversarial data samples. Because the different instances are instances of the same neural network that includes a probabilistic encoder and a classifier, the outputs of the different instances (the first instance and the second instance) include parameters for the probability distribution of the latent space representation and the classification results. By comparing and optimizing the difference between these outputs, resistance to adversarial attacks is improved.

[0015] Accordingly, one embodiment discloses a computer-implemented method for training a neural network. The method includes collecting a plurality of data samples, including clean data samples and adversarial data samples. Training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over a latent space representation. Additionally, training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result. Additionally, the method includes training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples. Furthermore, the method includes outputting the shared parameters between the first instance of the neural network and the second instance of the neural network.

[0016] Accordingly, another embodiment discloses an AI system for training a neural network. The AI ​​system includes a processor and a memory having instructions stored thereon. The processor is configured to execute the stored instructions to cause the AI ​​system to collect a plurality of data samples as input for training the neural network. The plurality of data samples includes clean data samples and adversarial data samples. Training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over a latent space representation. Further, training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result. Furthermore, the processor causes the AI ​​system to train shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples. Further, the processor causes the AI ​​system to output the shared parameters between the first instance of the neural network and the second instance of the neural network. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a schematic block diagram of a system for training a neural network to improve robustness against adversarial attacks, in accordance with some embodiments of the present disclosure. [Figure 2A] FIG. 1 is a schematic diagram of an AI system for training a neural network to improve robustness against adversarial attacks, according to some embodiments of the present disclosure. [Figure 2B] FIG. 10 is a block diagram of a representation of z in terms of x and x′ for mutual information sufficiency and minimality, according to various embodiments of the present disclosure. [Figure 3] 1 is a diagram illustrating a procedure for training a neural network to improve robustness against adversarial attacks, according to some embodiments of the present disclosure. [Figure 4] 1 is a representation illustrating a multi-objective loss function according to some embodiments of the present disclosure. [Figure 5] FIG. 1 is a block diagram of an AI system for generating adversarial data samples for training a neural network, according to some embodiments of the present invention. [Figure 6] FIG. 1 is a block diagram of a computer-based system for improving robustness against adversarial attacks, according to some embodiments of the present invention. [Figure 7] 10A-10C are use cases for using an AI system according to some other embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0018] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, devices and methods are shown only in block diagram form in order to avoid obscuring the present disclosure.

[0019] As used in this specification and claims, the words "for example," "for example," "e.g.," "e.g.," and the verbs "comprise," "have," "include," and other verb forms thereof, when used in conjunction with a list of one or more components or other items, should each be construed as open-ended, meaning that the list should not be viewed as excluding other additional components or items. The phrase "based on" means based at least in part on. Furthermore, it should be understood that the phraseology and terminology used herein are for purposes of description and should not be considered limiting. Any headings used within this description are for convenience only and have no legal or limiting effect.

[0020] 1 shows a schematic block diagram of a system 100 for training a neural network, such as neural network 110, to improve robustness against adversarial attacks, in accordance with some embodiments of the present disclosure. System 100 includes a plurality of data samples 102, neural network 110, classifier 104A, classifier 104B, probabilistic encoder 106A, and probabilistic encoder 106B.

[0021] The system 100 collects a plurality of data samples 102 as inputs for training the neural network 110. In some embodiments, the plurality of data samples 102 includes clean data samples x and adversarial data samples x′, where the clean data samples x are ground truth data used to train the neural network 110. As used herein, the adversarial data samples x′ refer to incorrect data samples (e.g., data samples with some perturbation) used to train the neural network 110. Additionally, training the neural network 110 involves fitting the plurality of data samples (i.e., the clean data samples x) to probability distributions (e.g., associated clean cross entropy σ) over the latent space representation z.

number

[0022] More specifically, the system 100 is configured to first train the neural network 110 based on clean data samples x. The clean data samples x are provided as inputs to the probabilistic encoder 106A. The probabilistic encoder 106A is further configured to generate a probabilistic representation (i.e., intermediate representation) z based on the execution of the probabilistic encoder 106A. The probabilistic representation corresponds to the latent space representation z. The probabilistic representation z is passed through the remaining layers 108A. The remaining layers 108A correspond to hidden layers of the neural network 110. Furthermore, the system 100 generates a clean cross-entropy

number

[0023] Similarly, training the neural network 110 includes training the probability encoder 106B to encode a plurality of data samples (i.e., the adversarial data samples x′) into a probability distribution over the latent space representation z′. Training the neural network 110 includes training the classifier 104B to classify instances (e.g., the second instance 114) of the latent space representation z′ to generate a classification result.

[0024] More specifically, the system 100 is configured to first train a neural network 110 based on adversarial data samples x'. The adversarial data samples x' are provided as inputs to a probabilistic encoder 106B. The probabilistic encoder 106B is further configured to generate a probabilistic representation (i.e., intermediate representation) z' based on the execution of the probabilistic encoder 106B. The probabilistic representation z' is passed through a hidden layer 108B. Furthermore, the system 100 is configured to perform adversarial cross-entropy (ACR) analysis.

number

[0025] The system 100 is further configured to train the shared parameters of the first instance 112 of the neural network 110 using the clean data sample x. Similarly, the system 100 is configured to train the shared parameters of the second instance 114 of the neural network 110 using the adversarial data sample x′.

[0026] Furthermore, system 100 is configured to generate output 116 based on shared parameters of first instance 112 of neural network 110 and second instance 114 of neural network 110. In this manner, neural network 110 is trained based on probabilistic representations z corresponding to clean data samples x and probabilistic representations z′ corresponding to adversarial data samples x′.

[0027] In one embodiment, system 100 is configured to train neural network 110 such that probabilistic representations z and z' contain shared information (i.e., mutual information) between x and x'. To achieve this, system 100 is configured to minimize the Kullback-Leibler divergence (KL divergence) between the latent space distributions generated by probabilistic encoder 106A and the latent space distributions generated by probabilistic encoder 106B, and maximize the shared information (i.e., mutual information) between z and z'.

[0028] System 100 is an artificial intelligence-based system (hereinafter AI system) that is further described in Figures 2A and 2B.

[0029] 2A shows a schematic block diagram 200A of an AI system 202 for training a neural network, such as neural network 110, to improve robustness against adversarial attacks, in accordance with some embodiments of the present disclosure. The AI ​​system 202 includes a processor 204 and a memory 206. The memory 206 stores instructions to be executed by the processor 204. The memory 206 also includes the neural network 110. The processor 204 is configured to execute the stored instructions to cause the AI ​​system 202 to collect a plurality of data samples 102 as input for training the neural network 110.

[0030] In some embodiments, examples of processor 204 include, but are not limited to, an application-specific integrated circuit (ASIC) processor, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a field programmable gate array (FPGA), etc. In some embodiments, memory 206 comprises logic, circuitry, and / or interfaces suitable for storing a set of computer-readable instructions for performing operations. Additionally, examples of memory 206 may include random access memory (RAM), read-only memory (ROM), a removable storage drive, a hard disk drive (HDD), etc. It will be apparent to those skilled in the art that the scope of the present disclosure is not limited to implementing memory 206 in AI system 202 as described herein.

[0031] 2A, a plurality of data samples 102 are provided as input to an AI system 202. The AI ​​system 202 invokes a processor 204 to execute instructions stored in a memory 206 to begin training the neural network 110.

[0032] In some embodiments, the plurality of data samples 102 includes a clean data sample x and an adversarial data sample x'. The AI ​​system 202 is configured to train the neural network 110 to improve the adversarial robustness of the neural network 110. In some embodiments, the neural network 110 includes a deep neural network (DNN), or the like. In some embodiments, the AI ​​system 202 is configured to perform the training of the neural network 110 in a supervised setting.

[0033] In some embodiments, the AI ​​system 202 is configured to train the neural network 110 based on a multi-objective loss function. The AI ​​system 202 is configured to train the neural network 110 with the objectives of (1) maximizing the shared information 116 between matched pairs of probabilistic representations, and (2) minimizing the shared information 116 between each probabilistic representation and the corresponding view of each probabilistic representation conditioned on the other views along with (3) a clean cross-entropy loss, and (4) an adversarial cross-entropy loss. For example, item (1) corresponds to maximizing the mutual information objective function value, and item (2) corresponds to minimizing the KL information objective function value.

[0034] The AI ​​system 202 is configured to improve robustness against adversarial attacks based on maximizing shared information 116 between probabilistic representations z and z', as captured by an objective function that maximizes mutual information between probabilistic representations z and z' corresponding to a matched pair of clean data sample x and adversarial data sample x'. Additionally, the training objectives of the neural network 110 include the symmetrized KL divergence between the posterior feature distributions of the clean data sample x and the adversarial data sample x', and the shared information 116 between the latent representations of the clean data sample x and the adversarial data sample x'.

[0035]

number

[0036]

number

[0037] The AI ​​system 202 is configured to learn latent space representations z and z' (corresponding to x and x', respectively), which contain only useful information shared by both x and x'. Mathematically, the generation of these representations is defined by the conditional distributions p(z|x) and p(z'|x'), satisfying the Markov chain z → x → x' → z'.

[0038] The AI ​​system 202 is further configured to improve generalization by learning a representation z or z' that captures only the information shared between x and x'. If a representation preserves only the shared information 116 (i.e., mutual information) from both x and x', it means that it contains only task-relevant information while discarding view-specific details (i.e., information from x' that leads to misperception), thus improving the robustness of the neural network 110 against adversarial attacks.

[0039] FIG. 2B illustrates a block diagram 200B of a representation of z in terms of x and x′ for mutual information sufficiency and minimality, according to various embodiments of the present disclosure.

[0040] Considering that I(z;x) is subdivided into three components by using the chain rule of mutual information, the Markov chain z→x→x' holds, so

number

[0041] Here, I(x;z|x') represents information in z that is unique to x and not shared by x', called view-specific information. The second term I(x;x') indicates the shared information 116 between x and x'. The last term I(x;x'|z) is the shared information 116 missing in z. The main goal here is for representation z to contain only the shared information 116 of x and x', so I(x;z) = I(x;x'). Therefore, the goal here is to minimize I(x;z|x') and I(x;x'|z). Representation z is defined as sufficient and minimal for all downstream tasks because it contains all task-relevant information (sufficiency) without including any extraneous information (minimality).

[0042] Diagram 200B includes a representation of sufficient, but not minimal, mutual information (a):

number

number

number

number

[0043] 3 shows a diagrammatic representation illustrating a procedure 300 for training the neural network 110, according to some embodiments of the present disclosure. The procedure 300 is performed by the AI ​​system 200.

[0044] In step 302, a plurality of data samples 102 are collected. The plurality of data samples 102 includes clean data samples x and adversarial data samples x′, where the clean data samples x are ground truth data used to train the neural network 110. As used herein, the adversarial data samples x′ refer to inaccurate data samples (e.g., data samples with some perturbation) used to train the neural network 110.

[0045] In step 304, training of the neural network 110 is performed. Training of the neural network 110 includes encoding the plurality of data samples 102 into probability distributions over the latent space representations z and z′. The plurality of data samples 102 are encoded using probability encoders 106A and 106B. The probability encoders 106A and 106B encode the plurality of data samples (i.e., clean data samples x and adversarial data samples x′) into probability distributions over the latent space representations z and z′, respectively (e.g.,

number

[0046] In step 306, the shared parameters of the first instance 112 of the neural network 110 and the second instance 114 of the neural network 110 are trained. The shared parameters of the first instance 112 of the neural network 110 are trained using the clean data sample x. The shared parameters of the second instance 114 of the neural network are trained using the adversarial data sample x'.

[0047] The neural network 110 is trained based on a multi-objective loss function. The first instance 112 of the neural network 110 and the second instance 114 of the neural network 110 are jointly trained to minimize the multi-objective loss function of the difference between corresponding outputs of the first instance 112 and the second instance 114 of the neural network 110. The corresponding outputs include the difference between the probability distributions determined by the probability encoders 106A and 106B of the first instance 112 and the second instance 114 of the neural network 110 and the classification results determined by the classifiers 104A and 104B of the first instance 112 and the second instance 114 of the neural network 110. The joint training of the first instance 112 of the neural network 110 and the second instance 114 of the neural network 114 is performed using latent representations z and z′ of the clean data sample x and the adversarial sample x′, respectively, which are sampled multiple times.

[0048] The AI ​​system 202 is configured to train the neural network 110 with the objectives of (1) maximizing the shared information 116 between matched pairs of probabilistic representations, and (2) minimizing the shared information 116 between each probabilistic representation and the corresponding view of each probabilistic representation conditioned on the other views along with (3) a clean cross-entropy loss, and (4) an adversarial cross-entropy loss.

[0049] The AI ​​system 202 is configured to improve robustness against adversarial attacks based on learning the shared information 116 or output between the clean data sample x and the adversarial data sample x′. Additionally, the training objectives of the neural network 110 include the symmetrized KL divergence between the posterior feature distributions of the clean data sample x and the adversarial data sample x′, and the shared information 116 between the latent representations of the clean data sample x and the adversarial data sample x′.

[0050] In step 308, the shared parameters of the first instance 112 and the second instance 114 of the neural network 110 are output.

[0051] 4 illustrates a representation 400 showing a multi-objective loss function 402 according to some embodiments of the present disclosure. In some embodiments, the neural network 110 of FIG. 1 calculates a multi-objective loss function 402 by combining the mutual information of the distributions over the latent space representations z and z′ determined by the probabilistic encoders 106A and 106B of the first and second instances 112 and 114 of the neural network 110, respectively, and the entropy loss of the classification results produced by the first and second instances 112 and 114 of the neural network 110.

number

[0052] In addition, the multi-objective loss function 402 includes terms corresponding to maximizing the mutual information between the probability distributions of the paired encodings of the clean data sample x and the adversarial data sample x′, minimizing the mutual information between the encodings of either the clean data sample x or the adversarial data sample x′ in the pair conditioned on one of the clean data sample x or the adversarial data sample x′ in the pair, and the clean cross-entropy loss required to classify the clean data sample x and the adversarial cross-entropy loss required to classify the adversarial data sample x′.

[0053] 2A-2B, the AI ​​system 202 is configured to learn a representation that includes only the shared information of x and x' by minimizing the view-specific information (I(x;z|x')) and the shared information not in z (I(x;x'|z)). In particular, minimizing I(x;x'|z) is equivalent to maximizing I(z;x') because I(z;x')=I(x;x')-I(x;x'|z), and I(x;x') is constant given x and x'.

number

[0054]

number

[0055] where λ1 and λ2 represent the Lagrange multipliers for constrained optimization. The objective function contains two mutual information terms that are difficult to calculate directly. To solve this problem, some alternative bounds on these two mutual information terms are derived.

[0056] Upper bound on I(x;z|x'): First, an upper bound on the view-specific information in the latent representation z is derived from the input. For example, I(x;z|x') can be computed as follows:

number

[0057] Here, the conditional distributions p(z|x) and p(z'|x') may be parameterized by the encoder network. Furthermore, this bound is strict whenever the representation z is the same as z'. Symmetrically, I(x';z'|x) is given by

number

[0058] Lower bound on I(z;x'): Furthermore, a lower bound on the mutual information between the clean representation and the corresponding adversarial sample is derived. I(z;x') can be calculated as follows:

number

[0059] Here, I(z;z'|x') = 0, since z', as a representation of x', is part of the Markov chain z → x → x' → z'. Note that this bound is also straightforward from this Markov chain and data processing inequality, but the above derivation shows that this bound is tight when z' is a sufficient statistic of z. Symmetrically, we can derive a similar bound for I(z';x) ≥ I(z;z'). Conceptually, this lower bound captures our goal of preserving information shared between representations despite adversarial perturbations.

[0060]

number

[0061]

number

[0062]

number

[0063] This symmetrized KL information can be calculated directly between two Gaussian posterior distributions. Alternatively, I(z;z') requires the use of a mutual information estimator. This disclosure utilizes the Hilbert-Schmidt independence criterion (HSIC) to measure the independence between z and z' and use this value to replace the mutual information term. Note that HSIC is used as a proxy for mutual information because the dependence between two mini-batch samples in a reproducing kernel Hilbert space (RKHS) can be measured directly without requiring density estimation or using an additional network for mutual information estimation.

[0064]

number

[0065] where α ε [0, 1] balances the tradeoff between cross-entropy loss on clean and adversarial samples, and β and λ adjust the importance of the symmetrized KL information and mutual information terms.

[0066] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automatic implementation may be performed, or at least assisted, through the use of a machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. The necessary tasks may be performed by a processor.

[0067] FIG. 5 shows a block diagram 500 of an AI system 202 for generating adversarial data samples x′ for training a neural network 110, according to some embodiments of the present invention. The block diagram 500 includes a communication channel 502 and a correction module 504. The AI ​​system 202 is configured to collect multiple data samples by performing a first step and a second step. The AI ​​system 202 performs the first step of receiving clean data samples x via the communication channel 502. The communication channel 502 comprises one or a combination of a wired channel and a wireless channel. The AI ​​system performs the second step of correcting each of the clean data samples x using the correction module 504 to generate corresponding adversarial data samples that form pairs of the clean data sample x and the adversarial data sample x′. The correction module 504 applies the adversarial sample generation method to the clean data samples x. The adversarial sample generation method includes one of the following: projected gradient descent, fast gradient sign, limited-memory Broyden-Fletcher-Goldfarb-Shanno, Jacobian-based saliency map attack, or Carlini & Wagner attack.

[0068] FIG. 6 illustrates a block diagram of a computer-based system 600 for improving robustness against adversarial attacks, according to some embodiments of the present disclosure. The system 600 includes at least one processor 604 and a memory 606 having stored thereon instructions, the instructions including executable instructions for execution by the at least one processor 604 during control of the system 600. The memory 606 may be embodied as a storage medium, such as a random access memory (RAM), a read-only memory (ROM), a hard disk, or any combination thereof. For example, the memory 606 stores instructions executable by the at least one processor 604. In one exemplary embodiment, the memory 606 is configured to store a neural network 608. The neural network 608 corresponds to the neural network 110 of FIG. 1.

[0069] The at least one processor 604 may be embodied as a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The at least one processor 604 is operatively connected to the sensor 602 and the receiver 610 via a bus 614. In one embodiment, the at least one processor 604 is configured to collect a plurality of data samples. In some exemplary embodiments, the plurality of data samples are collected from the receiver 610. The receiver 610 is connected to an input device 624 via a network 620. Each of the plurality of data samples is stored in storage 612. In some other exemplary embodiments, the plurality of data samples are collected from the sensor 602. The sensor 602 receives a measured data signal 622 from a source (not shown). In some embodiments, the sensor 602 is configured to sense the data signal 622 based on a source of the sensed data signal 622.

[0070] Additionally or alternatively, system 600 is integrated with a network interface controller (NIC) 618 to receive a plurality of data samples 102 (of FIG. 1) using a network 620. The plurality of data samples includes clean data samples and adversarial data samples.

[0071] The at least one processor 604 is also configured to train the neural network 608 to improve robustness against adversarial attacks. Training the neural network 608 includes encoding the plurality of data samples 102 into probability distributions over the latent space representation. The plurality of data samples 102 are encoded using a probabilistic encoder.

[0072] The trained neural network 608 generates an output of shared information that is transmitted via a transmitter 616. Additionally or alternatively, the transmitter 616 is coupled to an output device 626 to output the shared information over a wireless or wired communication channel, such as a network 620. The output device 626 includes a computer, laptop, smart device, or any computing device used to prevent adversarial attacks on an application installed on the output device 626.

[0073] Also, particular embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operations as a sequential process, many of the operations can be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process may terminate when its operations are completed, or may have additional steps not discussed or included in the diagram. Moreover, not all operations in any particularly described process may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the termination of the function may correspond to a return to the function's calling function or to the main function.

[0074] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automatic implementation may be performed, or at least assisted, through the use of a machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. The necessary tasks may be performed by a processor.

[0075] FIG. 7 illustrates a use case 700 using the AI ​​system 202, according to some other embodiments of the present disclosure. The use case 700 corresponds to a vehicle-assisted navigation system (not shown) of a vehicle 702A and a vehicle 702B. The vehicle-assisted navigation system is connected to the AI ​​system 202. The vehicle-assisted navigation system is connected to a camera of the vehicle 702A, such as a front camera that captures a road scene or view. In one illustrative example scenario, the camera captures a road sign 704 displaying a "No Parking" sign. The captured road sign 704 is transmitted to the AI ​​system 202. The AI ​​system 202 processes the captured road sign 704 using the trained neural network 110. The captured road sign 704 is processed using clean data samples and adversarial data samples to generate a robust model for identifying the "No Parking" sign in the road sign 704. The robust model is used by the vehicle-assisted navigation system to accurately identify road signs 704 and prevent vehicles 702A and 702B from parking in no-parking zones. Finally, one embodiment of the subject matter relates to a computer-implemented method for training a neural network, the method using a processor in combination with stored instructions implementing the method, the instructions, when executed by the processor, performing the steps of the method, the steps including: Collecting pairs of clean and adversarial data samples to train a neural network including a probabilistic encoder trained to encode input data samples into a probability distribution over a latent space and a classifier trained to classify instances in the latent space to generate a classification result; and jointly training parameters of a first instance of the neural network using clean data samples and parameters of a second instance of the neural network using adversarial data samples, wherein the first instance of the neural network and the second instance of the neural network are jointly trained to minimize a multi-objective loss function of differences between corresponding outputs of the first and second instances of the neural network determined for pairs of clean and adversarial data samples, the corresponding outputs comprising differences between probability distributions determined by probabilistic encoders of the first and second instances of the neural network and classification results determined by classifiers of the first and second instances of the neural network, and the steps further comprise: Outputting one or a combination of the parameters of the first instance of the neural network and the parameters of the second instance of the neural network.

[0076] The above-described embodiments of the present disclosure may be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. If implemented in software, the software code may be executed on any suitable processor or collection of processors, whether provided on a single computer or distributed across multiple computers. Such a processor may be implemented as an integrated circuit with one or more processors within an integrated circuit component. However, the processor may be implemented using any suitable form of circuitry.

[0077] Also, the various methods or processes outlined herein may be coded as software executable on one or more processors using any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0078] Also, embodiments of the present disclosure may be embodied as methods, of which an example is provided. Acts performed as part of a method may be ordered in any suitable manner. Thus, embodiments may be constructed in which acts are performed in an order other than that illustrated, including simultaneously performing some acts shown as sequential acts in an exemplary embodiment. Therefore, it is the object of the appended claims to cover all such variations and modifications as are within the true spirit and scope of the present disclosure.

[0079] Although the present disclosure has been described with reference to certain preferred embodiments, it is to be understood that various other adaptations and modifications can be made within the spirit and scope of the disclosure. It is therefore the object of the appended claims to cover all such variations and modifications that come within the true spirit and scope of the disclosure.

Claims

1. 1. A computer-implemented method for training a neural network to classify a plurality of data samples collected from a sensor, the method using a processor storing instructions for implementing the method, the instructions, when executed, causing the processor to perform the method, the method comprising: collecting a plurality of data samples as input for training the neural network, the plurality of data samples including clean data samples and adversarial data samples; training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over a latent space representation; training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result; the method includes: a method step of training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples; and outputting the shared parameters of the first instance of the neural network and the second instance of the neural network; the shared parameters of the different instances of the neural network are trained based on a multi-objective loss function; the first instance of the neural network and the second instance of the neural network are jointly trained to minimize the multi-objective loss function of the difference between corresponding outputs of the first instance and the second instance; the corresponding outputs comprise differences between the probability distributions determined by the probabilistic encoders of the first and second instances of the neural network and the classification results determined by the classifiers of the first and second instances of the neural network.

2. 2. The method of claim 1 , wherein the joint training of the first instance of the neural network and the second instance of the neural network is performed, the joint training being performed using the latent space representations for the clean data samples and the adversarial data samples that are sampled multiple times.

3. 2. The method of claim 1 , further comprising parameterizing the multi-objective loss function based on mutual information of the distributions over the latent space representations determined by the probabilistic encoders of the first and second instances of the neural network and an entropy loss of the classification results produced by the first and second instances of the neural network.

4. 4. The method of claim 3, wherein the multi-objective loss function includes terms corresponding to maximizing mutual information between the probability distributions of encodings of the pair of the clean data sample and the adversarial data sample, minimizing mutual information between encodings of the other of the clean data sample or the adversarial data sample in the pair conditioned on one of the clean data sample or the adversarial data sample in the pair, and a clean cross-entropy loss required to classify the clean data sample and an adversarial cross-entropy loss required to classify the adversarial data sample.

5. collecting the plurality of data samples receiving the clean data samples over a communication channel, the communication channel comprising one or a combination of a wired channel and a wireless channel; and collecting the plurality of data samples further comprising: The method of claim 4 , comprising modifying each of the clean data samples to generate a corresponding adversarial data sample that forms the pair of the clean data sample and the adversarial data sample.

6. The modification may include: The clean data samples are subjected to one of the following attacks: projected gradient descent, fast gradient sign, limited-memory Broyden-Fletcher-Goldfarb-Shanno, Jacobian-based saliency map attack, or Carlini & Wagner attack. The method of claim 5 , wherein the method applies an adversarial data sample generation method comprising:

7. 1. An artificial intelligence system (AI system) for training a neural network to classify a plurality of data samples collected from a sensor, comprising: a processor; and a memory having instructions stored therein, wherein the processor executes the stored instructions to provide the AI ​​system with: collecting a plurality of data samples as input for training the neural network, the plurality of data samples including clean data samples and adversarial data samples; training the neural network including training a probabilistic encoder to encode the plurality of data samples into a probability distribution over a latent space representation; training the neural network including training a classifier to classify instances of the latent space representation to generate a classification result; and the processor executes the stored instructions to cause the AI ​​system to: training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples; outputting the shared parameters of the first instance of the neural network and the second instance of the neural network; the shared parameters of the different instances of the neural network are trained based on a multi-objective loss function; the first instance of the neural network and the second instance of the neural network are jointly trained to minimize the multi-objective loss function of the difference between corresponding outputs of the first instance and the second instance; the corresponding output comprises a difference between the probability distributions determined by the probabilistic encoders of the first and second instances of the neural network and the classification results determined by the classifiers of the first and second instances of the neural network.

8. 8. The AI ​​system of claim 7, wherein the joint training of the first instance of the neural network and the second instance of the neural network is performed, the joint training being performed using the latent space representations for the clean data samples and the adversarial data samples that are sampled multiple times.

9. 8. The AI ​​system of claim 7, wherein the AI ​​system is configured to parameterize the multi-objective loss function based on mutual information of the distributions over the latent space representations determined by the probabilistic encoders of the first and second instances of the neural network and an entropy loss of the classification results produced by the first and second instances of the neural network.

10. 10. The AI ​​system of claim 9, wherein the multi-objective loss function includes terms corresponding to maximizing mutual information between the probability distributions of encodings of the pair of the clean data sample and the adversarial data sample, minimizing mutual information between encodings of the other of the clean data sample or the adversarial data sample in the pair conditioned on one of the clean data sample or the adversarial data sample in the pair, and a clean cross-entropy loss required to classify the clean data sample and an adversarial cross-entropy loss required to classify the adversarial data sample.

11. 11. The AI ​​system of claim 10, wherein the AI ​​system is configured to collect the plurality of data samples by performing a first step and a second step, wherein the AI ​​system performs the first step of receiving the clean data samples via a communication channel, and the AI ​​system performs the second step of modifying each of the clean data samples using a modification module to generate a corresponding adversarial data sample that forms the pair of the clean data sample and the adversarial data sample.

12. The correction module may apply a projected gradient descent, fast gradient sign, limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm, a Jacobian-based saliency map attack, or a Carlini & Wagner attack to the cleaned data samples.

12. The AI ​​system of claim 11, configured to apply an adversarial data sample generation method comprising one of the following attacks:

13. 1. A non-transitory computer-readable medium storing computer-executable instructions for training a neural network to classify a plurality of data samples collected from a sensor, the computer-executable instructions, when executed by a computer, causing the computer to perform operations, the operations including: collecting a plurality of data samples as input for training the neural network, the plurality of data samples including clean data samples and adversarial data samples; training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over a latent space representation; training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result; and the operations training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples; and outputting the shared parameters of the first instance of the neural network and the second instance of the neural network; the shared parameters of the different instances of the neural network are trained based on a multi-objective loss function; the first instance of the neural network and the second instance of the neural network are jointly trained to minimize the multi-objective loss function of the difference between corresponding outputs of the first instance and the second instance; the corresponding output comprising a difference between the probability distributions determined by the probabilistic encoders of the first and second instances of the neural network and the classification results determined by the classifiers of the first and second instances of the neural network.

Images