Method and system for training neural networks to improve robustness against adversarial attacks

Abstract

An embodiment of the present disclosure discloses a method and system for training a neural network to improve robustness against adversarial attacks. The method includes collecting a plurality of data samples, including clean data samples and adversarial data samples. Training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over the latent space representation. In addition, training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result. In addition, the method includes training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples. Furthermore, the method includes outputting the shared parameters of the first instance of the neural network and the second instance of the neural network.

Description

[Technical field]

[0001] The present disclosure relates generally to adversarial data perturbation, and more particularly to methods and systems for training neural networks to improve robustness against adversarial behavior. [Background technology]

[0002] In recent advances, machine learning and deep neural networks are widely used for data classification. However, machine learning models are often vulnerable to attacks based on adversarial manipulation of data. One type of adversarial manipulation of data is known as an adversarial example. The adversarial example is a sample of data that is intentionally modified with small feature perturbations. These feature perturbations are intended to make a machine learning or deep neural network (ML / DNN) model output an inaccurate prediction. In particular, the feature perturbations are imperceptible noise to the data, which causes the ML classifier to misclassify the data. Such adversarial examples can be used to perform attacks against ML systems, which raises security concerns. Adversarial examples pose a potential security threat to ML applications such as robots that perceive the world through cameras and other sensors, video surveillance systems, and mobile applications for image or audio classification.

[0003] Adversarial sample attacks are broadly classified into two classes of threat models, such as white-box adversarial attacks and black-box attacks. In white-box adversarial attacks, the attacker accesses the parameters of the target model. For example, the attacker accesses the parameters of the target model, such as the architecture, weights, gradients, etc. White-box adversarial attacks require strong adversarial access to perform a successful attack. In addition, such white-box adversarial attacks have higher computational overhead, such as time and attack iterations. In contrast, in black-box adversarial attacks, the adversarial access of the parameters of the target model is limited. For example, the adversarial access only involves accessing example input data and output data pairs for the target model. Alternatively, in black-box adversarial attacks, no information of the target model is used. In such adversarial attacks, an alternative or source model is trained with training data to generate adversarial perturbations. The generated adversarial perturbations are added to the input data to attack the target black-box DNN. For example, an input image is input to the alternative model to generate adversarial perturbations. Adversarial perturbations are then added to the input image to attack the target black-box DNN. In some cases, model queries are used to obtain information from the target black-box DNN.

[0004] Traditional techniques for making machine learning models more robust, such as weight decay and dropout, generally do not provide practical defenses against adversarial examples. To date, only two methods have provided significant defenses: adversarial training and defensive distillation. Adversarial training is a brute force approach that generates many adversarial examples and explicitly trains the model to not be fooled by them. Defensive distillation is a strategy that trains the model to output probabilities of different classes, rather than hard decisions about which class to output. The probabilities are supplied by a previous model trained on the same task using hard class labels. This creates a model whose surface is smoothed in the directions that an adversary typically tries to exploit, making it difficult for the adversary to discover adversarial input tweaks that lead to inaccurate classifications.

[0005] However, adversarial examples are difficult to defend against because it is difficult to build a theoretical model of the process of crafting adversarial examples. Adversarial examples are solutions to optimization problems that are nonlinear and nonconvex for many ML models, including neural networks. Adversarial examples are also difficult to defend against because they require machine learning models to generate good outputs for every possible input. In most cases, machine learning models perform very well, but only for a small fraction of all the many possible inputs they may encounter.

[0006] In addition, current techniques for making machine learning models more robust are not adaptive because they may block some types of attacks but leave vulnerabilities unprotected against others. Therefore, designing defenses that can protect against powerful adaptive attackers is a nontrivial but so far unsolved technical problem.

[0007] Therefore, there is a need to overcome the above problems. More specifically, there is a need to develop a method and system for training a neural network to improve the network's robustness to adversarial attacks while maintaining natural accuracy. Summary of the Invention

[0008] It is an object of some embodiments to provide a system and method for training a robust neural network model with improved resistance to adversarial attacks. Additionally or alternatively, it is an object of some embodiments to provide a system and method for classifying input data using a trained neural network with improved robustness to adversarial attacks. Additionally or alternatively, it is an object of some embodiments to provide a system and method for probabilistically classifying input data to improve classification accuracy under or against adversarial attacks.

[0009] To that end, some embodiments disclose a neural network that includes a probabilistic encoder configured to encode input data of a plurality of data samples into a distribution over the latent space representation, and a classifier configured to classify the encoding of the input data in the latent space representation. The probabilistic encoder is in contrast to a deterministic encoder. While a deterministic encoder encodes the input data into the latent space representation, a probabilistic encoder encodes the input data into a distribution over the latent space representation. For example, to encode the input data in the distribution of the latent space representation, the probabilistic encoder can output parameters of the distribution.

[0010] First, the classifier does not classify a distribution over the latent space representation, but classifies an instance of the latent space representation (either the first instance or the second instance) or a sample of the distribution encoded by the probabilistic encoder. This allows sampling the output of the probabilistic encoder multiple times and combining the results of the classification to obtain a more accurate classification result.

[0011] Second, probabilistic encoding allows us to improve the training of neural networks by imposing additional requirements not only on the classification results but also on the distribution over the latent space representation itself. Both of these advantages, alone or in combination, improve the robustness of the trained neural networks to adversarial attacks.

[0012] For example, some embodiments are based on the recognition that the performance of machine learning methods depends on the choice of data representation, and the goal of representation learning is to transform raw input x into a lower-dimensional representation z that preserves relevant information for tasks such as classification or regression. Adversarial samples are solutions to optimization problems that are nonlinear and nonconvex for many ML models. Some embodiments are based on the recognition that it is difficult to provide theoretical tools to describe solutions to these complex optimization problems. The information bottleneck (IB) principle provides an information-theoretic method for representation learning, where the representation should contain only the most relevant information from the input for downstream tasks. Representations learned by the IB principle are less affected by disturbing variations and may be more robust to adversarial perturbations. In addition, the multi-view information bottleneck can extend the IB principle to a multi-view unsupervised setting by maximizing the information shared between different views while minimizing view-specific information.

[0013] Some embodiments are based on the recognition that it is possible to extend the multi-view information bottleneck method to a supervised setting with adversarial training. For example, some embodiments can consider the adversarial sample as another view of the corresponding clean sample. As a result, the embodiments attempt to learn a representation that includes information shared between the clean sample and the corresponding adversarial sample, while eliminating information that is not shared between them. As mentioned above, having a probabilistic encoder that encodes input data into a distribution over the latent space representation, rather than into an instance of the latent space representation, allows different embodiments to explore the theoretical guarantees provided by the principle of multi-view information bottleneck to improve the robustness and / or performance of the trained neural network.

[0014] To take advantage of these principles, some embodiments train shared parameters of different instances of a neural network by optimizing a multi-objective loss function of the outputs of the different instances using pairs of clean and adversarial data samples. Since the different instances are instances of the same neural network including a probabilistic encoder and a classifier, the outputs of the different instances (the first instance and the second instance) include parameters of the probability distribution of the latent space representation and the results of classification. By comparing and optimizing the difference between these outputs, resistance to adversarial attacks is improved.

[0015] Accordingly, one embodiment discloses a computer-implemented method for training a neural network. The method includes collecting a plurality of data samples, including clean data samples and adversarial data samples. Training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over the latent space representation. Additionally, training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result. Additionally, the method includes training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples. Furthermore, the method includes outputting the shared parameters of the first instance of the neural network and the second instance of the neural network.

[0016] Accordingly, another embodiment discloses an AI system for training a neural network. The AI ​​system includes a processor and a memory having instructions stored therein. The processor is configured to execute the stored instructions to cause the AI ​​system to collect a plurality of data samples as inputs for training the neural network. The plurality of data samples includes clean data samples and adversarial data samples. Training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over the latent space representation. Furthermore, training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result. Furthermore, the processor causes the AI ​​system to train shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples. Furthermore, the processor causes the AI ​​system to output the shared parameters of the first instance of the neural network and the second instance of the neural network. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a schematic block diagram of a system for training a neural network to improve robustness against adversarial attacks, in accordance with some embodiments of the present disclosure. [Figure 2A] FIG. 1 is a schematic diagram of an AI system for training a neural network to improve robustness against adversarial attacks, in accordance with some embodiments of the present disclosure. [Figure 2B] FIG. 13 is a block diagram of a representation of z in terms of x and x′ for mutual information sufficiency and minimality, according to various embodiments of the present disclosure. [Diagram 3] 1 is a diagram illustrating a procedure for training a neural network to improve robustness against adversarial behavior, according to some embodiments of the present disclosure. [Figure 4] 1 is a representation illustrating a multi-objective loss function according to some embodiments of the present disclosure. [Diagram 5] FIG. 1 is a block diagram of an AI system for generating adversarial data samples for training a neural network, according to some embodiments of the invention. [Figure 6] FIG. 1 is a block diagram of a computer-based system for improving robustness against adversarial attacks in accordance with some embodiments of the present invention. [Figure 7] 13 is a use case of using an AI system according to some other embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] In the following description, for purposes of explanation, numerous specific details are set forth in order to facilitate a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, devices and methods are shown only in block diagram form in order to avoid obscuring the present disclosure.

[0019] As used herein and in the claims, the words "for example," "for example," "e.g.," "etc.," and the verbs "comprise," "have," "include," and other verb forms thereof, when used with a list of one or more components or other items, should each be construed as open-ended, meaning that the list should not be considered to exclude other additional components or items. The phrase "based on" means based at least in part on. Furthermore, it should be understood that the phrases and terms used herein are for purposes of description and should not be considered limiting. Any headings used within this description are for convenience only and have no legal or limiting effect.

[0020] 1 illustrates a schematic block diagram of a system 100 for training a neural network, such as neural network 110, to improve robustness against adversarial attacks, in accordance with some embodiments of the present disclosure. System 100 includes a plurality of data samples 102, neural network 110, classifier 104A, classifier 104B, probability encoder 106A, and probability encoder 106B.

[0021] The system 100 collects a plurality of data samples 102 as inputs for training the neural network 110. In some embodiments, the plurality of data samples 102 includes clean data samples x and adversarial data samples x′, where the clean data samples x are the ground truth data used to train the neural network 110. As used herein, the adversarial data samples x′ refer to incorrect data samples (e.g., data samples with some perturbation) used for training the neural network 110. In addition, training the neural network 110 involves fitting the plurality of data samples (i.e., the clean data samples x) to probability distributions (e.g., clean cross entropy associated with them) over the latent space representation z.

number

[0022] More specifically, the system 100 is configured to first train the neural network 110 based on the clean data samples x. The clean data samples x are provided as input to the probabilistic encoder 106A. The probabilistic encoder 106A is further configured to generate a probabilistic representation (i.e., intermediate representation) z based on the execution of the probabilistic encoder 106A. The probabilistic representation corresponds to the latent space representation z. The probabilistic representation z is passed through the remaining layer 108A. The remaining layer 108A corresponds to a hidden layer of the neural network 110. The system 100 further comprises a clean cross-entropy

number

[0023] Similarly, training the neural network 110 includes training the probability encoder 106B to encode a plurality of data samples (i.e., the adversarial data samples x′) into a probability distribution over the latent space representation z′. Training the neural network 110 includes training the classifier 104B to classify instances (e.g., the second instances 114) of the latent space representation z′ to generate a classification result.

[0024] More specifically, the system 100 is configured to first train a neural network 110 based on an adversarial data sample x'. The adversarial data sample x' is provided as an input to a probabilistic encoder 106B. The probabilistic encoder 106B is further configured to generate a probabilistic representation (i.e., intermediate representation) z' based on an execution of the probabilistic encoder 106B. The probabilistic representation z' is passed through a hidden layer 108B. The system 100 further comprises an adversarial cross-entropy (ADP) algorithm.

number

[0025] The system 100 is further configured to train shared parameters of a first instance 112 of the neural network 110 using the clean data sample x. Similarly, the system 100 is configured to train shared parameters of a second instance 114 of the neural network 110 using the adversarial data sample x′.

[0026] Furthermore, the system 100 is configured to generate an output 116 based on shared parameters of the first instance 112 of the neural network 110 and the second instance 114 of the neural network 110. In this manner, the neural network 110 is trained based on a probabilistic representation z corresponding to the clean data sample x and a probabilistic representation z' corresponding to the adversarial data sample x'.

[0027] In one embodiment, system 100 is configured to train neural network 110 such that probabilistic representations z and z' contain shared information (i.e., mutual information) between x and x'. To achieve this, system 100 is configured to minimize the Kullback-Leibler divergence (KL divergence) between the latent space distributions generated by probabilistic encoder 106A and the latent space distributions generated by probabilistic encoder 106B, and maximize the shared information (i.e., mutual information) between z and z'.

[0028] System 100 is an artificial intelligence based system (hereinafter AI system) that is further described in Figures 2A and 2B.

[0029] 2A illustrates a schematic block diagram 200A of an AI system 202 for training a neural network, such as neural network 110, to improve robustness against adversarial attacks, in accordance with some embodiments of the present disclosure. The AI ​​system 202 includes a processor 204 and a memory 206. The memory 206 stores instructions to be executed by the processor 204. The memory 206 also includes the neural network 110. The processor 204 is configured to execute the stored instructions to cause the AI ​​system 202 to collect a plurality of data samples 102 as inputs for training the neural network 110.

[0030] In some embodiments, examples of processor 204 include, but are not limited to, an application specific integrated circuit (ASIC) processor, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a field programmable gate array (FPGA), and the like. In some embodiments, memory 206 includes logic, circuitry, and / or interfaces suitable for storing a set of computer-readable instructions to perform operations. In addition, examples of memory 206 may include random access memory (RAM), read-only memory (ROM), a removable storage drive, a hard disk drive (HDD), and the like. It will be apparent to one skilled in the art that the scope of the present disclosure is not limited to implementing memory 206 in AI system 202 as described herein.

[0031] 2A, a plurality of data samples 102 are provided as input to an AI system 202. The AI ​​system 202 invokes a processor 204 to execute instructions stored in a memory 206 to begin training the neural network 110.

[0032] In some embodiments, the plurality of data samples 102 includes a clean data sample x and an adversarial data sample x′. The AI ​​system 202 is configured to train the neural network 110 to improve the adversarial robustness of the neural network 110. In some embodiments, the neural network 110 includes a deep neural network (DNN), or the like. In some embodiments, the AI ​​system 202 is configured to perform training of the neural network 110 in a supervised setting.

[0033] In some embodiments, the AI ​​system 202 is configured to train the neural network 110 based on a multi-objective loss function. The AI ​​system 202 is configured to train the neural network 110 with the objectives of (1) maximizing the shared information 116 between matched pairs of probabilistic representations, and (2) minimizing the shared information 116 between each probabilistic representation and the corresponding view of each probabilistic representation conditioned on the other views along with (3) a clean cross-entropy loss, and (4) an adversarial cross-entropy loss. For example, item (1) corresponds to maximizing the mutual information objective function value, and item (2) corresponds to minimizing the KL divergence objective function value.

[0034] The AI ​​system 202 is configured to improve robustness against adversarial behavior based on maximizing shared information 116 between probabilistic representations z and z', as captured by an objective function that maximizes mutual information between probabilistic representations z and z' corresponding to matched pairs of clean data samples x and adversarial data samples x'. In addition, the objectives of training the neural network 110 include symmetrized KL divergence between the posterior feature distributions of the clean data samples x and the adversarial data samples x', and shared information 116 between latent representations of the clean data samples x and the adversarial data samples x'.

[0035]

number

[0036]

number

[0037] The AI ​​system 202 is configured to learn latent space representations z and z' (corresponding to x and x', respectively), that contain only useful information shared by both x and x'. Mathematically, the generation of these representations is defined by the conditional distributions p(z|x) and p(z'|x'), satisfying the Markov chain z→x→x'→z'.

[0038] The AI ​​system 202 is further configured to improve generalization by learning a representation z or z' that captures only the shared information between x and x'. If the representation only preserves the shared information 116 (i.e., mutual information) from both x and x', it means that it only contains task-relevant information while discarding view-specific details (i.e., information from x' that leads to erroneous perception), thus improving the adversarial robustness of the neural network 110.

[0039] FIG. 2B illustrates a block diagram 200B of a representation of z in terms of x and x′ for mutual information sufficiency and minimality, according to various embodiments of the present disclosure.

[0040] Consider subdividing I(z;x) into three components using the chain rule of mutual information. Since the Markov chain z→x→x' holds,

number

[0041] Here, I(x;z|x') represents information in z that is unique to x and not shared by x', called view-specific information. The second term I(x;x') denotes the shared information 116 between x and x'. The last term I(x;x'|z) is the shared information 116 that is missing in z. The main objective here is for representation z to contain only the shared information 116 of x and x', so I(x;z)=I(x;x'). Thus, the objective here is to minimize I(x;z|x') and I(x;x'|z). Representation z is defined as sufficient and minimal for any downstream task, since it contains all task-relevant information (sufficiency) without any extraneous information (minimality).

[0042] Diagram 200B includes a representation of sufficient, but not minimal, mutual information (a):

number

number

number

number

[0043] 3 shows a diagrammatic representation illustrating a procedure 300 for training the neural network 110 according to some embodiments of the present disclosure. The procedure 300 is performed by the AI ​​system 200.

[0044] In step 302, a plurality of data samples 102 are collected. The plurality of data samples 102 includes clean data samples x and adversarial data samples x′, where the clean data samples x are ground truth data used to train the neural network 110. As used herein, the adversarial data samples x′ refer to inaccurate data samples (e.g., data samples with some perturbation) used to train the neural network 110.

[0045] In step 304, training of the neural network 110 is performed. Training of the neural network 110 involves encoding the plurality of data samples 102 into a probability distribution over the latent space representations z and z′. The plurality of data samples 102 are encoded using probability encoders 106A and 106B. The probability encoders 106A and 106B encode the plurality of data samples (i.e., clean data samples x and adversarial data samples x′) into a probability distribution over the latent space representations z and z′, respectively (e.g.,

number

[0046] In step 306, the shared parameters of the first instance 112 of the neural network 110 and the second instance 114 of the neural network 110 are trained. The shared parameters of the first instance 112 of the neural network 110 are trained using the clean data sample x. The shared parameters of the second instance 114 of the neural network are trained using the adversarial data sample x′.

[0047] The neural network 110 is trained based on a multi-objective loss function. The first instance 112 of the neural network 110 and the second instance 114 of the neural network 110 are jointly trained to minimize the multi-objective loss function of the difference between the corresponding outputs of the first instance 112 and the second instance 114 of the neural network. The corresponding outputs include the difference between the probability distributions determined by the probability encoders 106A and 106B of the first instance 112 and the second instance 114 of the neural network 110 and the classification results determined by the classifiers 104A and 104B of the first instance 112 and the second instance 114 of the neural network 110. The joint training of the first instance 112 of the neural network 110 and the second instance 114 of the neural network 114 is performed using latent representations z and z' of the clean data sample x and the adversarial sample x', respectively, sampled multiple times.

[0048] The AI ​​system 202 is configured to train the neural network 110 with the objectives of (1) maximizing the shared information 116 between matched pairs of probabilistic representations, and (2) minimizing the shared information 116 between each probabilistic representation and the corresponding view of each probabilistic representation conditioned on the other views along with (3) a clean cross-entropy loss, and (4) an adversarial cross-entropy loss.

[0049] The AI ​​system 202 is configured to improve robustness against adversarial attacks based on learning the shared information 116 or output between the clean data sample x and the adversarial data sample x'. Additionally, the objectives of training the neural network 110 include symmetrized KL divergence between the posterior feature distributions of the clean data sample x and the adversarial data sample x', and the shared information 116 between the latent representations of the clean data sample x and the adversarial data sample x'.

[0050] In step 308, the shared parameters of the first instance 112 and the second instance 114 of the neural network 110 are output.

[0051] 4 illustrates a representation 400 illustrating a multi-objective loss function 402 according to some embodiments of the present disclosure. In some embodiments, the neural network 110 of FIG. 1 calculates a multi-objective loss function 402 by using the mutual information of the distributions over the latent space representations z and z′ determined by the probability encoders 106A and 106B of the first and second instances 112 and 114 of the neural network 110, respectively, and the entropy loss of the classification results generated by the first and second instances 112 and 114 of the neural network 110.

number

[0052] In addition, the multi-objective loss function 402 includes terms corresponding to maximizing mutual information between the probability distributions of the paired encodings of the clean data sample x and the adversarial data sample x′, minimizing mutual information between the encodings of the other of the clean data sample x or the adversarial data sample x′ in the pair conditioned on one of the clean data sample x or the adversarial data sample x′ in the pair, and the clean cross-entropy loss required to classify the clean data sample x and the adversarial cross-entropy loss required to classify the adversarial data sample x′.

[0053] 2A-B, the AI ​​system 202 is configured to learn a representation that includes only shared information of x and x' by minimizing view-specific information (I(x;z|x')) and shared information not in z (I(x;x'|z)). In particular, minimizing I(x;x'|z) is equivalent to maximizing I(z;x'), because I(z;x')=I(x;x')-I(x;x'|z), and I(x;x') is constant given x and x'.

number

[0054]

number

[0055] where λ1 and λ2 represent the Lagrange multipliers for constrained optimization. The objective function contains two mutual information terms that are difficult to compute directly. To solve this problem, some alternative bounds on these two mutual information terms are derived.

[0056] Upper bound on I(x;z|x'): First, an upper bound on the view-specific information in the latent representation z is derived from the input. For example, I(x;z|x') can be computed as follows:

number

[0057] Here, the conditional distributions p(z|x) and p(z'|x') may be parameterized by the encoder network. Moreover, this bound is strict whenever the representation z is the same as z'. Symmetrically, I(x';z'|x) is given by

number

[0058] Lower Bound on I(z;x'): Furthermore, a lower bound on the mutual information between the clean representation and the corresponding adversarial samples is derived. I(z;x') can be computed as follows:

number

[0059] Here, z', as a representation of x', is part of the Markov chain z → x → x' → z', so I(z;z'|x') = 0. Note that this bound is also straightforward from this Markov chain and data processing inequality, but the above derivation shows that this bound is tight when z' is a sufficient statistic of z. Symmetrically, we can derive a similar bound for I(z';x) ≥ I(z;z'). Conceptually, this lower bound captures our goal of preserving the information shared between representations despite adversarial perturbations.

[0060]

number

[0061]

number

[0062]

number

[0063] This symmetrized KL information can be directly calculated between the two Gaussian posterior distributions. Alternatively, I(z;z') requires the use of a mutual information estimator. This disclosure utilizes the Hilbert-Schmidt Independence Criterion (HSIC) to measure the independence between z and z' and uses this value to replace the mutual information term. Note that HSIC is used as a proxy for mutual information because the dependence between two mini-batch samples in a reproducing kernel Hilbert space (RKHS) can be measured directly without requiring density estimation or using additional networks for mutual information estimation.

[0064]

number

[0065] Here, α ε [0, 1] balances the tradeoff between the cross-entropy loss on clean and adversarial samples, and β and λ adjust the importance of the symmetrized KL information and mutual information terms.

[0066] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automated implementation may be performed or at least assisted through the use of a machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine-readable medium. The necessary tasks may be executed by a processor.

[0067] FIG. 5 shows a block diagram 500 of an AI system 202 for generating an adversarial data sample x′ to train a neural network 110 according to some embodiments of the present invention. The block diagram 500 includes a communication channel 502 and a correction module 504. The AI ​​system 202 is configured to collect a plurality of data samples by performing a first step and a second step. The AI ​​system 202 performs a first step of receiving a clean data sample x through the communication channel 502. The communication channel 502 comprises one or a combination of a wired channel and a wireless channel. The AI ​​system performs a second step of correcting each of the clean data samples x using the correction module 504 to generate a corresponding adversarial data sample that forms a pair of the clean data sample x and the adversarial data sample x′. The correction module 504 applies the adversarial sample generation method to the clean data sample x. The adversarial sample generation methods include one of the following: projected gradient descent, fast gradient sign, memory-limited Broyden-Fletcher-Goldfarb-Shanno, Jacobian-based saliency map attack, or Carlini & Wagner attack.

[0068] FIG. 6 illustrates a block diagram of a computer-based system 600 for improving robustness against adversarial attacks, according to some embodiments of the present disclosure. The system 600 includes at least one processor 604 and a memory 606 having instructions stored thereon, the instructions including executable instructions for execution by the at least one processor 604 during control of the system 600. The memory 606 is embodied as a storage medium, such as a RAM (random access memory), a ROM (read only memory), a hard disk, or any combination thereof. For example, the memory 606 stores instructions executable by the at least one processor 604. In an exemplary embodiment, the memory 606 is configured to store a neural network 608. The neural network 608 corresponds to the neural network 110 of FIG. 1.

[0069] The at least one processor 604 may be embodied as a single core processor, a multi-core processor, a computing cluster, or any number of other configurations. The at least one processor 604 is operatively connected to the sensor 602 and the receiver 610 via a bus 614. In one embodiment, the at least one processor 604 is configured to collect a plurality of data samples. In some exemplary embodiments, the plurality of data samples are collected from the receiver 610. The receiver 610 is connected to an input device 624 via a network 620. Each of the plurality of data samples is stored in the storage 612. In some other exemplary embodiments, the plurality of data samples are collected from the sensor 602. The sensor 602 receives a measured data signal 622 from a source (not shown). In some embodiments, the sensor 602 is configured to sense the data signal 622 based on a source of the sensed data signal 622.

[0070] Additionally or alternatively, the system 600 is integrated with a network interface controller (NIC) 618 to receive a plurality of data samples 102 (of FIG. 1) using a network 620. The plurality of data samples includes clean data samples and adversarial data samples.

[0071] The at least one processor 604 is also configured to train the neural network 608 to improve robustness against adversarial attacks. Training the neural network 608 includes encoding the plurality of data samples 102 into a probability distribution over the latent space representation. The plurality of data samples 102 are encoded using a probabilistic encoder.

[0072] The trained neural network 608 generates an output of the shared information that is transmitted via a transmitter 616. Additionally or alternatively, the transmitter 616 is coupled to an output device 626 to output the shared information over a wireless or wired communication channel, such as a network 620. The output device 626 includes a computer, laptop, smart device, or any computing device used to prevent adversarial attacks in an application installed on the output device 626.

[0073] Also, particular embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process may be terminated when its operations are completed, but may have additional steps not discussed or included in the diagram. Moreover, not all operations in any process specifically described may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the end of the function may correspond to a return to the calling function or the main function of the function.

[0074] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automatic implementation may be performed or at least assisted through the use of a machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine-readable medium. The necessary tasks may be executed by a processor.

[0075] FIG. 7 illustrates a use case 700 using the AI ​​system 202 according to some other embodiments of the disclosure. The use case 700 corresponds to a vehicle-assisted navigation system (not shown) of a vehicle 702A and a vehicle 702B. The vehicle-assisted navigation system is connected to the AI ​​system 202. The vehicle-assisted navigation system is connected to a camera of the vehicle 702A, such as a front camera that captures a road scene or view. In one illustrative example scenario, the camera captures a road sign 704 displaying a "No Parking" sign. The captured road sign 704 is transmitted to the AI ​​system 202. The AI ​​system 202 processes the captured road sign 704 using the trained neural network 110. The captured road sign 704 is processed using clean data samples and adversarial data samples to generate a robust model for identifying the "No Parking" sign in the road sign 704. The robust model is used by the vehicle-assisted navigation system to accurately identify road signs 704 and prevent vehicles 702A and 702B from parking in no-parking zones.

[0076] The above-described embodiments of the present disclosure may be implemented in any of many ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on any suitable processor or collection of processors, whether provided on a single computer or distributed across multiple computers. Such a processor may be implemented as an integrated circuit with one or more processors within an integrated circuit component. However, the processor may be implemented using any suitable form of circuitry.

[0077] Also, the various methods or processes outlined herein may be coded as software executable on one or more processors using any one of a variety of operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0078] Also, the embodiments of the present disclosure may be embodied as a method, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable manner. Thus, embodiments may be constructed in which acts are performed in an order different from that illustrated, which may include performing simultaneously some acts shown as sequential acts in the exemplary embodiment. It is, therefore, the object of the claims to cover all such variations and modifications as are within the true spirit and scope of the present disclosure.

[0079] Although the disclosure has been described with reference to certain preferred embodiments, it will be understood that various other adaptations and modifications can be made within the spirit and scope of the disclosure. It is therefore the object of the appended claims to cover all such variations and modifications that come within the true spirit and scope of the disclosure.

Claims

1. 1. A computer-implemented method for training a neural network, the method using a processor storing instructions for implementing the method, the instructions, when executed, causing the processor to perform the method, the method comprising: collecting a plurality of data samples as input for training the neural network, the plurality of data samples including clean data samples and adversarial data samples, training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over a latent space representation, training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result, the method further comprising: training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples; and outputting the shared parameters of the first instance of the neural network and the second instance of the neural network.

2. 2. The method of claim 1 , wherein the first instance of the neural network and the second instance of the neural network are jointly trained to minimize a multi-objective loss function of differences between corresponding outputs of the first and second instances, the corresponding outputs comprising differences between the probability distributions determined by the probabilistic encoders of the first and second instances of the neural network and the classification results determined by the classifiers of the first and second instances of the neural network.

3. 3. The method of claim 2 , wherein the joint training of the first instance of the neural network and the second instance of the neural network is performed, the joint training being performed using the latent representations for the clean data samples and the adversarial samples that are sampled multiple times.

4. 2. The method of claim 1 , further comprising parameterizing a multi-objective loss function based on mutual information of the distributions over the latent space representations determined by the probabilistic encoders of the first and second instances of the neural network and an entropy loss of the classification results produced by the first and second instances of the neural network.

5. 5. The method of claim 4, wherein the multi-objective loss function includes terms corresponding to maximizing mutual information between the probability distributions of paired encodings of the clean data sample and the adversarial data sample, minimizing mutual information between encodings of the other of the clean data sample or the adversarial data sample in the pair conditioned on one of the clean data sample or the adversarial data sample in the pair, and a clean cross-entropy loss required to classify the clean data sample and an adversarial cross-entropy loss required to classify the adversarial data sample.

6. Collecting the plurality of data samples comprises: receiving the clean data samples over a communication channel, the communication channel comprising one or a combination of a wired channel and a wireless channel; and collecting the plurality of data samples further comprising: The method of claim 1 , comprising modifying each of the clean data samples to generate a corresponding adversarial data sample that forms the pair of the clean data sample and the adversarial data sample.

7. The modifying step comprises:

7. The method of claim 6, further comprising applying an adversarial sample generation method to the clean data samples, the adversarial sample generation method including one of the following: projected gradient descent, fast gradient sign, limited memory Broyden-Fletcher-Goldfarb-Shanno, Jacobian-based saliency map attack, or Carlini & Wagner attack.

8. 1. An artificial intelligence (AI) system for training a neural network to classify a plurality of data samples, comprising: A processor; and a memory having instructions stored therein, the processor executing the stored instructions to provide the AI ​​system with: and causing a plurality of data samples to be collected as input for training the neural network, the plurality of data samples including clean data samples and adversarial data samples, training the neural network including training a probabilistic encoder to encode the plurality of data samples into a probability distribution over a latent space representation, training the neural network including training a classifier to classify instances of the latent space representation to generate a classification result, the processor further executing the stored instructions to cause the AI ​​system to: training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples; The AI ​​system outputs the shared parameters of the first instance of the neural network and the second instance of the neural network.

9. 9. The AI ​​system of claim 8, wherein the first instance of the neural network and the second instance of the neural network are jointly trained to minimize a multi-objective loss function of differences between corresponding outputs of the first and second instances, the corresponding outputs comprising differences between the probability distributions determined by the probabilistic encoders of the first and second instances of the neural network and the classification results determined by the classifiers of the first and second instances of the neural network.

10. 10. The AI ​​system of claim 9, wherein the joint training of the first instance of the neural network and the second instance of the neural network is performed, the joint training being performed using the latent representations for the clean data samples and the adversarial samples that are sampled multiple times.

11. 9. The AI ​​system of claim 8, wherein the AI ​​system is configured to parameterize a multi-objective loss function based on mutual information of the distributions over the latent space representations determined by the probabilistic encoders of the first and second instances of the neural network and an entropy loss of the classification results produced by the first and second instances of the neural network.

12. 12. The AI ​​system of claim 11, wherein the multi-objective loss function includes terms corresponding to maximizing mutual information between the probability distributions of paired encodings of the clean data sample and the adversarial data sample, minimizing mutual information between encodings of the other of the clean data sample or the adversarial data sample in the pair conditioned on one of the clean data sample or the adversarial data sample in the pair, and a clean cross-entropy loss required to classify the clean data sample and an adversarial cross-entropy loss required to classify the adversarial data sample.

13. 9. The AI ​​system of claim 8, wherein the AI ​​system is configured to collect the plurality of data samples by performing a first step and a second step, wherein the AI ​​system performs the first step of receiving the clean data samples via a communication channel, and the AI ​​system performs the second step of modifying each of the clean data samples using a modification module to generate a corresponding adversarial data sample that forms the pair of the clean data sample and the adversarial data sample.

14. 14. The AI ​​system of claim 13, wherein the correction module is configured to apply an adversarial sample generation method to the clean data samples, the adversarial sample generation method including one of a projected gradient descent method, a fast gradient sign method, a limited-memory Broyden-Fletcher-Goldfarb-Shanno method, a Jacobian-based saliency map attack, or a Carlini & Wagner attack.

15. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a computer, cause the computer to perform operations, the operations including: collecting a plurality of data samples as input for training the neural network, the plurality of data samples including clean data samples and adversarial data samples, training the neural network includes training a probabilistic encoder to encode the plurality of data samples into a probability distribution over a latent space representation, training the neural network includes training a classifier to classify instances of the latent space representation to generate a classification result, the operations further comprising: training shared parameters between a first instance of the neural network using the clean data samples and a second instance of the neural network using the adversarial data samples; and outputting the shared parameters of the first instance of the neural network and the second instance of the neural network.

16. A computer-implemented method for training a neural network, the method using a processor in combination with stored instructions implementing the method, the instructions, when executed by the processor, perform steps of the method, the steps including: collecting pairs of clean and adversarial data samples to train the neural network, the neural network including a probabilistic encoder trained to encode input data samples into a probability distribution over a latent space and a classifier trained to classify instances in the latent space to generate a classification result; and jointly training parameters of a first instance of the neural network using the clean data samples and parameters of a second instance of the neural network using the adversarial data samples, wherein the first instance of the neural network and the second instance of the neural network are jointly trained to minimize a multi-objective loss function of differences between corresponding outputs of the first and second instances of the neural network determined for the pairs of the clean and adversarial data samples, the corresponding outputs comprising differences between the probability distributions determined by the probabilistic encoders of the first and second instances of the neural network and the classification results determined by the classifiers of the first and second instances of the neural network, the steps further comprising: outputting one or a combination of the parameters of the first instance of the neural network and the parameters of the second instance of the neural network.

Images