Systems and methods for defending against adversarial attacks using feature-diversification-based adversarial training

Abstract

Described herein are embodiments of feature-dispersion-based adversarial training methods for improving model robustness against adversarial attacks. Conventional adversarial training methods utilize targeted or untargeted supervision schemes to generate attacks for training, which often suffer from problems such as label leakage mentioned in recent work. Embodiments of the disclosed method generate adversarial images for training by feature dispersion in latent space, which is inherently unsupervised and avoids label leakage. More importantly, the proposed method generates perturbed images in a collaborative manner by considering the relationships between samples. Extensive experiments on different datasets demonstrate the effectiveness of the proposed embodiments compared to prior art methods.

Description

Technical Field

[0001] The present disclosure generally relates to a system and method for training an image model. More specifically, the present disclosure relates to a system and method for training an image model against adversarial attacks to improve the model's robustness. Background Art

[0002] Despite breakthroughs in many areas such as image classification using deep neural networks, these models can be easily fooled by so-called adversarial examples. In image classification, an adversarial example of a natural image is a modified version that is visually indistinguishable from the original but causes the classifier to produce a different label prediction. Adversarial examples have shown ubiquitous performance beyond classification, ranging from object detection to speech recognition.

[0003] There has been encouraging progress in improving model robustness against adversarial examples. Among them, adversarial training is one of the most popular techniques, which uses adversarially perturbed images instead of original images for model training. However, there are some challenges to be addressed. First, some adverse effects such as label leakage remain issues that hinder adversarial training. Currently available remedies increase the number of iterations for generating attacks, or use classes other than the true value to generate attacks. Increasing attack iterations will increase training time proportionally, while label leakage cannot be completely eliminated using non-true target methods. Second, previous methods for both standard training and adversarial training treat each training sample individually and in isolation with respect to other samples. Manipulating each sample individually ignores the relationship between samples and fails to fully exploit the potential of attack and defense, thereby limiting performance.

[0004] Therefore, there is a need for systems and methods for addressing the above-mentioned problems so as to train image models against adversarial attacks and improve the robustness of the models. Summary of the invention

[0005] Embodiments of the present disclosure provide a computer-implemented method for generating adversarial training data for training a parameterized model, a computer-implemented method for adversarially training an image model, and a non-transitory computer-readable medium.

[0006] In one aspect of the present disclosure, a computer-implemented method for generating adversarial training data for training a parameterized model includes: given a set of clean training data, generating an initial set of perturbed training data; using the parameterized model, extracting features from the set of clean training data and the initial set of perturbed training data; calculating a transfer cost matrix between features extracted from the set of clean training data and features extracted from the initial set of perturbed training data; determining a transfer matrix based on the transfer cost matrix, the transfer cost matrix representing the matching between features in the set of clean training data and features in the initial set of perturbed training data; determining a feature matching distance between the set of clean training data and the initial set of perturbed training data by calculating the product between the transfer cost matrix and the transfer matrix; and obtaining a set of feature dispersion-based training data based at least on the determined feature matching distance.

[0007] In another aspect of the present disclosure, a computer-implemented method for adversarially training an image model includes: receiving a batch of clean images and corresponding true value labels, and a perturbation budget; generating an empirical distribution from the batch of clean images and generating a perturbed empirical distribution from a batch of perturbed perturbed images to achieve initialization, wherein the initial perturbation of the perturbed images is achieved by the initial perturbation of the batch of clean images; in one or more iterative steps, updating the perturbed empirical distribution from the initially generated perturbed empirical distribution by maximizing the feature matching distance between the empirical distribution and the perturbed empirical distribution, and updating the perturbed images using the updated perturbated empirical distribution; and implementing one or more adversarial training steps using at least the updated perturbed images and the true value labels to learn parameters of the image model.

[0008] In another aspect of the present disclosure, a non-transitory computer-readable medium is disclosed, comprising one or more instruction sequences, which, when executed by one or more processors, cause execution of steps for generating adversarial training data for training a parameterized model, the steps comprising: generating a set of initial perturbation training data from a set of clean training data; extracting features from the set of clean training data and the set of initial perturbation training data; calculating a transmission cost matrix between features extracted from the set of clean training data and features extracted from the set of initial perturbation training data; determining a transmission matrix based on the transmission cost matrix, the transmission cost matrix representing the matching between features in the set of clean training data and features in the set of initial perturbation training data; determining a feature matching distance by calculating the product between the transmission cost matrix and the transmission matrix as the feature matching distance between the set of clean training data and the set of initial perturbation training data; and obtaining a set of feature dispersion-based training data based at least on the determined feature matching distance. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Reference will be made to embodiments of the present invention, examples of which may be shown in the accompanying drawings. These drawings are illustrative and not restrictive. Although the present invention is generally described in the context of these embodiments, it should be understood that this is not meant to limit the scope of the present invention to these specific embodiments. The items in the drawings are not drawn to scale.

[0010] Figure 1 A feature dispersion-based adversarial training pipeline according to an embodiment of the present disclosure is shown.

[0011] Figure 2 A process for feature dispersion-based adversarial training according to an embodiment of the present disclosure is shown.

[0012] Figure 3A An example of clean data with a decision boundary according to an embodiment of the present disclosure is graphically illustrated.

[0013] Figure 3B An example of perturbed data generated using a supervised adversarial generation method that moves toward a decision boundary according to an embodiment of the present disclosure is graphically illustrated.

[0014] Figure 3C An example of perturbed data generated using the proposed feature dispersion method according to an embodiment of the present disclosure is graphically illustrated.

[0015] Figure 4 A method for feature dispersion-based adversarial training according to an embodiment of the present disclosure is shown.

[0016] Figure 5AThe model performance under the projected gradient descent (PGD) attack with different attack budgets according to an embodiment of the present disclosure is graphically illustrated.

[0017] Figure 5B The model performance under PGD attack with different attack iterations according to an embodiment of the present disclosure is graphically illustrated, where the attack iterations range from 0 to 5.

[0018] Figure 5C The model performance under PGD attack with different attack iterations according to an embodiment of the present disclosure is graphically illustrated, where the attack iterations range from 0 to 100.

[0019] Figure 6 A simplified block diagram of a computing device / information processing system is shown according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] In the following description, for the purpose of explanation, specific details are set forth to provide an understanding of the present disclosure. However, it will be apparent to those skilled in the art that embodiments may be practiced without these details. In addition, those skilled in the art will recognize that the embodiments of the present disclosure described below may be implemented in a variety of ways, such as a process, apparatus, system, device, or method on a tangible computer-readable medium.

[0021] The components or modules shown in the figures are illustrations of exemplary embodiments of the present invention and are intended to avoid obscuring the present disclosure. It should also be understood that throughout the discussion, components may be described as separate functional units, which may include subunits, but those skilled in the art will recognize that various components or portions thereof may be separated into separate components or may be integrated together, including within a single system or component. It should be noted that the functions or operations discussed herein may be implemented as components. Components may be implemented with software, hardware, or a combination thereof.

[0022] In addition, the connections between components or systems in the figure are not limited to direct connections. Instead, the data between these components can be modified, reformatted or otherwise changed by intermediate components. In addition, more or fewer connections can be used. It should also be noted that the terms "connection", "connection" or "communication connection" should be understood to include direct connections, indirect connections through one or more intermediate devices, and wireless connections.

[0023] References in the specification to "one embodiment", "preferred embodiment", "embodiment" or "some embodiments" mean that the particular features, structures, characteristics or functions described in conjunction with the embodiment are included in at least one embodiment of the present invention, and may be in more than one embodiment. In addition, the above phrases appearing in different places in the specification do not necessarily all refer to the same one or more embodiments.

[0024] Certain terms are used in various places in the specification for the purpose of explanation and should not be construed as limiting. A service, function, or resource is not limited to a single service, function, or resource; use of these terms may refer to a distributed or aggregated set of related services, functions, or resources. An image may be a still image or an image from a video.

[0025] The terms "include," "comprising," "including," and "comprising" are to be construed as open-ended terms and any listed items that follow are examples and are not meant to be limited to the listed items. Any headings used herein are for organizational purposes only and should not be used to limit the scope of the specification or claims. Each reference cited herein is incorporated by reference in its entirety.

[0026] In addition, those skilled in the art should recognize that: (1) certain steps may be performed optionally; (2) the steps may not be limited to the specific order described herein; (3) certain steps may be performed in a different order; and (4) certain steps may be performed simultaneously.

[0027] A. introduction

[0028] Despite breakthroughs in many areas such as image classification using deep neural networks, these models can be easily fooled by so-called adversarial examples. In image classification, an adversarial example of a natural image is a modified version that is visually indistinguishable from the original, but causes the classifier to produce a different label prediction. Adversarial examples have shown ubiquitous superiority in classification ranging from object detection to speech recognition.

[0029] There has been encouraging progress in improving model robustness against adversarial examples. Among them, adversarial training is one of the most popular techniques, which uses adversarially perturbed images instead of original images for model training. However, there are some challenges to be addressed. First, some adverse effects such as label leakage remain issues that hinder adversarial training. Currently available remedies increase the number of iterations for generating attacks, or use classes other than the true value to generate attacks. Increasing attack iterations will increase training time proportionally, while label leakage cannot be completely eliminated using non-true target methods. Second, previous methods for both standard training and adversarial training treat each training sample individually and treat each sample in isolation from other samples. Manipulating each sample individually in this way ignores the relationship between samples and fails to fully exploit the potential of attack and defense, thereby limiting performance.

[0030] Manifolds and neighborhood structures have been shown to be effective in capturing inter-sample relationships. Natural images exist on a low-dimensional manifold with training and test images as samples. Modern classifiers are over-parameterized, and different local minima have been shown to be equally effective in the clean image setting. However, different solution points may make use of different sets of features for prediction. To learn a well-performing classifier on natural images, it is sufficient to adjust the classification boundary to intersect this manifold at locations with good separation between classes on the training data, since the test data will largely reside on the same manifold. However, the classification boundary that extends outside the manifold is less constrained, which helps in the presence of adversarial examples. For example, it has been pointed out that some clean trained models focus on a few features that are discriminative but not so robust, and are therefore vulnerable to adversarial attacks. Therefore, conventional supervised attacks that attempt to move feature points towards this decision boundary may ignore the original data manifold structure. When the decision boundary is close to the manifold for its off-manifold part, the adversarial perturbation causes a tilting effect on the data manifold; where the classification boundary is far from the manifold for its off-manifold part, the adversarial perturbation moves points towards the decision boundary, effectively shrinking the data manifold. Since adversarial examples reside in large continuous regions and a large part of the adversarial subspace is shared, purely label-guided adversarial examples will at least be chaotic in the shared adversarial subspace. In summary, while these effects force the model to pay more attention to the current decision boundary, they also cause the effective data manifold for training to deviate from the original decision boundary, potentially hindering performance.

[0031] In light of these observations, this paper shifts the previous focus on decision boundaries to inter-sample structures. Embodiments of the disclosed feature dispersion method can be intuitively understood as generating adversarial examples by perturbing the local neighborhood structure in an unsupervised manner and then performing model training on the generated adversarial images. Figure 1 The overall framework 100 of feature-diversified based adversarial training is shown, and Figure 2 The process of using the framework 100 to perform adversarial training based on feature dispersion according to an embodiment of the present disclosure is shown. The adversarial perturbation is jointly generated by feature dispersion, that is, maximizing the clean sample {x i} and the disturbed sample {x′ j}. By using the perturbed image {x′ j} as training samples to minimize the cross entropy loss and update the model parameters θ.

[0032] refer to Figure 1 and Figure 2 , feature extractor f θ (·) 120 clean images from the clean batch {x i}105 extract (205) multiple feature vectors {fi}130, and from the perturbed image {x′ j}115 extract multiple disturbance feature vectors {f ' j}135. In one or more embodiments, f θ (·) is implemented as an image model, such as a deep neural network up to the softmax layer. In one or more embodiments, the perturbed images x′ of the perturbed batch can be initially obtained by applying the initial perturbation to the clean batch of images j In one or more embodiments, by i and radius ∈ l ∞ Each clean image {x i} applies random perturbations to complete the initial perturbation of the clean batch. Based on multiple feature vectors {f i}130 and multiple perturbation feature vectors {f′ j}135 obtain (210) a transmission cost matrix C140, where the element C of the transmission cost matrix C ij is defined as the image feature x i and x′ j The OT solver 145 is used to obtain (215) one or more transmission matrices T 150 based on the transmission cost matrix C. The best transmission is selected (220) from the one or more transmission matrices T to obtain the best transmission for the two batches of images {x i} and {x′ j} feature matching distance between The maximum value is 155. The feature matching distance can be obtained by the Frobenius dot product between the transmission matrix T and the transmission cost matrix C.

[0033] In one or more embodiments, one or more iteration steps may be performed ( Figure 1 (not shown) to use the maximum feature matching distance obtained To update (225) the perturbed image x′ of the perturbed batch j , until a predetermined threshold, such as the number of iterations, is reached.

[0034] In one or more embodiments, a plurality of perturbed feature vectors {f′} of perturbed images from an updated perturbed batch j}135 is used to determine (230) the label y′ in the softmax layer 160 of the deep neural network j . It is possible to obtain (235) based on the determined label y′ j and the true value label y corresponding to the clean batch 105 jThe loss function 165 of 110 is used to train the deep neural network, and the goal is to minimize the loss function. One or more training epochs can be performed until a predetermined number of training epochs is reached and the final model parameters θ of the deep neural network are output.

[0035] although Figure 1 and Figure 2 Some embodiments disclosed below are related to image models, but those skilled in the art will appreciate that adversarial training based on feature dispersion can also be used for other applications, including but not limited to speech recognition, etc. Such changes should still be within the scope of the present invention.

[0036] Some contributions of this paper can be summarized as follows:

[0037] Embodiments of novel feature-dispersed methods for generating adversarial images to facilitate adversarial training in a collaborative and unsupervised manner are disclosed;

[0038] Embodiments of adversarial training formulations are proposed that deviate from the conventional minimax formulation and fall into the broader category of bi-level optimization;

[0039] The proposed implementation is analyzed and compared with several state-of-the-art techniques, and extensive experiments are performed on many standard benchmarks to verify the effectiveness of the proposed method.

[0040] B. Some related work

[0041] 1. Adversarial attack, defense, and adversarial training

[0042] Adversarial examples have attracted a lot of attention recently. It has been pointed out that CNN is vulnerable to adversarial examples, and an algorithm based on L-BFGS has been proposed to generate adversarial examples. The Fast Gradient Sign Method (FGSM) has been developed for generating adversarial attacks and used for adversarial training. Many variants of the attack have been developed later. Meanwhile, many efforts have been made to defend against adversarial examples. Recently, some studies have shown that many existing defense methods have a misunderstanding of their robustness against adversarial attacks due to gradient masking, and adversarial training is one of the effective defense methods against adversarial attacks. It improves model robustness by solving the minimax problem as follows:

[0043]

[0044] Among them, the maximization operation of the inner layer essentially generates an attack, while the minimization operation of the outer layer corresponds to minimizing the "adversarial loss" caused by the attack of the inner layer. For example, a single-step method such as the Fast Gradient Sign Method (FGSM) (I. Goodfellow et al. "Explaining and Taming Adversarial Examples" in the 2015 International Conference on Learning Representations) or a multi-step projected gradient descent (PGD) method can be used to approximate the inner layer's maximization operation:

[0045]

[0046] in, It is to project the input into the feasible region S x In the PGD method, the original image x is randomly perturbed to a point x in B(x, ∈) 0 , ∈ cube around x, and then goes through multiple PGD steps with step size α, as shown in formula (2).

[0047] Label leakage and gradient masking are some of the well-known problems that hinder adversarial training. Label leakage occurs when the additive perturbations are highly correlated with the true label. Therefore, when added to an image, the network can directly tell the class label by decoding the additive perturbations without relying on the true content of the image, resulting in higher adversarial accuracy than clean images during training. Gradient masking refers to the effect where an adversarially trained model learns to “improve” robustness by generating less useful gradients against adversarial attacks, which may be ignored by the alternative model used to generate the attack, giving a false sense of robustness.

[0048] 2. Different distances for feature and distribution matching

[0049] The Euclidean distance is arguably one of the most commonly used metrics for measuring the distance between pairs of points. When two sets of points are involved, it is natural to sum the distances of the individual pairs as a measure of the distance between the two sets, given an appropriate correspondence. Alternatively, each set can be viewed as an empirical distribution, and the distance between them measured using the Kullback-Leibler (KL) or Jensen-Shannon (JS) divergence. The challenge with learning with the KL or JS divergence is that it does not provide useful gradients when the two empirical distributions have disjoint supports or have a non-empty intersection that is contained in the null set of the measure. The optimal transport (OT) distance is an alternative measure of the distance between distributions that has advantages over KL and JS in the previously mentioned scenarios. The OT distance between the two probability metrics μ and v can be defined as:

[0050]

[0051] where Π(μ, v) denotes the set of all joint distributions γ(x, y) with marginal μ(x) and v(y), and c(x, y) is a cost function (e.g., Euclidean distance or cosine distance). Intuitively, is the minimum cost of transferring γ from μ to v. It provides a weaker topology than many other metrics, which is important for applications where the data often resides on a low-dimensional manifold in the input embedding space (which happens to be the case for natural images). The OT distance has been widely used in many tasks such as generative modeling, autoencoding, and dictionary learning.

[0052] C. Example for feature-diversification-based adversarial training

[0053] In this section, in order to improve adversarial robustness, an embodiment of adversarial training based on feature dispersion is disclosed.

[0054] 1. Examples for feature matching and feature dispersion

[0055] Feature matching. Conventional training treats the training data as independent and uniformly distributed (iid) samples from the data distribution, thus ignoring the connections between samples. The same assumption is used when generating adversarial samples for training, where the direction in which the sample is perturbed is purely based on the direction from the current data point to the decision boundary, regardless of other samples. Although effective, it ignores the interrelationships between different feature points, since the adversarial perturbation is calculated for each sample individually, thus ignoring any collective distributional properties. In addition, reference Figure 3A ,like Figure 3B As shown in , supervised generation of attacks makes the generated perturbations highly biased towards the decision boundary. FIG. 3A to FIG. 3C , the coverage boundary 315 comes from a model trained on clean data. This is less than ideal because it may ignore other directions that are critical for learning a robust model and may cause label leakage due to the high correlation between the perturbation and the decision boundary. In contrast, Figure 3C An example of perturbed data generated using the proposed feature dispersion method according to an embodiment of the present disclosure is graphically illustrated.

[0056] In this paper, an embodiment of using the relationship between samples for learning dates is used to perform adversarial perturbations. By using the OT distance in formula (3), the pursuit of exploiting local structures and the pursuit of seamless integration with the end-to-end training framework naturally inspires OT-based soft matching schemes. In one or more embodiments, OT between discrete distributions is considered hereafter, because one of the focuses of this paper is to apply OT distance to image features. Specifically, consider two discrete distributions It can be written as and where δ xis a Dirac function centered at X. In one or more embodiments, the two discrete distributions may have the same or different sizes. The weight vector and All belong to n-dimensional simplex, that is, ∑ i u i =∑ i v i = 1, because both μ and v are probability distributions. In this setting, calculating the OT distance defined in formula (3) is equivalent to solving the following network traffic problem

[0057]

[0058] in 1 n are vectors that are the same in all n dimensions. <·,·〉 represents the Frobenius dot product. T is the transmission matrix. C is the transmission cost matrix such that C ij =c(x i , x′ j ). In one or more embodiments, the transmission cost may be defined as the cosine distance between image features:

[0059]

[0060] where f θ (·) represents a feature extractor with parameter θ. In one or more embodiments, f θ (·) is implemented as a deep neural network up to the softmax layer. The feature matching distance can be formally defined as follows.

[0061] Definition 1. (Feature matching distance) The feature matching distance between two sets of images is defined as The OT distance between the empirical distributions μ and v of these two groups.

[0062] It should be noted that when f θ (·) When used to extract features in the calculation of ground distance as in formula (5), the feature matching distance is also a function of θ (i.e. ). In one or more embodiments, when there is no danger of confusion to minimize conceptual confusion, the symbol is simply used below.

[0063] Feature dispersion. Based on the feature matching distance defined above, the proposed feature dispersion method can be expressed as follows:

[0064]

[0065] In one or more embodiments, this can be intuitively interpreted as maximizing the feature matching distance S between the original empirical distribution and the perturbed empirical distribution with respect to the domain-constrained input. μ

[0066]

[0067] where B(x,∈)={z|||zx|| ∞ ≤∈} means l with center x and radius ∈ ∞ Cube. Formally, the concept of feature dispersion can be expressed as follows.

[0068] Definition 2. (Feature Dispersion) Given a set of clean data {x i}, which can be expressed as an empirical distribution as where ∑ i u i = 1, the feature dispersion process is defined as maximizing (the feature matching distance between μ and v subject to domain constraints and budget constraints) to generate the perturbed empirical distribution where ∑ i v i =1.

[0069] Since feature diversification is performed over a batch of samples using inter-sample structure, it is more effective as an adversarial attack than structure-agnostic random perturbations, while being less constrained than supervised perturbations that are oriented toward decision boundaries and suffer from label leakage. Empirical comparisons are provided in Section E.

[0070] 2. Example of adversarial training with feature dispersion

[0071] In this subsection, embodiments of adversarial training with feature dispersion are disclosed.

[0072] In one or more embodiments, feature dispersion for adversarial training can be exploited by the following mathematical formula:

[0073]

[0074] In one or more embodiments, the proposed formula is different from the conventional minimax formula for adversarial training. More specifically, it can be viewed as an instance of a more general two-level optimization problem. Feature dispersion is effective for adversarial training scenarios because more data is required. Unlike conventional supervised methods, feature dispersion promotes data diversity without significantly changing the structure of the data manifold like conventional supervised methods with label leakage (which is an apparent phenomenon). Secondly, the feature matching distance connects samples within a batch together, thereby collaboratively generating generated adversarial attacks by considering inter-sample relationships. Thirdly, feature dispersion implicitly causes connection regularization (described in detail below) on model training, thereby utilizing inter-sample structure for joint regularization.

[0075] The overall process of the disclosed method can be presented in Method 1 shown below.

[0076] Method 1: Adversarial training based on feature dispersion

[0077]

[0078] Figure 4 A method for feature-dispersion-based adversarial training according to an embodiment of the present disclosure is shown. An image dataset S, a training epoch K, a batch size n, an attack budget ∈, and an attack iteration T are received (405) as input. For each training epoch, a random batch of images is selected (410) from the image dataset S. The random batch includes a plurality of images x i , where each image is associated with a corresponding true value label y i Implement (415) initialization, that is, generate an empirical distribution μ from a random batch of images and a perturbed batch of images x′ with an initial perturbation to the random batch of images i In one or more embodiments, the empirical distribution μ is an n-dimensional simplex vector. i and radius ∈ l ∞ The initial perturbation of a random batch of images is done by applying random perturbations within the cube B(x,∈). Then, based on focusing on each of the initial perturbed images x′ i The Dirac function and weight vector v i The perturbed empirical distribution v is initially obtained by using the distribution of . In one or more embodiments, the distribution of the weight vector is initially a uniform distribution.

[0079] One or more iteration steps are performed (420) to maximize the feature matching distance between μ and v. To update the perturbation experience distribution v from the initial perturbation experience distribution, and use the updated perturbation experience distribution v to update the perturbation batch image x′ iIn one or more embodiments, the feature matching distance is the OT distance between the empirical distribution μ and v. In one or more embodiments, the perturbed batch image x′ is calculated i of Gradient And the gradient is used to update the perturbed batch image. After that, the sign operator sign(·) is applied to this gradient, and then the signed gradient is used to perturb the batch of images x′ i Take an ascending step size scaled by the step size of the budget ∈ and then project these images into the feasible region S x to update the perturbed batch images. The above iterative process can be repeated for a predetermined attack iteration T to output the perturbed images of the final perturbed batch. Then one or more adversarial training steps are performed (425) to learn the image model parameters θ using at least the perturbed images and true value labels of the final perturbed batch. More training epochs can be performed (430) until a predetermined number of training epochs K is reached and the final model parameters θ are output. In each training epoch, a random batch is sampled from the dataset, and the above training process is repeated for each training epoch. In one or more embodiments, the random batch is sampled without replacement so that any image in the dataset has only one chance to be selected.

[0080] D. discuss

[0081] Manifold-based defenses. Some have suggested defending by projecting the perturbed image onto an appropriate manifold. Some have used similar ideas of manifold projection to approximate this step by nearest neighbor search against a web-scale database. Differently, in embodiments of this paper, manifolds are exploited in the form of relationships between samples to generate perturbations, which induce implicit regularization of the model when used in an adversarial training framework. While defenses can be implemented by shrinking the perturbed input toward the manifold, in embodiments of this paper, the manifold is expanded to generate perturbed inputs for adversarial training using feature dispersion.

[0082] Inter-sample regularization. Some people generate training samples by linear interpolation between pairs of natural samples, thereby introducing a linear inductive bias near the training samples. Therefore, the model is expected to reduce the amount of undesired oscillations of non-manifold samples. Some people have proposed adversarial logit pairing to increase the original training loss of the "pairing" loss, which measures the difference between the logits of clean images and adversarial images. The idea is to use the natural logit as a reference to suppress pseudo-logit responses. Similarly, some people have proposed a regularization method, namely virtual adversarial training, which uses a regularization term based on the KL divergence of the predicted probabilities of the original image and the adversarial perturbation image. In one or more embodiments of the proposed model, the inter-sample relationship is exploited to generate an adversarial perturbation that induces an implicit regularization term in the objective function connected to all input samples.

[0083] Wasserstein GAN and OT-GAN. Generative adversarial networks (GANs) are a technique that implicitly captures data distributions by directly generating samples. It initially suffered from problems with training instability and mode collapse. OT-related distances have been used to overcome difficulties encountered in original GAN ​​training. The technique has been further extended to generate discrete data, such as text. Unlike GANs, which maximize the discrimination criterion with respect to the parameters of the discriminator in order to better capture the data distribution, in the embodiments of this article, feature matching distances are maximized relative to the perturbed input to generate appropriate training data to improve model robustness.

[0084] E. experiment

[0085] It should be noted that these experiments and results are provided by way of illustration and were performed under specific conditions using one or more specific embodiments; therefore, neither these experiments nor their results should be used to limit the scope of the disclosure of this patent document.

[0086] Baselines and implementation details. The implementation of feature-distribution-based adversarial training is presented in PyTorch in one or more experimental settings. Extensive experiments are conducted on several benchmark datasets including CIFAR10 and CIFAR100 ("Learning multiple layers of features from tiny images" in the technical report by A. Krizhevsky, 2009). In one or more experimental settings, WideResNet WRN-28-10 (S. Zagoruyko et al., “Wide residual networks”, British Conference on Machine Vision, 2016) is used as a network structure following Madry et al. (“Towards deep learning models resistant to adversarial attacks”, International Conference on Learning Representations, 2018). The performance of embodiments of the proposed method is compared with a number of baseline methods, including: i) a model trained using standard methods for clean images (Standard) (A. Krizhevsky, Technical Report, 2009, “Learning multiple layers of features from tiny images”); ii) a PGD-based method from Madry; iii) another recent method that performs adversarial training with both image and label adversarial perturbations (Bilateral) (J. Wang et al., “Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks”). For training, the initial learning rate γ is 0.1 for CIFAR. In one or more experimental settings, the number of epochs is set to 100 for both standard and Madry methods, with transition epochs of {60, 90}, as it is empirically observed that the performance of trained models stabilizes before 100 epochs. The 200 epoch training schedule is similar to Bilateral (using the same transition epoch as empirically observed), which helps improve model performance, possibly due to increased data variation via feature dispersion.In one or more experimental settings, standard data augmentation is performed, including random crops with 4-pixel padding and random horizontal flips during training. A perturbation budget of ∈=8 is used in training following Madry. In one or more experimental settings, the Sinkhorn algorithm is used with label smoothing of 0.5, attack iterations of 1, and regularization of 0.01. For testing, model robustness is evaluated by approximately computing an upper bound on the robustness on the test set, by measuring the accuracy of the model under different adversarial attacks, including white-box FGSM, PGD (Madry), CW (CW loss within the PGD framework) (N. Carlini and D. Wagner, "Towards Evaluating the Robustness of Neural Networks" at the 2017 IEEE Symposium on Security and Privacy), and variants of black-box attacks.

[0087] 1. Visual classification performance under white-box attack

[0088] CIFAR10. One or more experiments are conducted on CIFAR10, a popular dataset that is widely used in the adversarial training literature with 10 classes, 5K training images per class, and 10K test images. The accuracy is reported for the original test images (clean) as well as under PGD and CW attacks with T iterations (PGDT and CWT). The evaluation results are summarized in Table 1. Under different white-box attacks, it is observed that the standard model fails drastically. The Madry method significantly improves the model robustness over the standard model. Under the standard PGD20 attack, it achieves 44.9% accuracy. The Bilateral method further improves the performance to 57.5%. The proposed method outperforms both methods by a large margin, improving over Madry by 25.6% and outperforming Bilateral by 13.0%, achieving 70.5% accuracy under the standard 20-step PGD attack. A similar pattern has been observed for the CW metric.

[0089] In one or more experiments, the robustness of the model against PGD attackers under different attack budgets is further evaluated with a fixed attack step size of 20. The results are shown in Figure 5A As shown in Figure 2, it is observed that the performance of the standard model decreases rapidly with the increase of the attack budget. The Madry model significantly improves the model robustness over a wide range of attack budgets. Under different attack budgets, the proposed method further significantly improves the performance over the Madry model. Experiments are also conducted using a PGD attacker with different attack iterations with a fixed attack budget of 8. Figure 5B and Figure 5Cand is also shown in Table 1. It is observed that both Madry and the proposed method can maintain fairly stable performance when the number of attack iterations increases. It should be noted that the proposed method consistently outperforms the Madry model over a wide range of attack iterations. It is also observed from Table 1 that the proposed method also outperforms Bilateral under all variants of PGD and CW attacks. In one or more experimental settings, PGD / CW attackers with ∈=8 and resulting attack step sizes of 20 and 100 are used as part of the threat model.

[0090] Table 1: Accuracy comparison of the proposed method with the standard, Madry and Bilateral methods on CIFAR10 under different threat models

[0091]

[0092] Table 2: Accuracy comparison on CIFAR100

[0093]

[0094] CIFAR100. Experiments are also conducted on the CIFAR100 dataset, which has 100 classes, 50K training images, and 10K test images. It should be noted that this dataset is more challenging than CIFAR10 because the number of training images for each class is ten times smaller than that of CIFAR10. As shown in the results in Table 2, the proposed method significantly outperforms all baseline methods, outperforming Madry and Bilateral by about 20% under the PGD attack, and outperforming by about 10% under the CW attack. The superior performance of the proposed method on this dataset further demonstrates the importance of exploiting the inter-sample structure for learning.

[0095] 2. Ablation studies

[0096] In this subsection, the impact of algorithmic factors is studied.

[0097] Importance of feature dispersion. The effectiveness of feature dispersion is empirically validated by comparing the performance of models trained using different perturbation schemes: I) Random: a natural baseline method that randomly perturbs each sample within the ε neighborhood; ii) Supervised: perturbations generated in a supervised manner using the true value labels; iii) Feature dispersion: perturbations generated using the proposed feature dispersion method. Except for the perturbation scheme used, all other hyperparameters remain exactly the same. The results are summarized in Table 3. Clearly, the disclosed feature dispersion (FeaScatter) method outperforms both the random and supervised methods, demonstrating its effectiveness. In addition, since it is the main factor that differs from the conventional adversarial training pipeline, this result shows that feature dispersion is the main contributor to improving adversarial robustness.

[0098] Table 3: Importance of feature dispersion

[0099]

[0100] Table 4: Impact of different matching schemes

[0101]

[0102] Role of matching. The role of the matching scheme in the feature dispersion factor is further investigated by comparing several different schemes: i) uniform matching, which matches each clean sample uniformly with all perturbed samples in the batch; ii) identity matching, which matches each clean sample only with its perturbed sample; iii) OT-matching: the proposed method is to assign soft matches between clean samples and perturbed samples according to an optimization criterion. The results are summarized in Table 5. It is observed that all variants of the matching scheme produce comparable or better performance than the state-of-the-art methods, which means that the proposed framework is generally effective. It is worth noting that OT-matching produces the best results, which shows the importance of correct matching for feature dispersion.

[0103] Impact of OT-solvers. Exact minimization of formula (4) within T is generally intractable. In one or more experiments, two practical solvers, the Sinkhorn algorithm (M. Cuturi, "Sinkhorn distances: Lightspeed computation of optimal transport", in Progress in Neural Information Processing Systems, 2013) and the Inexact Proximal Point Method for Optimal Transport (IPOT) algorithm (Y. Xie et al., "A Fast Proximal Point Method For Wasserstein Distance", arXiv:1802.04307, 2018) were compared. The results are summarized in Table 5. The results show that different instantiations of the proposed method using different OT solvers lead to comparable performance, which means that the proposed method is generally effective regardless of the choice of OT solver.

[0104] Table 5: Impact of OT solvers. Embodiments of the proposed method using different OT solvers perform well.

[0105]

[0106] 3. Performance under black-box attacks

[0107] To further verify whether the degradation minimum is obtained, the robustness of the model trained using the proposed method is evaluated against black-box attacks (B-attacks). Two different models are used to generate test time attacks: i) Undefended: an undefended model trained using the standard method; ii) Siamese: a robust model obtained from another training session using the proposed method. As shown in the results in Table 5 below, the model trained using the proposed method is robust against different types of black-box attacks, verifying that a non-degenerate solution is learned.

[0108] Table 5 Performance of different models under black box attack

[0109]

[0110] F. Some conclusions

[0111] Embodiments of feature-dispersion-based adversarial training methods are disclosed herein. Embodiments of the proposed method are distinguished from other embodiments by using an unsupervised feature-dispersion method for generating adversarial training images, which exploits inter-sample relationships for collaborative perturbation generation. In one or more embodiments, a joint regularization term is derived from feature dispersion for adversarial training. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed method.

[0112] G. System Example

[0113] In various embodiments, aspects of this patent document may be directed to, may include, or may be implemented on one or more information processing systems / computing systems. A computing system may include any means or collection of means operable to compute, calculate, determine, classify, process, send, receive, retrieve, initiate, route, switch, store, display, communicate, display, detect, record, reproduce, dispose of, or utilize any form of information, intelligence, or data. For example, a computing system may be or may include a personal computer (e.g., a notebook computer), a tablet computer, a tablet computer, a personal digital assistant (PDA), a smart phone, a smart watch, a smart bag, a server (e.g., a blade server or a rack server), a network storage device, a camera, or any other suitable device, and may vary in size, shape, performance, functionality, and price. A computing system may include random access memory (RAM), one or more processing resources, such as a central processing unit (CPU) or hardware or software control logic, ROM, and / or other types of memory. Additional components of a computing system may include one or more disk drives, one or more network ports for communicating with external devices and various input and output (I / O) devices such as keyboards, mice, touch screens, and / or video displays. The computing system may also include one or more buses operable to route communications between the various hardware components.

[0114] Figure 6 600 is a simplified block diagram of a computing device / information processing system (or computing system) according to an embodiment of the present disclosure. It should be understood that the functions shown in system 600 can operate to support various embodiments of the computing system - although it should be understood that the computing system can be configured differently and include different components, including having Figure 6 Fewer or more components than shown.

[0115] like Figure 6 As shown, the computing system 600 includes one or more central processing units (CPUs) 601 that provide computing resources and control the computer. The CPU 601 can be implemented with a microprocessor, etc., and can also include one or more graphics processing units (GPUs) 619 and / or floating-point coprocessors for mathematical calculations. The system 600 can also include a system memory 602, which can be in the form of a random access memory (RAM), a read-only memory (ROM), or both.

[0116] Multiple controllers and peripherals are also available, such as Figure 6As shown. Input controller 603 represents an interface to various input devices 604, such as a keyboard, mouse, touch screen and / or stylus. The computing system 600 may also include a storage controller 607 for interfacing with one or more storage devices 608, each storage device including a storage medium such as a tape or disk, or an optical medium that can be used to record instruction programs of operating systems, utilities and applications, which may include embodiments of programs that implement various aspects of the present invention. Storage device 608 may also be used to store processed data or data to be processed according to the present invention. System 600 may also include a display controller 609 for providing an interface to a display device 611, which may be a cathode ray tube (CRT), a thin film transistor (TFT) display, an organic light emitting diode, an electroluminescent panel, a plasma panel, or other type of display. The computing system 600 may also include one or more peripheral device controllers or interfaces 605 for one or more peripheral devices 606. Examples of peripheral devices may include one or more printers, scanners, input devices, output devices, sensors, etc. The communication controller 614 can be connected to one or more communication devices 615 interfaces, which enables the system 600 to connect to remote devices via any of a variety of networks including the Internet, cloud resources (e.g., Ethernet cloud, Fibre Channel over Ethernet (FCoE) / Data Center Bridging (DCB) cloud, etc.), local area network (LAN), wide area network (WAN), storage area network (SAN) or via any suitable electromagnetic carrier signal (including infrared signal).

[0117] In the system shown, all major system components can be connected to bus 616, which can represent more than one physical bus. However, various system components may or may not be physically close to each other. For example, input data and / or output data can be sent remotely from one physical location to another physical location. In addition, programs that implement various aspects of the present invention can be accessed from a remote location (e.g., a server) over a network. Such data and / or programs can be transmitted via any of a variety of machine-readable media, including but not limited to: magnetic media such as hard disks, floppy disks, and tapes; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices specifically configured to store or store and execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices.

[0118] Aspects of the present invention may be encoded on one or more non-transitory computer-readable media with instructions that are executed by one or more processors or processing units so that the steps are performed. It should be noted that one or more non-transitory computer-readable media should include volatile and non-volatile memory. It should be noted that alternative embodiments are possible, including hardware embodiments or software / hardware embodiments. Hardware-implemented functions can be implemented using ASICs, programmable arrays, digital signal processing circuits, etc. Therefore, the term "device" in any claim is intended to cover software and hardware embodiments. Similarly, the term "computer-readable medium" used herein includes software and / or hardware containing instruction programs thereon, or a combination thereof. In view of these alternative embodiments, it should be understood that the drawings and accompanying descriptions provide functional information required for those skilled in the art to write program code (i.e., software) and / or manufacture circuits (i.e., hardware) to perform the desired processing.

[0119] It should be noted that embodiments of the present invention may also relate to computer products having non-transitory tangible computer-readable media having computer code thereon for performing various computer-implemented operations. The media and computer code may be those specifically designed and constructed for the purposes of the present invention, or they may be of a type known or available to those skilled in the relevant art. Examples of tangible computer-readable media include, but are not limited to: magnetic media such as hard disks, floppy disks, and tapes; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices specifically configured to store or store and execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices. Examples of computer code include machine code, such as code generated by a compiler, and files containing high-level code executed by a computer using an interpreter. Embodiments of the present invention may be implemented in whole or in part as machine executable instructions in program modules executed by a processing device. Examples of program modules include libraries, programs, routines, objects, components, and data structures. In a distributed computing environment, program modules may be physically located in a local, remote, or both setting.

[0120] Those skilled in the art will recognize that the computing system or programming language does not play a crucial role in the practice of the present invention. Those skilled in the art will also recognize that many of the above elements can be physically and / or functionally divided into submodules or combined together.

[0121] Those skilled in the art will appreciate that the foregoing examples and embodiments are exemplary and do not limit the scope of the present disclosure. All permutations, enhancements, equivalents, combinations, and improvements that are apparent to those skilled in the art upon reading the specification and studying the drawings are included within the true spirit and scope of the present disclosure. It should also be noted that the elements of any claim may be arranged in different ways, including with multiple dependencies, configurations, and combinations.

Claims

1. A computer-implemented method for generating adversarial training data for training a parameterized model, comprising: Generate an initial set of perturbed images from a set of clean images; Using the parameterized model, extracting features from the set of clean images and the set of initial perturbed images; calculating a transfer cost matrix between features extracted from the set of clean images and features extracted from the set of initial perturbed images; Determine a transfer matrix based on the transfer cost matrix using an OT solver, wherein the transfer cost matrix represents a degree of matching between features in the set of clean images and features in the set of initial perturbed images; Determining a feature matching distance between the set of clean images and the set of initial perturbation images by calculating a product between the transfer cost matrix and the transfer matrix; as well as A set of feature dispersion-based training data is obtained based at least on the determined feature matching distance.

2. The computer-implemented method of claim 1 , further comprising: extracting features from the set of feature-dispersion-based training data using the parameterized model; For each data sample, determining a class label probability based on features extracted from the set of feature-dispersion-based training data; determining a loss based on the determined class label probabilities and a true label associated with each of the data samples; as well as Model parameters of the parameterized model are updated using the determined loss.

3. The computer-implemented method of claim 1, wherein the parameterized model is an image classification model.

4. The computer-implemented method of claim 3, wherein the initial set of perturbed images is generated by adding random noise within a predetermined image perturbation budget to the clean images.

5. The computer-implemented method of claim 3, wherein the transfer cost matrix comprises a plurality of elements, wherein each element represents a distance between a feature of a clean image and a feature of a perturbed image.

6. The computer-implemented method of claim 1 , wherein the determined feature matching distance corresponds to a minimum feature matching distance between an empirical distribution from the set of clean images and a perturbed empirical distribution from the initial set of perturbed images.

7. The computer-implemented method of claim 1 , wherein obtaining the set of feature dispersion-based training data based at least on the determined feature matching distance comprises: Obtaining a gradient of the determined feature matching distance; applying a symbolic operator to the gradient; Taking an ascending step size operation scaled by a step size on the set of initial perturbed images; as well as The set of initial perturbed images with the ascending step size is projected to a feasible region to obtain the set of feature dispersion-based training data.

8. The computer-implemented method of claim 7, wherein the gradient is relative to the set of initial perturbed images.

9. The computer-implemented method of claim 7, wherein the step size is a predetermined perturbation budget.

10. A computer-implemented method for adversarially training an image model, the method comprising: Receive a batch of clean images and their corresponding ground truth labels, as well as a perturbation budget; Generating an empirical distribution from the batch of clean images and generating a perturbed empirical distribution from a batch of perturbed perturbed images to achieve initialization, wherein the initial perturbation of the perturbed images is achieved by the initial perturbation of the batch of clean images; In one or more iterative steps, updating the perturbed empirical distribution from an initially generated perturbed empirical distribution by maximizing a feature matching distance between the empirical distribution and the perturbed empirical distribution, and updating the perturbed image using the updated perturbed empirical distribution; as well as One or more adversarial training steps are performed using at least the updated perturbed images and the true value labels to learn parameters of the image model.

11. The computer-implemented method of claim 10, wherein the batch of clean images is a random sample from an image dataset.

12. The computer-implemented method of claim 10, wherein the initial perturbation of the batch of clean images is accomplished by applying random perturbations within a predetermined perturbation budget.

13. The computer-implemented method of claim 12, wherein the perturbed empirical distribution is initially obtained based on an initial uniform distribution of Dirac functions and weight vectors centered on each initial perturbed image.

14. The computer-implemented method of claim 10, wherein the feature matching distance is an optimal transmission (OT) distance.

15. A non-transitory computer-readable medium comprising one or more sequences of instructions which, when executed by one or more processors, cause execution of steps for generating adversarial training data for training a parameterized model, the steps comprising: Generate an initial set of perturbed images from a set of clean images; Extracting features from the set of clean images and the set of initial perturbed images; calculating a transfer cost matrix between features extracted from the set of clean images and features extracted from the set of initial perturbed images; Determine a transfer matrix based on the transfer cost matrix using an OT solver, wherein the transfer cost matrix represents a degree of matching between features in the set of clean images and features in the set of initial perturbed images; Determining a feature matching distance between the set of clean images and the set of initial perturbation images by calculating a product between the transfer cost matrix and the transfer matrix; as well as A set of feature dispersion-based training data is obtained based at least on the determined feature matching distance.

16. The non-transitory computer readable medium of claim 15, wherein the steps further comprise: extracting features from the set of feature-dispersion-based training data using the parameterized model; For each data sample, determining a class label probability based on features extracted from the set of feature-dispersion-based training data; determining a loss based on the determined class label probabilities and a true label associated with each of the data samples; as well as Model parameters of the parameterized model are updated using the determined loss.

17. The non-transitory computer-readable medium of claim 16, wherein obtaining the set of feature dispersion-based training data based at least on the determined feature matching distance comprises: Obtaining a gradient of the determined feature matching distance; applying a symbolic operator to the gradient; Applying a step-up operation scaled by a predetermined perturbation budget to the set of initial perturbation images; as well as The set of initial perturbed images with the ascending step size is projected to a feasible region to obtain the set of feature dispersion-based training data.

18. The non-transitory computer readable medium of claim 17, wherein the determined gradient of feature matching distance is relative to the initial set of perturbed images.

19. The non-transitory computer-readable medium of claim 17, wherein the transfer cost matrix comprises a plurality of elements, wherein each element represents a distance between a feature of a clean image and a feature of a perturbed image.

Images