Computer-implemented method, apparatus, computer program, and computer-readable medium

Abstract

To provide a computer-implemented method, program, storage medium and apparatus for realizing constrained neural networks through alternative encoding for robustness.SOLUTION: A neural network 302 includes: associating an additional layer 316 fully connected to an output layer and having a lower dimensionality than at least one intermediate layer; applying encoding to the additional layer, where the encoding includes an encoding vector for each output class; training the neural network including the additional layer and the applied encoding so as to learn a reduced feature set representing one or more features that contain a high amount of information about at least one output class; and using the trained neural network to perform classification that is robust against adversarial examples.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates generally to information security, and more particularly to protecting neural networks against adversarial attacks. [Background technology]

[0002] Machine learning technology is a key component of modern artificial intelligence (AI) services and has shown great success in providing human-level capabilities for a variety of tasks, including image recognition, speech recognition, and natural language processing. Many major technology companies build their AI products and services using deep learning models (e.g., deep neural networks (DNNs)) as a key component. Building a production-grade deep learning model is a nontrivial task, requiring massive training data, powerful computing resources, and specialized skills. For example, creating a convolutional neural network (CNN) for image classification can take days to weeks on multiple GPUs using an image dataset containing millions of images. In addition, designing a deep learning model requires significant machine learning expertise and a lot of trial-and-error to define the model architecture and select model hyperparameters.

[0003] Deep learning has been shown to be effective in a variety of real-world applications, including computer vision, natural language processing, and speech recognition. It has also shown great potential in medical informatics, such as medical diagnosis and adjustment decisions, including aiding in representation learning of patient records, disease phenotyping, and making predictions. However, recent research has shown that these models are vulnerable to adversarial attacks, which are designed to intentionally insert small perturbations (also known as "adversarial examples") into the model's data input, causing misclassification. In image classification, researchers have shown that subtle changes in the input can fool classifiers. In the text domain, synonym substitutions or character- or word-level modifications to a few words can also cause models to misclassify. These perturbations, while barely perceptible to humans, can easily fool high-performance deep learning models.

[0004] Since the discovery of adversarial examples, numerous techniques have been proposed to defend neural networks against them, but such techniques are either expensive or unreliable. To date, only data augmentation has been shown to be effective. First, data augmentation defenses and prejudicial training involve iteratively generating adversarial examples during training and adding these samples incrementally as training inputs. Integrating adversarial examples during model training is effective in defending neural networks against attacks that use the generation of adversarial training examples, but the generation process is computationally expensive and is only effective for the specific type of adversarial examples used during training.

[0005] Another extension defense worth mentioning is randomized smoothing, an approach that augments the training dataset with a noisy version of the dataset created using Gaussian noise. This approach is premised on the idea that models with inhibition robustness also tend to be naturally robust to noisy inputs. While randomized smoothing mitigates the computational overhead introduced by adversarial training (because no adversarial examples need to be generated), this approach has the disadvantage of making the final model relatively weak.

[0006] In addition to these data augmentation defenses, numerous other methods have been proposed to improve the inhibition robustness of neural networks, for example, through the use of preprocessing techniques or the addition of new types of network layers. Many of these techniques rely on gradient shattering or some modification of it. Specifically, traditional white-box adversarial attacks use the network's loss gradient to generate adversarial examples. Gradient shattering typically defeats these attacks by causing the loss gradient to vanish through non-differential operations. Previous studies have shown that gradient shattering is an ineffective defense because adversarial attacks can be adapted, for example, to skip non-differential operations or to use coarse approximations of differential operations.

[0007] Therefore, there remains a need in the art to provide techniques for ensuring the reliability and robustness of neural network classifiers in the face of adversarial examples. Summary of the Invention [Problem to be solved by the invention]

[0008] The present invention aims to provide a constrained neural network with alternative encodings for robustness. [Means for solving the problem]

[0009] The techniques herein provide for generating and operating inhibition-robust neural networks. Neural networks configured as inhibition-robust typically include a first layer, i.e., an input layer, a final layer, i.e., an output layer, and one or more intermediate layers that can be concealed. In this approach, the network is extended to include one or more additional layers configured with an encoding scheme that provides an alternative to traditional one-hot encoding for classification. Preferably, the alternative encoding is a vector-based encoding, in which a particular classification label is represented by a vector of 0s (numeric zeros) and 1s (numeric ones). In one exemplary embodiment, at least one additional layer of this type is positioned between the intermediate layer and the final layer, and the additional layer has a lower dimensionality (i.e., fewer neurons) than the one or more intermediate layers. The alternative layer is preferably a fully connected layer and is sized to include a sufficient number of neurons to uniquely label the network's output classification (using the encoding). Thus, for example, if the neural network is a classifier with more than 10 output classes, the substitution layer is sized to contain at least 4 neurons, in this example 4 bits (≥ 10) to represent the encoding of the 10 output classes. 1 / 2 ) position. After including additional layers and determining their size (for a given number of output classifications), encoding is applied to the weights within the layers. The weights comprise a weight matrix, where each column of the matrix is ​​the i-th encoding vector for the i-th output classification. This alternative encoding is sometimes referred to herein as "n-hot" encoding to distinguish it from traditional one-hot encoding, where the value of "n" is configured as a hyperparameter.

[0010] Neural network classifiers extended to include these additional layers are typically trained in a conventional manner. Based on this training, encoding involves enforcing the network with a reduced number of activation features, each of which has a hot position (i.e., 1s in the weight matrix), thereby encouraging the network to learn features that contain a high degree of information about each classification. The additional layers thus constrain the network and add constraints between these features and the output classification, particularly on the number of features and / or layers used to predict each classification. Once trained in this manner, the inhibition-robust neural network is then applied to classification tasks.

[0011] The foregoing has outlined some of the more relevant features of the present subject matter. These features should be construed as merely exemplary. Many other valuable results can be obtained by applying or modifying the present subject matter in different ways, as will be explained hereinafter.

[0012] For a more complete understanding of the present subject matter and its advantages, reference is now made to the following descriptions taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0013] [Figure 1] Figure 1 shows a typical deep learning model (DNN). [Figure 2] Figure 2 shows a deep learning model used in conjunction with a deployment system or application. [Figure 3] FIG. 3 illustrates an extension of the neural network of FIG. 1 to include an n-hot encoding layer in accordance with the present disclosure. [Figure 4] Figure 4 shows a representative process flow for constructing and applying n-hot encoding used in additional layers of a neural network according to one embodiment. [Figure 5] FIG. 5 is a block diagram of a data processing system in which exemplary aspects of the exemplary embodiments may be implemented. [Figure 6] FIG. 6 is a diagram of a computer-readable recording medium according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] As described below, the techniques herein improve the robustness of neural networks against adversarial attacks. For background purposes, the following describes the basic principles of deep learning.

[0015] As is well known, deep learning is a form of machine learning framework that automatically learns hierarchical data representations from training data without requiring manually generated feature representations. Deep learning methods are based on architectures called deep natural networks (DNNs), which consist of a large number of basic neural network units, such as linear perceptrons, convolutions, and nonlinear activation functions. These network units are organized into layers (ranging from a few to thousands or more), which are trained directly from coarse data to recognize complex concepts. Lower network layers often correspond to low-level features (e.g., image corners and edges in image recognition), while higher-level layers typically correspond to semantically meaningful features.

[0016] In particular, deep neural networks (DNNs) take a coarse training data representation as input and map it through a parameter function to produce an output. The parameter function is determined by both the network architecture and the collective parameters of all neural network units used within the network architecture. Each network unit receives an input vector from its connected neurons and outputs a value to pass to subsequent layers. For example, a linear unit outputs the dot product between its weight parameter and the output value of its connected neuron from the previous layer. To increase the ability of DNNs to model complex structures in the training data, different types of network units have been developed, such as nonlinear activation units (e.g., hyperbolic tangent, sigmoid, rectified linear unit), max pooling, and batch normalization, which are used in combination with linear activation functions. When the goal of a neural network is to classify data into a finite set of classes, the activation function in the output layer is typically a softmax function, which can be viewed as the predicted classification distribution of the class set.

[0017] The first step before training the network weights for a DNN is to determine the model architecture, which often requires nontrivial domain expertise and technical effort. For a given network architecture, the network behavior is determined by the values ​​of the network parameters. More formally, the training data is given by Equation (1):

[0018]

number

[0019] In the above formula (1),

[0020]

number

[0021]

number

[0022] Formally, a DNN is notionally described by the following function (3):

[0023]

number

[0024] In the above formula (4), X is the input space and Y is the output space representing the categorized set. A sample x, which is an element of X, is defined as follows:

[0025]

number

[0026] f i represent layers, and F Lis the final output layer. The final output layer generates a mapping from the hidden space to the output space (classification labels) by a softmax function that sums up to the number 1 and outputs a real vector in the range [0,1]. The output of the softmax function is a probability distribution of the input x over the different possible output classifications C.

[0027] FIG. 1 illustrates a DNN 100, sometimes referred to as an artificial neural network. As illustrated, the DNN 100 is an interconnected group of nodes (neurons), where each node 103 represents an artificial neuron, and lines 105 represent connections from the output of one artificial neuron to the input of another. In a DNN, the output of each neuron is calculated by some nonlinear function on the sum of its input values. The connections between neurons are known as edges. Neurons and edges typically have weights that regulate the learning process. The weights increase or decrease the strength of the signal in the connection. As illustrated, in DNNs, neurons are typically organized into layers, and different layers can perform different transformations on their input values. As illustrated, multiple signals (typically real-valued) travel from the first layer (input layer) 102 to the last layer (output layer) 106, traversing one or more intermediate (hidden) layers 104. The hidden layer 104 provides the ability to extract features from the input layer 102. As shown in FIG. 1, there are two hidden layers, but this is not a limitation. Typically, the number of hidden layers (and the number of neurons in each layer) is a function of the problem the network is to address. A network with too many neurons in the hidden layer will overfit, thereby limiting the network's ability to memorize and integrate input patterns. On the other hand, if there are too few neurons in the hidden layer(s), the network will be unable to represent features in the input space, which in turn limits the network's ability to integrate. In general, the smaller the network (fewer neurons and weights), the better the network will be.

[0028] The DNN100 is trained using a training dataset, resulting in the generation of a set of weights corresponding to the trained DNN. Formally, the training set is (x i ,y i) and contains N labeled inputs, with the i-th input being described by x i ,y i ) is fed through the network. The output of the network is the predicted value g(x i ) to train the DNN. i ) and its true value label, i is given by the loss function in equation (6) below, which inversely propagates the error through the network and updates the model parameters.

[0029]

number

[0030] Typically, neural network models, such as those shown in Figure 1, receive numerical values ​​as input. To work with categorical data, such data typically needs to be encoded in some way. One-hot encoding is a known technique for converting categorical data into integers or vectors of 1s and 0s. In this approach, the vector length depends on the expected number of classifications or categories, and each element in the vector represents a category. In one-hot encoding, the number 1 is used to indicate a category, and all other values ​​in the vector are zero. In other words, if the categorical data does not indicate order, one-hot encoding thus provides a convenient way to work with the above data. Label encoding converts categorical variables into a machine-readable numerical representation.

[0031] Figure 2 shows a DNN 200 deployed as a front end to a deployment system, application, or task 202. The deployment system can be any form of machine learning used to support decision making. As mentioned above, the described neural network is susceptible to adversarial attacks, which are designed to intentionally inject small perturbations ("adversarial examples") into the model's data inputs, resulting in misclassification. In image classification, researchers have shown that imperceptible changes in the input can mislead classifiers. In the text domain, synonym substitutions or character / word-level modifications to a few words can also cause models to misclassify. These perturbations are barely perceptible to humans, yet can easily fool high-performance deep learning models. For illustrative purposes, assume that the above-described DNN 100 (Figure 1) or DNN 200 (Figure 2) is subjected to an adversarial attack. The techniques of this disclosure are then used to improve the robustness of the network. The resulting networks are hereafter described as inhibition-robust, meaning that they are better able to perform the required classification tasks even in the face of adversarial examples than networks that do not incorporate the techniques of this disclosure.

[0032] The particular neural network, its classification properties, particular deployment system, or strategy, or combination thereof, is not a limitation of the technology herein and may be used to enhance any type of network classifier, regardless of its structure and use.

[0033] With the above as the background, the techniques of the present disclosure will now be described.

[0034] Constrained Neural Networks with Alternative Encodings for Robustness Referring to FIG. 3 , a standard DNN 300, shown on the left, is transformed into an inhibition-robust neural network 302, shown on the right. In this example, the standard DNN 300 includes an input layer 304, hidden (intermediate) layers (L1, L2, and L3) 306, and an output layer 305. The inhibition-robust neural network 302 similarly includes an input layer 310, an output layer 312, and a hidden layer 314. Standard training produces network 300, which is composed of robust and non-robust features. However, network 300 is not inhibition-robust. To address this deficiency, the techniques of this disclosure extend DNN 300 to produce inhibition-robust network 302. As shown, the difference between networks 300 and 302 is the inclusion of an additional layer 316, which has reduced dimensionality compared to the other intermediate layers. Here, the notion of dimensionality refers in its usual way, i.e., to the number of input variables or features for a particular dataset (in this case, an additional layer). As shown in FIG. 3, the additional layer 316 is positioned concomitantly with the output layer 312, in this case, positioned to the left of the final hidden layer L3. This placement is not limiting, and the additional layer 316 could also be positioned between the final hidden layer L3 and the output layer 312. In general, the additional layer 316 will hereafter be referred to as being positioned at or near the output layer 312 of the neural network. As used herein, the notion of having reduced dimensionality means that the additional layer has a smaller number of neurons compared to at least the final hidden layer to which it is connected. The dimensionality of the additional layer itself varies based on the size of the hidden layer(s), and preferably, the additional layer has a significantly lower dimensionality than one or more (or at least adjacent) hidden layers. As will be explained hereinafter, the additional layer 316 is preferably a fully connected layer. Furthermore, although only one additional layer 316 is shown in FIG. 3, there may be one or more similar additional layers.In one non-limiting embodiment, a fully connected layer 316 is added before the logit feature layer (e.g., L3) (or equivalently after the penultimate layer). In this embodiment, the fully connected layer has a sigmoid activation, although other activations (e.g., tanh) can also be used.

[0035] As shown, additional layer 316, sometimes referred to herein as "n-hot," implements an n-hot encoding scheme, which, as will be explained further below, is an alternative to traditional one-hot encoding for classification. The number of neurons in n-hot encoding layer 316 is preferably less than the number of neurons in the immediately preceding layer, but the number of neurons is not less than the minimum number required to represent the number of output categories of the classifier. For example, if a classifier has 10 output categories, n-hot layer 316 should have at least 4 neurons (≥ 10) to uniquely label the categories. 1 / 2 ) The optimal size of the n-hot layer can be determined during training, e.g., experimentally, or pre-configured sizing can be used to enforce the constraints mentioned above. After adding an n-hot encoding layer 316 to the model and determining the layer size, an n-hot encoding, preferably a vector of 0s and 1s, is applied to the layer weights. This encoding can be manually configured, randomly generated, or deterministically provided through other means, e.g., domain knowledge, an ontology, or a knowledge graph.

[0036] Figure 4 shows the process flow of one approach to randomly generating encodings in this embodiment. In step 400, the layer size and the number of output classifications are specified. Then, in step 402, n-hot vectors with a number of hot bits (i.e., the number of 1s) greater than or equal to 0 are randomly generated for each output classification. In step 404, these random n-hot vectors are introduced into the network as fixed weights for the connections between the added n-hot layers and the final output layer. More formally, the output of the n-hot encoding layer (where the value n can be set as a hyperparameter) is expressed as follows:

[0037]

number

[0038] In the above equation (7), x is the input to the layer, W is a weight matrix, and the columns of W are as shown in the following equation (8).

[0039]

number

[0040] Here, E in the above formula (8) iis the ith random n-hot encoding vector for output classification i. In machine learning, a hyperparameter is a parameter whose value is used to control the learning process. As mentioned above, this encoding is sometimes referred to herein as n-hot encoding to distinguish it from traditional 1-hot encoding, and the value "n" is preferably configured or designed as a hyperparameter. The neural network extended to include this additional layer is then trained, step 406. Based on the training, this encoding involves reducing the number of activation features for each hot position (i.e., multiple 1's in the weight matrix), thereby encouraging the network to learn these features that contain more information about each classification. Thus, the additional layer specifically limits (constrains) the network on the number of features used to predict each classification, or the layer imposes constraints on the relationship between these features and the output classification, or both. Once trained by this method, the resulting inhibition-robust neural network is then output in step 405. In step 410, i.e., after training, the inhibition-robust neural network is applied to a classification task. As noted above, the nature of the classification task performed by an inhibition-robust network classifier varies and typically depends on the particular deployment or deployment objectives.

[0041] Thus, according to the present disclosure, one or more layers are added to a neural network and provide an alternative encoding to traditional one-hot encoding for classification. As described above, n-hot encoding applied to the additional layers is used during training and refines it to associate multiple features (feature sets) with each hot position. In the embodiment shown in FIG. 4, the encodings are randomly generated, and a random n-hot vector is generated for each output classification. As the network trains (typically in the conventional manner), the fixed encoding weights are preferably left unchanged. The output of the n-hot encoding layer is then expressed as Equation (7) below, as described above.

[0042]

number

[0043] As mentioned above, the specific placement of the alternative encoding layers can vary. Typically, it is preferable to place the alternative encoding layer before the penultimate layer to minimize noise. However, technically, this layer can be placed anywhere before the last layer. Regardless of the specific placement, the low-dimensional layer acts to discover relevant features. In particular, by constraining the intermediate layers to be low (dimensionally) relative to the surrounding layers, the model must effectively perform the compression and decompression (i.e., reconstruction) steps near the low-dimensional layers. Therefore, to maximize the accuracy of feature reconstruction, the features learned in the low-dimensional layers are those that contain the most information relevant to each output classification. In the described n-hot encoding framework, it is possible to use either a predefined encoding or random encoding (the default). If the encoding is fixed at that point and a classification is given as input, the network must identify which features in the layer before the n-hot layer should be combined to obtain the fixed encoding. This type of operation is easier to understand when using predefined encodings instead of random encodings (e.g., digital clock encoding for digits), but the benefits can be obtained in both scenarios. In particular, and throughout training, the network extracts and learns prescribed features in the n-hot layers, rather than the usual training method of learning a set of relevant features in a purely random manner.

[0044] The technique described above has significant advantages. It is significantly more computationally efficient than traditional data augmentation techniques, and the approach does not rely on the use of non-differential structures to hide the loss gradient from adversarial examples. The technique is easy to implement, the additional layers have significantly lower dimensionality than other network layers, and the encoding scheme ensures that the network quickly and reliably learns a reduced number of activation features. Furthermore, the approach does not require any modifications to existing training.

[0045] Another implementation of the above technique is to solve the problem as multi-task learning, training a classifier to classify inputs using multiple classification sets. An exemplary solution to multi-task learning involves training the above model with a weighted sum of multiple loss functions considering multiple sets of classifications, including the original classification set and auxiliary / feature sets of classifications. This approach promotes robust and semantically meaningful learning of input encodings and promotes models that use generalizable encodings. For example, consider an image classifier that classifies a given image as one of {bird, aircraft, dog}. Then, instead of training the classifier to classify inputs into exactly one of these three classes, the approach can also consider other sets, such as {feather, featherless} and {animal, plant, inanimate}.

[0046] The above-described variants introduce constraints on the encoding shared by the loss functions, but each loss function is calculated independently. Instead of using each loss function independently, another approach is to first use multi-task learning, then build a hyperclassifier on the input vertices of the original classification set, and train a neural network to classify the inputs into feature classes. The hyperclassifier is then constrained to use only the extracted features; the hyperclassifier is then trained or a manual mapping assigned to the hyperclassifier. More generally, this approach involves training a classifier to map inputs to features and then training a new classifier to map features to output classes. This provides an end-to-end classification pipeline. In any of these variants, the auxiliary / feature set of classes or the mapping from such features to the original target classes is learned using training data, manual generation, or extraction from an ontology or knowledge graph.

[0047] The techniques described herein can be implemented as architectural modifications alone or in combination with other existing adversarial defense methods such as data augmentation (e.g., adversarial training, Gaussian smoothing).

[0048] One or more aspects of the present disclosure (e.g., extending the NN, testing the adversarial examples, etc.) may be implemented as a service by a third party, for example. The present subject matter may be implemented in or in combination with a data center that provides cloud-based computing, data storage, or related services.

[0049] In a typical use case, a SIEM or other security system issues API queries to the trained model and has an accompanying interface that is used to receive responses to those queries, including response indicators of adversarial input.

[0050] The approach herein is designed to be implemented on-demand or in an automated manner.

[0051] Access to model training or use to identify adversarial inputs can be accomplished via any suitable request-response protocol or workflow, with or without an API.

[0052] 5 illustrates an exemplary distributed data processing system in which a distributed system or any other computing task involving the techniques herein may be implemented. Data processing system 500 is an example of a computer in which computer-usable program code or instructions for implementing the present processes may be located, as an exemplary embodiment. In this exemplary embodiment, data processing system 500 includes communications functionality 502, which provides communications between a processor unit 504, memory 506, persistent storage 505, communications unit 510, input / output (I / O) unit 512, and display 514.

[0053] Processor unit 504 provides instruction execution for software that may be loaded into memory 506. Processor unit 504 may be a set of one or more processors or may be a multi-processor core, depending on the particular implementation. Additionally, processor unit 504 may be implemented using one or more heterogeneous processor systems, in which a main processor exists along with secondary processors on a single chip. In another illustrative embodiment, processor unit 504 may be a symmetric multi-processor (SMP) system, including multiple processors of the same type.

[0054] Memory 506 and persistent storage 505 are examples of storage devices. A storage device is any hardware component capable of storing information on a temporary basis, a persistent basis, or both. Memory 506, in these examples, may be, for example, random access memory or any other volatile or non-volatile storage device. Persistent storage 505 may take various forms, depending on the particular embodiment. For example, persistent storage 505 may be a hard drive, flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination thereof. The media used by persistent storage 505 may also be removable. For example, a removable hard drive may be used as persistent storage 505.

[0055] Communications unit 510, in these examples, provides for communications with other data processing systems or devices. In these examples, communications unit 510 is a network interface card. Communications unit 510 provides communications through the use of either or both physical and wireless communications links.

[0056] Input / output unit 512 allows for the input and output of data to and from other devices that may be connected to data processing system 500. For example, input / output unit 512 may provide a connection for user input through a keyboard and a mouse. Additionally, input / output unit 512 may send output to a printer. Display 514 provides the ability to display information to a user.

[0057] Instructions for the operating system and applications or programs are located on persistent storage 505. These instructions are loaded into memory 506 and executed by processor unit 504. The processes of the different embodiments may be performed by processor unit 504 using computer-implemented instructions that may be located in a memory, such as memory 506. These instructions are referred to as program code, computer usable program code, or computer readable program code that may be read and executed by a processor in processor unit 504. The program code in the different embodiments may be embodied on different physical or tangible computer-readable media, such as memory 506 or persistent storage 505.

[0058] Program code 516 may be located in the functional form of computer-readable medium 515, which may be selectively removable and may be loaded onto or transferred to data processing system 500 for execution by processor unit 504. Program code 516 and computer-readable medium 515 form a computer program product and a computer-readable recording medium in these examples. In one embodiment, computer-readable medium 515 may be in the form of a tangible optical or magnetic disk that is inserted into or placed in a drive or other device that is part of persistent storage 505 for transfer to a storage device, such as a hard drive that is part of persistent storage 505. In a tangible form, computer-readable medium 515 may also take the form of persistent storage, such as a hard drive, thumb drive, or flash memory that is connected to data processing system 500. The tangible form of computer-readable medium 515 may also be referred to as a computer-recordable recording medium. In some embodiments, computer-recordable medium 515 is not removable.

[0059] Alternatively, program code 516 can be transferred to data processing system 500 from computer-readable medium 515 through a communications link to communications unit 510 or through input / output unit 512. The communications link or connection, or both, in an exemplary embodiment, can be physical or wireless. Computer-readable media also can take the form of non-tangible media, such as a communications link or wireless connection containing program code. The different components illustrated for data processing system 500 are not meant to provide architectural limitations to the manner in which different embodiments may be implemented. Different exemplary embodiments may be implemented in data processing systems including components in addition to or instead of those illustrated for data processing system 500. Components other than those illustrated in FIG. 5 can differ from the illustrated exemplary embodiment. As an example, storage devices in data processing system 500 can be any hardware device capable of storing data. Memory 506, persistent storage 505, and computer-readable medium 515 are examples of storage devices in a tangible form.

[0060] In another embodiment, a bus system may be used to implement communications function 502 and may be comprised of one or more buses, such as a system bus or an input / output bus. Of course, the bus system may be implemented using any suitable type of architecture that provides for a transfer of data between any components or devices attached to the bus system. Additionally, a communications unit may include one or more devices used for transmitting and receiving data, such as a modem or a network adapter. Further, a memory may be, for example, memory 506 or a cache, such as found in an interface and memory controller hub that may be present in communications function 502.

[0061] The technology herein can be used with a host machine (or a set of machines, e.g., operating as a cluster) operating in a standalone manner or in a networked environment, such as a cloud computing environment. Cloud computing is an information technology (IT) delivery model whereby resources, software, and information are provided on-demand to computers and other devices over the Internet. Using this approach, application instances are hosted and made available from Internet-based resources accessible over HTTP through traditional web browsers or mobile applications. Cloud computing resources are typically housed within large server farms running one or more networked applications using a virtualization architecture, where multiple applications run within virtual servers or so-called "virtual machines" (VMs) that are mapped onto physical servers within a data center facility. Virtual machines typically run on top of a hypervisor, which is a control program that allocates physical resources to virtual machines.

[0062] A typical cloud computing model is as follows:

[0063] Software as a Service (SaaS): The functionality offered to the consumer is the use of the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through a thin-client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or individual application functionality, except for limited user-specific application configuration settings.

[0064] Platform as a Service (PaaS): The capability offered to consumers is to deploy applications they create or acquire, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does control the deployed applications and, possibly, the configuration of the application hosting environment.

[0065] Infrastructure as a Service (IaaS): The functionality provided to the consumer is the provision of processing, storage, network, and other basic computing resources on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and possibly limited control over select networking components (e.g., host firewalls).

[0066] The deployment model is as follows:

[0067] Private Cloud: Cloud infrastructure operates solely for one organization. It can be managed by that organization or a third party and can exist on or off-premises.

[0068] Community Cloud: Cloud infrastructure is shared by several organizations to support a specific community with common interests (e.g., mission, security requirements, policy, and compliance considerations). It can be managed by those organizations or a third party and can reside on or off premises.

[0069] Public Cloud: Cloud infrastructure is made available to the public or large industry groups and is owned by organizations that sell cloud services.

[0070] Hybrid Cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technologies that allow for data and application portability (e.g., cloud bursting for load balancing between clouds).

[0071] Some clouds are based on traditional IP networks. For example, a cloud may be based on a special single-layer IP routing CLOS-based network that uses MAC address hashing.

[0072] The system, and in particular the modeling and consistency checking components, are typically each software, i.e., a set of computer program instructions executing within one or more hardware processors. Components may also be integrated with each other, in whole or in part. A component may have multiple subcomponents that execute together. One or more components may have multiple subcomponents that execute together to provide functionality. There is no requirement that a particular function performed by a particular component, as designated above, functionality (or any aspect thereof) herein, can be implemented in other or multiple systems.

[0073] This approach can be implemented by any service provider operating the infrastructure. It can be available as a managed service, for example, offered by cloud services. A representative deep learning architecture of this type is IBM® Watson® Studio.

[0074] Components implement workflows that are synchronous or asynchronous, continuous or periodic, or a combination thereof.

[0075] This approach can be integrated into other enterprise or network-based security methods and systems, such as SIEM, APT, and graph-based cybersecurity analytics.

[0076] Computer-readable program instructions for carrying out the operations of the present invention can be written in any combination of programming languages, including one or more object-oriented programming languages ​​such as Java®, Smalltalk, C++, etc., procedural programming languages ​​such as the "C" programming language, or similar programming languages. The computer-readable program instructions can execute entirely on the user computer, partially on the user computer as a stand-alone software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user computer through any type of network, including a local area network (LAN), a wide area network (WAN), or the connection can be to an external computer (e.g., through an Internet service provider).

[0077] Those skilled in the art will appreciate that the hardware in Figure 5 varies depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disk drives, may be used in addition to or in place of the hardware depicted. Additionally, the processes of the illustrative embodiments may be applied to multiprocessor data processing systems other than the SMP systems described above without departing from the spirit and scope of the disclosed subject matter.

[0078] The functionality described in this disclosure may be implemented in whole or in part as a standalone approach, e.g., software-based functionality executed by a hardware processor, or may be available as a managed service (including a web service via a SOAP / XML interface). Specific hardware and software implementation details described herein are for illustrative purposes only and are not meant to limit the scope of the described subject matter.

[0079] More generally, computing devices within the context of the described subject matter are data processing systems (such as those shown in Figure 5) each including hardware and software, and these entities communicate with each other over a network, such as the Internet, an intranet, an extranet, a private network, or any other communications medium or link.

[0080] The schemes described herein can be implemented in or in combination with a variety of server-side architectures, including simple n-tire architectures, web portals, federated systems, etc. The techniques herein can be practiced in loosely coupled server (including "cloud" based) environments.

[0081] More generally, the subject matter described herein can take the form of an entirely hardware environment, an entirely software environment, or an environment containing both hardware and software elements. In a preferred embodiment, the functionality is implemented in software, including, but not limited to, firmware, embedded software, and microcode. Furthermore, as described above, the identity-context-based access control functionality can take the form of a computer program accessible from a computer-usable or computer-readable medium that provides program code for use by a computer or any instruction execution system or combination thereof. For purposes of this description, a computer-usable or computer-accelerated medium can be any apparatus that contains or can store a program for use by, or in combination with, an instruction execution system, apparatus, or device. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device). Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), rigid magnetic disks, and optical disks. Current examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read / write (CD-R / W), and DVD. A computer-readable medium is a tangible article.

[0082] In an exemplary embodiment, the techniques described herein are implemented in software executed by a special-purpose computer, preferably one or more processors. The software is stored in one or more data stores or memories associated with the one or more processors, and the software can be implemented as one or more computer programs. Collectively, this special-purpose hardware and software includes the functionality described above.

[0083] While the above describes a particular order of operations performed by certain embodiments, such order is exemplary and alternative embodiments may perform operations in a different order, combine certain operations, overlap certain operations, etc. Reference in the specification to certain embodiments indicates that the described embodiments may include particular features, structures, or characteristics, but it is not necessary for every embodiment to include a particular feature, structure, or characteristic.

[0084] Finally, while certain components of the present system have been described separately, those skilled in the art will recognize that some of the functionality may be combined or shared within a given instruction, program sequence, code portion, execution thread, etc.

[0085] The technology herein provides improvements to other technologies or technical fields, such as, for example, without limitation, deep learning systems, real-world applications of deep learning models, medical classification, other security systems, as well as location systems that use functionality to direct and control operations in connection with location systems using deep learning models.

[0086] As mentioned above, the techniques herein can be used in any domain and with any application where neural network classifiers may be subject to adversarial attacks.

[0087] The techniques herein are not limited to use with any particular form of deep learning model. The approach can be extended to any machine learning model that has internal processing state (i.e., hidden weights), including, but not limited to, support vector machines (SVMs), logical regression (LR) models, etc. The approach can also be extended for use with decision tree-based models.

[0088] The particular classification tasks that can be implemented are not intended to be limiting. Exemplary classification tasks include, but are not limited to, image classification, text recognition, speech recognition, natural language processing, and many others.

[0089] Having described the present subject matter, the following claims are set forth: [Explanation of symbols]

[0090] 300: Network 302: Network 304: Input layer 305: Layer 310: Input layer 312: Output layer 314: Hidden Layer 316: Additional layer (fully connected layer)

Claims

1. 1. A computer-implemented method for constraining and manipulating a neural network to improve its robustness against adversarial attacks, the neural network including an input layer, an output layer, and one or more hidden layers, the computer-implemented method comprising: The output layer is accompanied by an additional fully connected layer of lower dimension than at least one intermediate layer; applying, in the additional layer, encoding with a plurality of encoding vectors associated with a plurality of features for each of the hot positions for each output classification; training the neural network with the additional layers and encoding applied to learn a reduced feature set representing one or more features that are highly information-bearing relative to at least one output classification; performing robust classification on adversarial examples using the trained neural network; 20. A computer-implemented method comprising:

2. 2. The computer-implemented method of claim 1, wherein the encoding is a vector bit encoding comprising a set of bit vectors, wherein an ith encoding vector of a set of bit vectors represents an ith output classification of the neural network.

3. moreover, Associating a fixed set of encoding weights with the connections between the additional layers and the output layer; and maintaining the fixed encoding weights unchanged through the training; 3. A computer-implemented method according to claim 1 or 2, comprising:

4. The computer-implemented method of any one of claims 1 to 3, wherein the training also adds one or more constraints between the one or more features and the output layer.

5. 5. The computer-implemented method of claim 1, wherein the additional layer is fully connected using sigmoid activation and is placed before a logit function layer of the neural network.

6. The computer-implemented method of any one of claims 2 to 5, wherein the training associates one or more features for at least the output classification with specific bit positions of the encoding vector.

7. 7. The computer-implemented method of claim 1, further comprising sizing the additional layers to include a sufficient number of neurons to encode a unique label for each output classification of the neural network.

8. The computer-implemented method of any one of claims 1 to 7, wherein the training uses a weighted sum of multiple loss functions over a set of classifications.

9. a processor; a computer memory holding computer program instructions executed by said processor for constraining and manipulating a neural network to improve its robustness against adversarial attacks; An apparatus comprising: The output layer is accompanied by an additional fully connected layer of lower dimension than at least one intermediate layer; applying, in the additional layer, encoding with a plurality of encoding vectors associated with a plurality of features for each of the hot positions for each output classification; training the neural network with the additional layers and encoding applied to learn a reduced feature set representing one or more features that are highly information-bearing relative to at least one output classification; performing robust classification on adversarial examples using the trained neural network; A device that performs the above.

10. 10. The apparatus of claim 9, wherein the encoding is a vector bit encoding comprising a set of bit vectors, wherein an ith encoding vector of a set of bit vectors represents an ith output classification of the neural network.

11. moreover, Associating a fixed set of encoding weights with the connections between the additional layers and the output layer; and maintaining the fixed encoding weights unchanged through the training; 11. The apparatus of claim 9 or 10, comprising:

12. The apparatus of any one of claims 9 to 11, wherein the training also adds one or more constraints between the one or more features and the output layer.

13. The apparatus according to any one of claims 9 to 12, wherein the additional layer is fully connected using sigmoid activation and is placed before a logit function layer of the neural network.

14. The apparatus of any one of claims 10 to 13, wherein the training involves associating specific bit positions of the encoding vector with one or more features for at least the output classification.

15. 15. The apparatus of claim 9, further comprising sizing the additional layers to include a sufficient number of neurons to encode a unique label for each output classification of the neural network.

16. The apparatus of any one of claims 9 to 15, wherein the training uses a weighted sum of multiple loss functions over a set of classifications.

17. 1. A computer program for constraining and manipulating a neural network to improve the robustness of a data processing system against adversarial attacks, the neural network including an input layer, an output layer, and one or more hidden layers, the computer program comprising: The output layer is accompanied by an additional fully connected layer of lower dimension than at least one intermediate layer; applying, in the additional layer, encoding with a plurality of encoding vectors associated with a plurality of features for each of the hot positions for each output classification; training the neural network with the additional layers and encoding applied to learn a reduced feature set representing one or more features that are highly information-bearing relative to at least one output classification; performing robust classification on adversarial examples using the trained neural network; A computer program that executes the following:

18. 18. The computer program product of claim 17, wherein the encoding is a vector bit encoding comprising a set of bit vectors, wherein an ith encoding vector of a set of bit vectors represents an ith output classification of the neural network.

19. moreover, Associating a fixed set of encoding weights with the connections between the additional layers and the output layer; and maintaining the fixed encoding weights unchanged through the training; 19. A computer program according to claim 17 or 18, comprising:

20. The computer program product of any one of claims 17 to 19, wherein the training also adds one or more constraints between the one or more features and the output layer.

21. 21. The computer program product of claim 17, wherein the additional layer is fully connected using sigmoid activation and is placed before a logit function layer of the neural network.

22. 22. The computer program of claim 17, wherein the training involves associating specific bit positions of the encoding vector with one or more features for at least the output classification.

23. 23. The computer program product of claim 18, further comprising sizing the additional layers to include a sufficient number of neurons to encode a unique label for each output classification of the neural network.

24. 24. The computer program of claim 17, wherein the training uses a weighted sum of multiple loss functions over a set of classifications.

25. A computer-readable recording medium having recorded thereon the computer-executable computer program according to any one of claims 17 to 24.

Images