Systems and methods for explainable anomaly detection

Abstract

The present disclosure provides an anomaly detector comprising: an input interface configured to accept input data; a first neural network having an autoencoder architecture including an encoder trained to encode the input data and a decoder trained to decode the encoded input data and reconstruct the input data; a loss estimator configured to compare portions of the input data with corresponding portions of the reconstructed input data to determine a sequence of losses for different components of the reconstruction error; a second neural network trained in a supervised manner to classify the sequence of losses to detect anomalies and generate an anomaly detection result including one or a combination of an anomaly type and an anomaly severity; and an output interface configured to render the anomaly detection result.

Description

[Technical Field]

[0001] The present disclosure relates generally to anomaly detection, and more particularly to a system and method for explainable anomaly detection using neural network architectures. [Background technology]

[0002] Anomaly detection generally involves the task of detecting abnormal situations. This task can be widely applied in various applications such as security, safety, quality control, fault monitoring, process control, etc. The goal of anomaly detection in various applications is typically to raise an alert about unusual situations that require further investigation and possibly response action to mitigate any adverse issues. Human investigation and response can be costly due to the large amount of information flow. Therefore, it is desirable for an anomaly detector to provide information that helps explain the reason for a detected anomaly to guide the investigation and response action.

[0003] Typically, anomaly detectors use an autoencoder to reconstruct input data and use the reconstruction loss between the input data and the reconstructed data to detect anomalies in the input data. One of the major problems in training anomaly detectors is the lack of training data with anomalies. Therefore, autoencoders configured to detect anomalies are currently trained on normal, non-anomalous data. The concept of training an autoencoder only on normal, non-anomalous data is called one-class learning, which models the data distribution of "normal" (non-anomalous) data samples. In one-class learning, the reconstruction loss of the autoencoder serves as an indicator of whether the input data examples (given at test time) are anomalous. For example, if the reconstruction loss is greater than a threshold, the input data is inferred to be anomalous. An autoencoder trained according to one-class learning is called a one-class classifier.

[0004] However, one-class classifiers may not be able to adequately handle the rich context of real-world practical applications, where the input data used contains multiple, potentially heterogeneous features. Therefore, there is a need for an anomaly detector that analyzes each feature of the input data to detect anomalies and / or provides information explaining the reasons for detected anomalies. Summary of the Invention

[0005] It is an objective of some embodiments to provide an anomaly detector that not only detects anomalies but also explains the detected anomalies, e.g., determines the type of anomaly and the severity of the anomaly. It is also an objective of some embodiments to provide a multi-stage training method for training a neural network for anomaly detection. Additionally or alternatively, it is an objective of some embodiments to adapt a neural network trained on data from one domain to another domain.

[0006] The anomaly detector uses an autoencoder, which includes an encoder and a decoder, for anomaly detection. The autoencoder can be trained with data samples for anomaly detection. Some embodiments are based on the recognition that, in practice, a large set of normal data samples may be available, but a sufficient number of abnormal data samples may not be available because it is difficult to enumerate all possible types of abnormal samples. Therefore, anomaly detection is an unsupervised learning problem that does not rely on prior information; in other words, it is assumed that all data samples are unlabeled.

[0007] Therefore, during the first training stage of the multi-stage training, the autoencoder is trained in an unsupervised manner using unlabeled data samples. In particular, the autoencoder is trained on the unlabeled data samples to learn their representations and then uses these representations to reconstruct the unlabeled data samples. In one embodiment, the unlabeled data samples are structured proxy data samples that include a mixture of different types of features, including, but not limited to, character features, categorical features, and numerical features. Therefore, different embedding methods are used to process different features. For example, the encoder of the autoencoder embeds character features using a truncation and padding process. Meanwhile, the encoder embeds numerical and categorical features separately and expands across multiple dimensions to accommodate character features of the same size. Furthermore, for the decoder topology, a convolutional decoder network is symmetrically configured using convolutional layers, max pooling layers, skip connection modules, and fully connected layers.

[0008] The reconstructed unlabeled data samples are compared with the unlabeled data samples to determine the corresponding reconstruction loss. Because the unlabeled data samples contain different types of features, different loss functions are utilized to determine the reconstruction loss for the different types of features. For example, in one embodiment, a cross-entropy loss function is used for character features and categorical features, and a mean square error (MSE) loss function is used for numerical features. Furthermore, a total loss is calculated as the sum of all the reconstruction losses for the different types of features multiplied by their weights. The total loss is used for backpropagation and autoencoder update during autoencoder training.

[0009] Additionally, in one embodiment, to achieve the best performance of the autoencoder, a hyperparameter optimization framework is used to adjust hyperparameter settings (e.g., learning rate) and some network structures (e.g., channel size of convolutional layers) over a wide range of trials. The goal of the hyperparameter optimization framework is to maximize the test accuracy when testing the trained autoencoder against a new anomaly detection dataset, which helps determine the best topology and hyperparameter settings for the autoencoder.

[0010] Furthermore, during the second training stage of the multi-stage training, model stacking is performed by replacing the weighted sum calculation of the autoencoder's total loss with a classification model. In particular, the autoencoder trained according to the first training stage is stacked with the classification model to take all different loss terms (i.e., reconstruction losses of different types of features) as inputs and predict the class of anomaly as output. The classification model is constructed using a logistic regression (LR) network or a multilayer perceptron (MLP) network. During the second training stage, the combination of the autoencoder trained according to the first training stage and the classification model is trained in a supervised learning manner using labeled data samples.

[0011] During the second training phase, some networks may be frozen to achieve the best performance on the test dataset and enhance the robustness of the entire model (i.e., the autoencoder and the classification model). For example, the encoder and decoder of the autoencoder may be frozen, and only the classification model is trained using labeled data samples.

[0012] In some embodiments, the unlabeled data samples and the labeled data samples may be from the same domain. Therefore, an autoencoder and / or a classification model trained using the unlabeled data samples and the labeled data samples has limited generality against new attacks and cannot adapt to new domains even if they have similar distributions. Therefore, some embodiments aim to adapt the autoencoder and the classification model trained according to the second training stage to new domains. Such domain adaptation is achieved by performing a third training stage.

[0013] During a third training phase, the autoencoder and classification model trained according to the second training phase are trained in a supervised manner using labeled samples from a new domain. This new domain is different from the domain of the unlabeled and labeled data samples. During the third training phase, a learning rate smaller than that used in the second training phase is utilized to slightly modify the classification boundary of the classification model to fit the new attack samples without degrading the test accuracy on the previously trained dataset (i.e., the unlabeled and labeled data samples). Such domain adaptation is applied multiple times to identify different but related anomalies, which may even detect invisible anomalies.

[0014] In some embodiments, some networks may be frozen during the third training stage. For example, the encoder and decoder of an autoencoder may be frozen, and only the classification model is trained using labeled examples of the new domain. In another example, only the encoder is frozen, and the decoder and classification model are trained using labeled examples of the new domain.

[0015] The multi-stage training results in a neural network architecture trained for anomaly detection, which includes a trained autoencoder and a classification model. During real-time operation, input data / test samples are provided to the autoencoder. The autoencoder includes an encoder trained to encode the input data and a decoder trained to decode the encoded input data and reconstruct the input data. A loss estimator compares multiple portions of the input data with corresponding multiple portions of the reconstructed input data to determine a sequence of losses for different components of the reconstruction error. A supervised trained classification model classifies the sequence of losses to detect anomalies and generates an anomaly detection result including one or a combination of anomaly type and anomaly severity.

[0016] Accordingly, one embodiment discloses an anomaly detector comprising at least one processor and a memory storing instructions forming an anomaly detector module, the at least one processor configured to execute the instructions of the anomaly detector module. The module comprises an input interface configured to accept input data and a first neural network having an autoencoder architecture including an encoder trained to encode the input data and a decoder trained to decode the encoded input data and reconstruct the input data. The module further comprises a loss estimator configured to compare multiple portions of the input data with corresponding multiple portions of the reconstructed input data to determine a sequence of losses for different components of the reconstruction error. The module further comprises a second neural network trained in a supervised manner to classify the sequence of losses to detect anomalies and generate an anomaly detection result including one or a combination of an anomaly type and an anomaly severity. The anomaly detector further comprises an output interface configured to render the anomaly detection result.

[0017] Accordingly, one embodiment discloses a method for detecting anomalies, the method including: receiving input data; encoding the input data based on a first neural network having an autoencoder architecture; and decoding the encoded input data to reconstruct the input data. The method further includes obtaining a sequence of losses by comparing multiple portions of the input data with corresponding multiple portions of the reconstructed input data; classifying the sequence of losses to detect anomalies based on a second neural network trained in a supervised manner, generating an anomaly detection result including one or a combination of anomaly type and anomaly severity; and rendering the anomaly detection result.

[0018] Accordingly, one embodiment discloses a non-transitory computer-readable storage medium having embedded thereon a program executable by a processor to perform an anomaly detection method. The method includes receiving input data, encoding the input data based on a first neural network having an autoencoder architecture, and decoding the encoded input data to reconstruct the input data. The method further includes obtaining a sequence of losses by comparing multiple portions of the input data with corresponding multiple portions of the reconstructed input data, classifying the sequence of losses to detect anomalies based on a second neural network trained in a supervised manner, and generating an anomaly detection result including one or a combination of anomaly type and anomaly severity, and rendering the anomaly detection result. [Brief explanation of the drawings]

[0019] [Figure 1A] FIG. 1A is a diagram illustrating an anomaly detector for anomaly detection according to an example embodiment. [Figure 1B] FIG. 1B illustrates a functional diagram of an anomaly detector, according to an example embodiment. [Figure 2A] FIG. 2A illustrates a first training stage according to an example embodiment. [Figure 2B]FIG. 2B illustrates a second training stage according to an example embodiment. [Figure 2C] FIG. 2C illustrates a third training stage, according to an example embodiment. [Figure 3A] FIG. 3A illustrates an architecture of an anomaly detector including a reconstruction loss-based classifier, according to an example embodiment. [Figure 3B] FIG. 3B is a diagram illustrating an anomaly detector architecture including an attribute-based classifier, according to an example embodiment. [Figure 4] FIG. 4 is a diagram illustrating an architecture of an anomaly detector's explainability model, according to an example embodiment. [Figure 5A] FIG. 5A is a diagram illustrating decomposition of internet proxy log data into categorical and numerical features according to an example embodiment. [Figure 5B] FIG. 5B illustrates a concatenation module configured to concatenate data corresponding to multiple features of a uniform resource locator (URL), according to an example embodiment. [Figure 6] FIG. 6 illustrates a flowchart of a method for detecting anomalies, according to an example embodiment. [Figure 7] FIG. 7 illustrates a block diagram for detecting cyber-attacks on a substation automation system (SAS), according to an example embodiment. [Figure 8] FIG. 8 illustrates a block diagram for anomaly detection in video data according to an example embodiment. [Figure 9] FIG. 9 illustrates a block diagram of a computer-based anomaly detector for detecting anomalies, according to an example embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, devices and methods are shown in block diagram form solely to avoid obscuring the present disclosure.

[0021] As used in this specification and claims, the terms "for example," "for instance," and "such as," as well as the verbs "comprising," "having," "including," and other forms of these verbs, when used in conjunction with a list of one or more components or other items, should each be construed as open-ended, meaning that the list should not be considered to exclude further components or items. The term "based on" means based at least in part on. Furthermore, it should be understood that the style and terminology used herein are for purposes of description and should not be considered to be limiting. Any headings used herein are for convenience only and are not to be considered to have any legal or limiting effect.

[0022] 1A shows a block diagram 100A of an anomaly detector 101 for anomaly detection, according to an example embodiment. The anomaly detector 101 includes an input interface 105 for receiving input data 103. Additionally, the anomaly detector 101 includes a processor 107 and a memory 109. Examples of the processor 107 include, but are not limited to, an application-specific integrated circuit (ASIC) processor, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a field-programmable gate array (FPGA), etc.

[0023] In some embodiments, memory 109 includes appropriate logic, circuitry, and / or interfaces for storing a set of computer-readable instructions for performing operations. Additionally, examples of memory 109 may include random access memory (RAM), read-only memory (ROM), removable storage drives, hard disk drives (HDDs), etc. It will be apparent to those skilled in the art that the scope of the present disclosure is not limited to implementing memory 109 within anomaly detector 101 as described herein.

[0024] The memory 109 is configured to store instructions forming modules of the anomaly detector 101. The modules include a first neural network 111, a loss estimator 113, a second neural network 115, and an explainability model 119. Additionally, in some embodiments, the input interface 103 and the output interface 117 are also stored in the memory 109 in the form of modules. The processor 107 executes the instructions stored in the memory 109 in the form of modules. The anomaly detector 101 is configured to process the input data 103 using the modules stored in the memory 109 to generate anomaly detection results 121, as described in FIG. 1B below.

[0025] 1B shows a functional diagram 100B of an anomaly detector 101 according to an example embodiment. The anomaly detector 101 receives input data 103. The input data 103 may correspond to sequential data including a heterogeneous mixture of features. The sequential data may include text data (e.g., internet proxy log data), image data, video data, audio data, etc. In some embodiments, the input data 103 may include multiple features, such as categorical features, numerical features, character features, etc. The input data 103 is input to a first neural network 111.

[0026] The first neural network 111 has an autoencoder architecture including an encoder and a decoder. The encoder is trained to encode input data 103, and the decoder is trained to generate reconstructed input data 111a by decoding the encoded input data and reconstructing the input data. The reconstructed input data 111a is provided to a loss estimator 113. The loss estimator 113 is configured to compare multiple portions of the input data 103 with corresponding multiple portions of the reconstructed input data 113a to determine a sequence of losses 113a for different components of the reconstruction error. The sequence of losses 113a corresponds to the difference between the input data 103 and the reconstructed input data 111a.

[0027] Furthermore, the loss sequence 113a is provided to a second neural network 115. The second neural network 115 is trained in a supervised manner to classify the loss sequence 113a to detect anomalies and generate anomaly detection results 121. The anomaly detection results 121 are rendered to a user via an output interface 117. The anomaly detection results 121 include classes of anomalies. For example, the second neural network 115 may classify the loss sequence 113a into a group of classes (class1 to class2). n ) 115a. Class 1 may be a "non-anomalous class" indicating that no anomaly is detected. n may correspond to different types of anomalies. The second neural network 115 may determine, for example, that a class 5 anomaly has been detected.

[0028] In some embodiments, the second neural network 115 determines the probability of a class to which the anomaly may belong. The class probability is compared with a threshold probability to determine the class of the anomaly. For example, the second neural network 115 may output class 2 with a probability of 0.2, meaning that an anomaly of class 2 is likely to be present in the input data 103. If the probability of class 2 is greater than the threshold probability, it is inferred that an anomaly of class 2 has been detected.

[0029] Additionally or alternatively, the anomaly detection result 121 may include a severity of the anomaly. Examples of severity of the anomaly may include low, medium, high, and severe. For example, the second neural network 115 may determine that the anomaly is of class 2, which has a severe severity.

[0030] The neural network architecture (shown in FIG. 1B), including first neural network 111 and second neural network 115, is trained for anomaly detection in multiple stages, each of which is described below in FIGS. 2A, 2B, and 2C.

[0031] 2A illustrates a first training stage 200A according to an example embodiment. Some embodiments recognize that, in practice, while a large set of normal data samples may be available, a sufficient number of abnormal data samples may not be available because it is difficult to enumerate all possible types of abnormal samples. Therefore, anomaly detection is an unsupervised learning problem that does not rely on prior information; in other words, all data samples are assumed to be unlabeled.

[0032] Therefore, during the first training stage 200A, the first neural network 111 is trained in an unsupervised manner using unlabeled data samples 201. In particular, an autoencoder included in the first neural network 111 is trained on the unlabeled data samples 201 to learn their dimensional representations and use these representations to reconstruct the unlabeled data samples 201. In one embodiment, the unlabeled data samples 201 are structured proxy data samples that include a mixture of different types of features, which may include, but are not limited to, character features, categorical features, and numerical features. Therefore, different embedding methods are used to process different features. For example, the encoder of the autoencoder embeds character features using a truncation and padding process. On the other hand, the encoder embeds numerical and categorical features separately and expands across multiple dimensions to accommodate character features of the same size. Furthermore, for the decoder topology, a convolutional decoder network is configured symmetrically using convolutional layers, max pooling layers, skip connection modules, and fully connected layers.

[0033] The reconstructed unlabeled data sample is compared with the unlabeled data sample 201 to determine the corresponding reconstruction loss. Because the unlabeled data sample 201 includes different types of features, different loss functions are utilized to determine the reconstruction loss for the different types of features. For example, in one embodiment, a cross-entropy loss function is used for character features and categorical features, and a mean squared error (MSE) loss function is used for numerical features. Furthermore, a total loss is calculated as the sum of all the reconstruction losses for the different types of features multiplied by their weights. The total loss is used for backpropagation and updating the autoencoder during training of the first neural network 111.

[0034] Additionally, in one embodiment, a hyperparameter optimization framework is used to adjust hyperparameter settings (e.g., learning rate) and some network structures (e.g., channel size of convolutional layers) in extensive trials to achieve the best performance of the first neural network 111. The goal of the hyperparameter optimization framework is to maximize test accuracy when testing the trained first neural network 111 against a new anomaly detection dataset, which helps determine the best topology and hyperparameter settings for the autoencoder.

[0035] FIG. 2B illustrates a second training stage 200B according to an example embodiment. During the second training stage 200B, model stacking is performed by replacing the weighted sum calculation of the total loss of the first neural network 111 with the second neural network 115. In particular, the first neural network 111 trained according to the first training stage is stacked with the second neural network 115 to take all different loss terms (i.e., different types of feature reconstruction losses) as inputs and predict the class of anomaly as output. The second neural network 115 is constructed using a logistic regression (LR) network or a multilayer perceptron (MLP) network. During the second training stage 200B, the combination of the first neural network 111 trained according to the first training stage 200A and the second neural network 115 is trained in a supervised learning manner using labeled data samples 203. In this manner, the first neural network 111 and the second neural network 115 may be jointly trained end-to-end.

[0036] During the second training stage 200B, some networks may be frozen to achieve the best performance on the test dataset and to enhance the robustness of the entire model (i.e., the first neural network 111 and the second neural network 115). As an example, the encoder and decoder of the first neural network 111 may be frozen, and only the second neural network 115 is trained with the labeled data samples 203.

[0037] In some embodiments, the unlabeled data samples 201 and the labeled data samples 203 may be from the same domain. Therefore, the first neural network 111 and / or the second neural network 115 trained using the unlabeled data samples 201 and the labeled data samples 203 have limited generality against new attacks and cannot adapt to new domains even if they have similar distributions. Therefore, some embodiments aim to adapt the first neural network 111 and the second neural network 115 trained according to the second training stage to new domains. Such domain adaptation is achieved by performing a third training stage, as described in FIG. 2C below.

[0038] 2C illustrates a third training stage 200C according to an example embodiment. During the third training stage 200C, the first neural network 111 and the second neural network 115, which were trained according to the second training stage 200B, are trained in a supervised manner using labeled samples 205 of a new domain. This new domain is different from the domain of the unlabeled data samples 201 and the labeled data samples 203. During the third training stage 200C, a learning rate smaller than that used in the second training stage 200B is utilized to slightly modify the classification boundary of the second neural network 115 to fit the new attack samples without degrading the accuracy of testing on the previously trained dataset (i.e., the unlabeled data samples 201 and the labeled data samples 203). Such domain adaptation is applied multiple times to identify different but related anomalies, potentially even detecting invisible anomalies.

[0039] In some embodiments, some networks may be frozen during the third training stage 200C. For example, the encoder and decoder of the first neural network 111 may be frozen, and only the second neural network 115 is trained with labeled examples 205 of the new domain. In another example, only the encoder is frozen, and the decoder and second neural network 115 are trained with labeled examples 205 of the new domain.

[0040] Some embodiments are based on the recognition that in addition to detecting anomalies, it is useful to explain the detected anomalies, for example, to explain which characters / features in the input data 103 were anomalous. Explanations for detected anomalies are useful for investigating the anomalies and formulating appropriate response strategies. To that end, in some embodiments, an explainability model is provided that is configured to explain the detected anomalies. In one embodiment, the explainability model is a reconstruction loss-based classifier. In another embodiment, the explainability model is an attribute-based classifier.

[0041] 3A illustrates the architecture of an anomaly detector 300A including a reconstruction loss-based classifier according to an example embodiment. For purposes of illustration, the input data 103 is considered to be Internet proxy log data. The anomaly detector 300A is configured for cyber-attack anomaly detection on the Internet proxy log data 103. The Internet proxy log data is sequential data that includes multiple features, such as categorical features, embedded features, and numerical features. For example, the input data 103 includes multiple features such as URL features 301, categorical features 303, and numerical features 305. Categorical features are features that have values from a discrete set of possibilities, and the size of this set is manageable.

[0042] For example, in Internet proxy log data, examples of categorical features include HTTP response / error codes (which belong to a relatively small set), some top-level domain categories, protocol categories, file extensions, etc. Embedded features are categorical features in nature, and the size of the set of possible values is too large and unwieldy. The set of possible words that can appear in a domain name is an example of a categorical feature that requires embedding, representing only a subset of the most common words. Numeric features are features that are numerical in nature, such as the size of a response, or character occurrence statistics extracted from text. Such different features present in Internet proxy log data are described in more detail in Figure 5A.

[0043] In order to detect anomalies in the Internet proxy log data, it is important to analyze the input data 103 corresponding to each feature among the plurality of features (301, 303, and 305). To achieve this, the Internet proxy log data may be divided into multiple portions based on the plurality of features present in the Internet proxy data.

[0044] To this end, anomaly detector 300A receives input data 103 as input, and further divides input data 103 into multiple portions based on URL features 301, category features 303, and numeric features 305.

[0045] Therefore, the portion of the input data 103 corresponding to the URL features is provided to a character-level embedding module 307. The character-level embedding module 307 is configured to perform character or word embedding to generate fixed-dimensional numeric vector representations. A word embedding is a learned representation of text in which words or characters with the same meaning have similar representations. In word embedding techniques, individual words are represented as real-valued vectors in a predetermined vector space. Each word is represented by a real-valued vector, which is often several or even hundreds of dimensions. Unlike word embeddings, the vectorized embedded feature data from the character-level embedding module 307 is provided to a concatenation module 311.

[0046] Similarly, the portion of the input data corresponding to the categorical features 303 is converted into a numeric vector via a one-hot encoding module 309. The one-hot encoding module 309 is configured to one-hot encode data corresponding to the categorical features 303 to convert the categorical features 303 into a numeric vector representation. The one-hot encoding module 309 performs binarization of the categorical features 303 and includes the binarized features in a numeric vector. Therefore, the numeric vector generated using the one-hot encoding module 309 includes 0 and 1. The vectorized categorical data from the one-hot encoding module 309 is provided to a concatenation module 311.

[0047] Additionally, the numeric features 305 are numerical in nature. In some embodiments, the numeric features may be normalized before being provided to the concatenation module 311.

[0048] The concatenation module 311 combines the numeric vectors corresponding to all the features to form concatenated data. The concatenated data is provided to a first neural network 111 having an autoencoder architecture, which encodes and decodes the concatenated data to reconstruct the input data 103. The reconstructed input data includes multiple features, namely, embedded features, categorical features, and numeric features. To analyze the data corresponding to each feature, the reconstructed input data is provided to a loss estimator 113. The loss estimator 113 generates a URL loss, a categorical loss, and a numeric loss using a cross-entropy loss function 313, a cross-entropy loss function 315, and an MSE loss function 317, respectively.

[0049] Further, the URL loss, category loss, and numeric loss are provided to a second neural network 115. The second neural network 115 determines a class level as an output based on the URL loss, category loss, and numeric loss. A logistic regression network or a multilayer perceptron network is used as the second neural network 115. The explainability model 119 outputs an anomaly detection result 319 based on the output of the second neural network 115.

[0050] Instead, in some embodiments, the explainability model 119 is an attribute-based classifier. Figure 3B shows the architecture of an anomaly detector 300B including an attribute-based classifier according to an example embodiment. The input data 103, the character-level embedding module 307, the one-hot encoding module 309, the concatenation module 311, the first neural network 111, and the loss estimator 113 are as described in Figure 3A. The second neural network 115 determines attribute scores 321. The attribute scores 321 indicate how many of each character, categorical feature, and numeric feature of the internet proxy log data (i.e., the input data 103) can be identified as an anomalous sample. Furthermore, the attribute scores 321 are input to the explainability model 119, which is an attribute-based classifier. The explainability model 119 interprets the anomalies detected by the anomaly detector based on the attribute scores 321 and outputs an anomaly detection result 319 including the class of the detected anomaly.

[0051] In some embodiments, the explainability model 119 utilizes an integrated gradients (IG) method to determine the type of attack. IG represents the integral of the gradient with respect to the input along the path from a given baseline to the input. In general, IG provides sufficient sensitivity to track different predictions for different inputs and baselines. In addition, the IG method satisfies implementation invariance, which means that attributes can be colloquially defined and accumulated for output to multiple features. An IS score accompanies the trace from the baseline to the input.

[0052] FIG. 4 illustrates an architecture 400 of the explainability model 119 of the anomaly detector 101 according to an example embodiment. The model 401 is a combination of the first neural network 111, loss estimator 113, and second neural network 115 (described in FIG. 1B). The model 401 outputs benign / attack decisions and attribute / IG scores, which are further input to the explainability model 119. The explainability model 119 uses the IS scores 403 of benign samples, a two-sample Kolmogorov-Smirnov (KS) test 405, and an attack type classifier 407 to determine justified benign / attack 409 and / or classified attack 411. To distinguish between attack and benign samples, the two-sample KS test 405 is performed. The two-sample KS test 405 uses cumulative distribution functions (CDFs) on a set of attack and historical benign samples. The two-sample KS test 405 provides a further decision on the benign / attack decision 409. If both the justified benign / attack 409 and the two-sample KS test 405 provide an anomaly (attack) decision, the attack type classifier 407 classifies the attack and outputs a classified attack 411.

[0053] The IG method is implemented by the anomaly detector 101, and the model 401 assigns an attribute score to each input feature to identify important features associated with anomalous samples. In one example, for a single anomalous sample, the IG method can identify an anomaly indicator from the characters, categorical features, and numeric features of its URL. Because the length of the URL varies for each group of samples, the IG method instead identifies an anomaly indicator at the coded vector level, which is a fixed length, by encoding the characters of the original URL of each sample into a compressed vector representation.

[0054] In some other embodiments, the explainability model 119 utilizes a correlation heatmap, a graphical representation of a 2D correlation matrix using colored cells, to represent the correlation between different variables. In the case of the IG score, the rows and columns of the correlation matrix are attributes of multiple features, and the value of each element in the correlation matrix represents the dependency, also known as correlation, between two corresponding row and column features. The values of the elements in the correlation matrix, also known as correlation coefficients, are visualized as colored cells, implementing a heatmap to show the predictive dependency relationship. As an example, a green cell represents a positive correlation, and a red cell represents a negative correlation. The darker the cell color, the stronger the correlation between the two features. The correlation heatmap visualizes statistical measurements of the correlation between different features as a graphical representation, directly showing their dependency on the IG score.

[0055] Some embodiments recognize that when trying to validate numerical features, for example, when trying to identify the top five positive and negative correlations, it is difficult to group all of the features by correlation coefficient. To alleviate such problems, correlation heatmap clustering is introduced, i.e., clustering correlation coefficients into close groups. Pairwise positive and negative correlation information is used to determine the optimal number of clusters and rearrange the row and column features in the correlation heatmap. Each cluster in this new correlation heatmap represents a group of features that all have close positive or negative correlations, indicating their strong dependency in predicting data samples. Using correlation clustering of IG attributes or scores, the explainability model 119 generates different heatmaps as unique fingerprints for different types of attacks.

[0056] 5A shows a schematic diagram 500A for decomposing Internet proxy log data into categorical and numerical features, according to an example embodiment. Internet proxy log data 501 includes information associated with requests made by users to the network. For example, Internet proxy log data 501 includes the host ID, client ID, and user ID of the user who requested the network to access a particular website or web content. Internet proxy log data 501 also includes the data time, time zone, and the command used by the user to access the particular website or web content, along with information about the status of the command and the number of bytes used by the command.

[0057] The Internet proxy log data 501 is raw data that includes sequences of log entries of Internet traffic requests from many different users, and these sequences of log entries are essentially interleaved in the Internet proxy log data 501. Therefore, to detect anomalies in the Internet proxy log data 501, the anomaly detector 101 first deinterleaves the sequences of log entries generated by different users and then treats each user's sequence independently. Furthermore, the alternative of simply processing all the sequences while they are interleaved may burden the training of the first neural network 111 with additional and unnecessary complexity.

[0058] A uniform resource locator (URL) 531 corresponding to one of the deinterleaved sequences may be obtained by the anomaly detector 101, which decomposes the URL 531 into multiple parts based on multiple features included in the URL 531. The URL 531 includes different information associated with a request made by a user to access a website or web content. The information included in the URL 531 is decomposed into categorical features 533 and numeric features 535. The information decomposed into categorical features 533 includes a method name used by a user to access a website, where the method name corresponds to “GET,” which is the default HTTP method used to retrieve resources from a specific URL. The information included in categorical features 533 further includes a subdomain word, in this case “download,” a domain word, in this case “windowsupdate,” a generic top-level domain (TLD) “co.”, a country code TLD “.jp,” and a file extension “.exe.” Because the word vocabulary size of the subdomain words and domain words is very large, they may be further categorized into embedded features.

[0059] Furthermore, the information of the URL 531 classified into the numeric features 535 includes the number of levels (#), the number of lowercase letters, the number of uppercase letters, the number of numeric values, the number of special characters, and the number of parameters. The data corresponding to each feature is vectorized. The vectorized data corresponding to the categorical features 533 and the numeric features 535 are provided to the concatenation module 311.

[0060] 5B shows a block diagram 500B of a concatenation module 311 for concatenating vectorized data corresponding to all categorical features 533 and numeric features 535, according to some embodiments of the present disclosure. The concatenation module 311 concatenates the vectorized data corresponding to all categorical features 533 and numeric features 535 to generate concatenated data 537.

[0061] To vectorize the data (text) in the domain words and subdomain words, the character-level embedding module 307 is trained using training data containing words that form the vocabulary of the most common words. Words outside the most common set may be labeled as an "other" group in the training of the character-level embedding module 307. However, the required vocabulary may still be very large, making the vocabulary size unmanageable. Therefore, to handle the vocabulary size during training, a word embedding module may be pre-trained for each feature in the multiple features present in the URL 531 to convert each word in the domain words and subdomain words into a vector of features with smaller dimensions, rather than using a very large one-hot categorical encoding. In this way, these embedding vectors (i.e., feature vectors) are used as processed features for the first neural network 111 to work with, instead of the original domain words / subdomain words.

[0062] The concatenated data 537 is provided to the first neural network 111, and the autoencoder uses an encoder to encode the concatenated data 537 into a latent space representation. The autoencoder architecture further uses a decoder to reconstruct the concatenated data 537 (i.e., vectorized URLs 531) from the latent space representation of the concatenated data 537. The reconstructed concatenated data 537 is further processed by the loss estimator 113, the second neural network 115, and the explainability model 119, as described in FIG. 3A, to generate the anomaly detection results 313.

[0063] 6 shows a flowchart of a method 600 for detecting anomalies, according to an example embodiment. Method 600 begins at step 601, where an anomaly detector (e.g., anomaly detector 101) may receive input data (e.g., input data 103) for which anomalies are to be detected. The input data may correspond to sequential data including a heterogeneous mixture of features. The sequential data may include text data (e.g., internet proxy log data), image data, video data, audio data, protein sequences, etc. In some embodiments, the input data may include multiple features, such as categorical features, numerical features, etc.

[0064] Furthermore, in step 603, the input data may be provided to a first neural network of the anomaly detector (e.g., the first neural network 111 having an autoencoder architecture), and the input data may be encoded by an encoder neural network included in the autoencoder architecture. The input data may be compressed and further encoded into a latent space representation by the encoder.

[0065] In step 605, the encoded input data may be reconstructed using a decoder neural network of the autoencoder architecture of the first neural network. The decoder may reconstruct the input data using a latent space representation of the input data.

[0066] In step 607, a sequence of losses is determined by comparing portions of the input data with corresponding portions of the reconstructed input data. In step 609, the sequence of losses is classified based on a second neural network (e.g., first neural network 115) trained in a supervised manner to detect anomalies and generate an anomaly detection result. The anomaly detection result includes one or a combination of an anomaly type and an anomaly severity.

[0067] According to some embodiments, the anomaly detector 101 may be used to detect cyber attacks against a substation automation system (SAS) in a power grid. The SAS uses IEC 61850-based protocols, such as Goose, SMV, and MMS. Cyber attacks against the SAS are possible via IEC 61850 because it uses Ethernet-based communications that do not support encryption in substation field devices. For example, Generic Object Oriented Substation Event (GOOSE) messages are used to send trip signals to substation circuit breakers. However, a cyber attacker may spoof and monitor MMS packets, which contain critical information such as TCP flag session (e.g., port number, sequence number, ACK number), IP address, and MMS field information (e.g., item ID, read / write status, etc.). The SAS may collect information about such events as event log data, including log entries. The log entries include categorical information, numerical information, and textual features that describe vital information about the data packets and / or other system-level events. The anomaly detector 101 may be used to detect cyber-attacks against the SAS, as described below with respect to FIG.

[0068] 7 illustrates a block diagram 700 for detecting a cyber-attack on a SAS 701 according to an example embodiment. The substation 701 is communicatively coupled to an anomaly detector 101. Data corresponding to the operation of the SAS 701 is input to the anomaly detector 101. For example, event log data collected by the SAS 701 is input to the anomaly detector 101. The anomaly detector 101 processes the input data as described in FIG. 1B to generate an anomaly detection result 703 that includes one or a combination of the type of cyber-attack on the SAS 701 and the severity of the cyber-attack.

[0069] FIG. 8 shows a block diagram 800 for anomaly detection in video data 801 according to an example embodiment. The video data 801 is in the form of sequential data and may be real-time video and / or recorded video. In FIG. 8, the video data 801 is of a patient 801a lying in bed, whose heart rate is being monitored using an electrocardiogram (ECG) machine 801b. The video data 801 is provided to anomaly detector 803. The anomaly detector 803 may correspond to the anomaly detector 101. Upon receiving the video data 801, the anomaly detector 803 may process the video data 801. Each image frame of the video data 801 includes different features, e.g., different color channels such as green, red, blue, etc. The different features of the video data 801 may include pre-processed motion vectors in addition to raw images. Furthermore, each image frame is processed by an anomaly detector 803 using various tools such as object detection, skeleton tracking, etc., which result in multiple features in the video data 801 in addition to the raw pixel values.

[0070] For example, the anomaly detector 803 may use object detection tools to detect the ECG machine 801b in the image frames and zoom in or out on the ECG machine 801b in the image frames. Further, the image of the ECG graph on the ECG machine 801b may be analyzed to detect an abnormality in the heartbeat of the patient 801a. The anomaly detector 803 may determine a sequence of losses corresponding to the image of the ECG graph on the ECG machine 801b included in one or more image frames of the video data 801. The anomaly detector 803 uses the sequence of losses to determine an anomaly detection result 805 including a type of abnormality in the heartbeat of the patient 801a and / or a severity of the abnormality.

[0071] In another embodiment, the anomaly detector 803 may be used to detect abnormalities in the posture (or position) of the patient 801a. For example, when the patient 801a is about to fall off the bed, the patient 801a may be in an abnormal position. Furthermore, the abnormal position of the patient 801a may be due to a seizure. Based on the video data 801, the anomaly detector 803 may determine multiple features associated with the movement of the patient 801a from various image frames of the video data 801. Furthermore, a skeleton tracking tool may be used by the anomaly detector 803 to detect abnormalities in the position (or posture or position) of the patient 801a. The anomaly detector 803 may then determine the type of abnormality in the position of the patient 801a.

[0072] Illustrative implementation

[0073] 9 shows a block diagram of a computer-based anomaly detector 900 for detecting anomalies from input data, according to an example implementation. The computer-based anomaly detector 900 includes several interfaces that connect the anomaly detector 900 to other systems and devices. The anomaly detector 900 includes an input interface 901 configured to accept input data 103, where the input data 103 includes sequential data such as internet proxy data, video data, audio data, image data, etc.

[0074] In some embodiments, the anomaly detector 900 includes a network interface controller (NIC) 905 configured to acquire input data 103 over a network 907, which may be one or a combination of wired and wireless networks.

[0075] A network interface controller (NIC) 905 is adapted to connect the anomaly detector 900 via a bus 923 to a network 907 that connects the anomaly detector 900 to an input device 903. The input device 903 may correspond to a proxy log data recorder that records proxy log data, which is provided to the anomaly detector 900 for detecting anomalies in the recorded proxy log data. In another embodiment, the input device 903 may correspond to a video recorder that records video, which is provided to the anomaly detector 900 for detecting anomalies in the recorded video data.

[0076] Additionally or alternatively, anomaly detector 900 may include a human machine interface (HMI) 911. Human machine interface 911 in anomaly detector 900 connects anomaly detector 900 to a keyboard 913 and a pointing device 915, which may include a mouse, trackball, touchpad, joystick, pointing stick, stylus, or touchscreen, among others.

[0077] Anomaly detector 900 includes a processor 921 configured to execute stored instructions 917 and a memory 919 that stores instructions executable by processor 921. Processor 921 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Memory 919 may include random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory system. Processor 921 may be connected to one or more input and output devices via a bus 923.

[0078] The instructions 917 may implement a method for detecting anomalies, according to some embodiments, for which the computer memory 919 stores the first neural network 111, the loss estimator 113, and the second neural network 115.

[0079] The first neural network 111 has an autoencoder architecture including an encoder trained to encode input data and a decoder trained to decode the encoded input data and reconstruct the input data. The loss estimator 113 compares portions of the input data with corresponding portions of the reconstructed input data to determine a sequence of losses for different components of the reconstruction error. The second neural network 115 is trained in a supervised manner to classify the sequence of losses to detect anomalies and generate anomaly detection results including one or a combination of anomaly type and anomaly severity.

[0080] In some embodiments, output interface 927 may be configured to render the anomaly detection results on a display device 909. Examples of display device 909 include a computer monitor, a television, a projector, or a mobile device, among others. Computer-based anomaly detector 900 may also be connected to an application interface 925, which is adapted to connect computer-based anomaly detector 900 to external devices 923 to perform various tasks.

[0081] Embodiment

[0082] This specification provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing one or more exemplary embodiments. Various changes are contemplated that may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter, as set forth in the appended claims.

[0083] In the following description, specific details are set forth to provide a thorough understanding of the embodiments. However, those skilled in the art will recognize that the embodiments may be practiced without these specific details. For example, systems, processes, and other elements of the disclosed subject matter may be shown as components in block diagram form to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments. Furthermore, like reference numbers and names in the various drawings refer to like elements.

[0084] Individual embodiments may also be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process may be terminated when its operations are completed, or may have additional steps not shown or included in the drawings. Moreover, not all operations in any specifically described process occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the end of the function may correspond to the function returning to the calling function or the main function.

[0085] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, manually or automatically. The manually or automatically implemented forms may be executed or at least assisted by a machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments for performing the necessary tasks may be stored on a machine-readable medium. A processor may perform the necessary tasks.

[0086] Furthermore, the embodiments of the present disclosure and the functional operations described herein can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware, including the structures disclosed herein and their structural equivalents, or in combinations of one or more of them.

[0087] Furthermore, some embodiments of the present disclosure may be implemented as one or more computer programs, i.e., as one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or to control the operation of a data processing apparatus. Still further, the program instructions may be encoded on an artificially generated propagated signal, for example, an electrical, optical, or electromagnetic signal generated by a machine, the propagated signal being generated to encode information that is transmitted to a suitable receiving device for execution by a data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random-access or serial-access memory device, or one or more combinations thereof.

[0088] A computer program (which may also be called or described as a program, software, software application, module, software module, script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, such as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in part of a file that holds other programs or data, for example, in one or more scripts stored in a markup language document, in a single file dedicated to the program, or in multiple coordinated files, for example, files storing one or more modules, subprograms, or portions of code.

[0089] A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communications network. Computers suitable for running computer programs may, by way of example, be based on general-purpose or special-purpose microprocessors or both, or any other type of central processing unit. Typically, a central processing unit receives instructions and data from a read-only memory or a random-access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data.

[0090] Typically, a computer also includes one or more mass storage devices, such as magnetic, magneto-optical, or optical disks, for storing data, or is operatively coupled to such disks to receive data from, transfer data to, or both. However, a computer need not have such devices. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0091] For user interaction, embodiments of the subject matter described herein may be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, e.g., a mouse or trackball, for allowing the user to provide input to the computer. Other types of devices may be used for user interaction; for example, feedback provided to the user may be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including speech input, verbal input, or tactile input. Additionally, the computer may achieve user interaction by sending documents to and receiving documents from a device used by the user, e.g., by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0092] Embodiments of the subject matter described herein can be implemented in a computing system that includes a back-end component, e.g., a data server, or includes a middleware component, e.g., an application server, or includes a front-end component, e.g., a client computer having a graphical user interface or web browser that allows a user to interact with an implementation of the subject matter described herein, or includes any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include local area networks ("LANs") and wide area networks ("WANs"), e.g., the Internet.

[0093] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship.

[0094] Although the present disclosure has been described with reference to certain preferred embodiments, it will be understood that various other adaptations and modifications can be made within the spirit and scope of the disclosure. It is, therefore, the object of the following claims to cover all such variations and modifications that come within the true spirit and scope of the disclosure.

Claims

1. An anomaly detector, the anomaly detector comprising: at least one processor; a memory having instructions stored thereon forming a module of the anomaly detector, wherein the at least one processor is configured to execute the instructions of the module of the anomaly detector, the module comprising: an input interface configured to accept input data; a first neural network having an autoencoder architecture including an encoder trained to encode the input data and a decoder trained to decode the encoded input data and reconstruct the input data; a loss estimator configured to compare portions of the input data with corresponding portions of the reconstructed input data to determine a sequence of losses for different components of the reconstruction error; a second neural network trained in a supervised manner to classify the sequence of losses to detect anomalies and generate an anomaly detection result including one or a combination of the type of the anomaly and the severity of the anomaly; an output interface for rendering the anomaly detection results.

2. The anomaly detector of claim 1 , wherein the second neural network is a deep neural network.

3. The anomaly detector of claim 1 , wherein the first neural network and the second neural network are jointly trained.

4. The anomaly detector of claim 1 , wherein the first neural network and the second neural network are jointly trained end-to-end.

5. The anomaly detector of claim 1 , wherein during a first training stage, the first neural network is trained in an unsupervised manner using unlabeled data samples.

6. 6. The anomaly detector of claim 5, wherein during a second training stage, the first neural network trained with the unlabeled data samples and the second neural network are trained in a supervised manner with labeled data samples.

7. The anomaly detector of claim 6 , wherein during the second training stage, the second neural network is trained in the supervised learning manner using the labeled data samples.

8. 8. The anomaly detector of claim 7, wherein during a third training stage, the first neural network and the second neural network trained in the second training stage are trained using labeled samples.

9. 9. The anomaly detector of claim 8, wherein during the third training stage, the second neural network trained in the second training stage is trained using the labeled samples.

10. The anomaly detector of claim 9 , wherein a domain of the labeled samples is different from a domain of the unlabeled and unlabeled data samples.

11. The anomaly detector of claim 1 , wherein the module further comprises an explainability module configured to predict a class of an anomaly.

12. The anomaly detector of claim 11 , wherein the explainability model corresponds to an attribute-based classifier configured to determine a class of anomaly based on attribute scores corresponding to the input data.

13. The anomaly detector of claim 12 , wherein the second neural network is configured to determine the attribute scores corresponding to the input data.

14. The anomaly detector of claim 1 , wherein the input data corresponds to one of internet proxy log data, image data, video data, or audio data.

15. 1. A method for anomaly detection, the method using a processor coupled with stored instructions implementing the method, the instructions, when executed by the processor, performing steps of the method, the steps of the method comprising: Receiving input data; encoding the input data based on a first neural network having an autoencoder architecture and decoding the encoded data to reconstruct the input data; determining a sequence of losses by comparing portions of the input data with corresponding portions of the reconstructed input data; classifying the sequence of losses to detect anomalies based on a second neural network trained in a supervised manner, and generating anomaly detection results including one or a combination of the type of the anomaly and the severity of the anomaly; and rendering the anomaly detection results.

16. The method of claim 15 , wherein the input data corresponds to internet proxy log data, image data, video data, and audio data.

17. The method of claim 15 , wherein the second neural network is a deep neural network.

18. The method of claim 15 , wherein the first neural network and the second neural network are jointly trained.

19. 16. The method of claim 15, wherein the first neural network and the second neural network are jointly trained end-to-end.

20. A non-transitory computer-readable storage medium having embedded therein a program executable by a processor to perform a method, the method comprising: Receiving input data; encoding the input data based on a first neural network having an autoencoder architecture and decoding the encoded data to reconstruct the input data; determining a sequence of losses by comparing portions of the input data with corresponding portions of the reconstructed input data; classifying the sequence of losses to detect anomalies based on a second neural network trained in a supervised manner, and generating anomaly detection results including one or a combination of the type of the anomaly and the severity of the anomaly; and rendering the anomaly detection results.

Images