Network security apparatuses and related methods and security systems

Abstract

An apparatus includes an input terminal configured to receive a plurality of network packets. The apparatus further includes processing circuitry that implements a variational autoencoder trained to encode the plurality of network packets into a latent space representation of the plurality of network packets. The processing circuitry is configured to provide the plurality of network packets to the variational autoencoder to generate a latent space representation of the plurality of network packets and provide the latent space representation of the plurality of network packets to a clustering model to classify each packet of the plurality of network packets based on the latent space representation of the plurality of network packets. The processing circuitry is also configured to generate a three-dimensional plot graph based, at least in part, on the classification of each of the one or more network packets and the latent space representation of the one or more packets.

Description

[0001] NETWORK SECURITY APPARATUSES AND RELATED METHODS AND SECURITY SYSTEMS

[0002] PRIORITY CLAIM

[0003] This application claims the benefit of the filing date of United States Provisional Patent Application Serial No. 63 / 592,850, filed October 24, 2023, for “NETWORK SECURITY AND RELATED APPARATUSES, METHODS, AND SECURITY SYSTEMS,’' the disclosure of which is hereby incorporated herein in its entirety by this reference.

[0004] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0005] This invention was made with government support under Contract No. DE-AC07-05-ID14517 awarded by the United States Department of Energy. The government has certain rights in the invention.

[0006] TECHNICAL FIELD

[0007] Embodiments of the disclosure relate generally to systems and methods for classifying and visualizing network packet data using machine-learning models.

[0008] BACKGROUND

[0009] Midsize to large information technology (IT) organizations may operate hundreds to thousands of systems. Operation of such systems may be expensive and complicated. Cybercrime may cost $10.5 billion USD or more per year worldwide. Per the Mitre attack matrix, the first step for much cybercrime may begin with reconnaissance and vulnerability scanning. Around 20% of initial access attack vectors may come via network packet-based attacks.

[0010] DISCLOSURE

[0011] In some embodiments, an apparatus includes an input terminal configured to receive a plurality of network packets. The apparatus further includes processing circuitry that implements a variational autoencoder trained to encode the plurality of network packets into a latent space representation of the plurality of network packets. The processing circuitry is configured to provide the plurality of network packets to the variational autoencoder to generate a latent space representation of the plurality' of network packets and provide the latent space representation of the plurality of network packets to a clustering model to classify each packet of the plurality of network packets based on the latent space representation of the plurality of network packets. The processing circuitry is also configured to generate a three-dimensional plot graph based, at least in part, on the classification of each of the one or more network packets and the latent space representation of the one or more packets.

[0012] In some embodiments, a method of operating a security system, the method may include receiving first network data including one or more benign network packets, training a variational autoencoder for classify ing network traffic as anomalous or benign based, at least in part, on the first network data, providing second network data including one or more network packets as input to the variational autoencoder to encode the second network data into a latent space representation of the network data and to classify each of the one or more packets as anomalous or benign, and generating, one or more three-dimensional plot graphs based, at least in part, on each of the classified network packets of the second network data and the latent space representation of the second network data.

[0013] In further embodiments, a security system includes a destination device, an electronic display, a network interface, and a processing circuitry. The destination device is configured to receive one or more user inputs via a user input device in connection with the destination device. The electronic display is in connection with the destination device. The network interface is configured to receive a stream of packets from a network. The processing circuitry is configured to provide the received stream of packets to a variational autoencoder model and a convolutional neural network model to generate a first probability and a second probability, respectively, each of the first probability and the second probability indicative of a probability that a packet payload associated with a packet of the stream of packets is anomalous, provide the first probability and the second probability to a logical regression model to generate a third probability' indicative of a probability' for each packet of the stream of packets that the packet is benign, generate a three-dimensional plot graph representative of a latent space representation of the received stream of packets via the variational autoencoder wherein each plot point of the three-dimensional plot graph is representative of a packet of the received stream of packets. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] While this disclosure concludes with claims particularly pointing out and distinctly claiming specific embodiments, various features and advantages of embodiments within the scope of this disclosure may be more readily ascertained from the following description when read in conjunction with the accompanying drawings, in which:

[0015] FIG. 1 is a block diagram of a security system, according to one or more embodiments of the disclosure;

[0016] FIG. 2 is a block diagram of a processing circuitry, which is an example processing circuitry of the security system of FIG. 1;

[0017] FIG. 3 is a block diagram of a variational autoencoder, which is an example of a variational autoencoder of FIG. 1 and / or of FIG. 2;

[0018] FIG. 4 is a flowchart of illustrating a method of operating a security’ system (e.g., the security system of FIG. 1), according to some embodiments;

[0019] FIG. 5 illustrates a hybrid machine-learning model trained using both supervised and unsupervised methods according to one or more embodiments of the disclosure.

[0020] FIG. 6 is a flowchart illustrating a method of training a variational autoencoder, according to some embodiments;

[0021] FIG. 7A is a latent space representation of one or more network packets encoded via a variational autoencoder, according to one or mor embodiments of the disclosure;

[0022] FIG. 7B shows the results of adding a clustering layer to the variational autoencoder after encoding network packets into a latent space representation of the network packets, according to one or more embodiments of the disclosure;

[0023] FIG. 8 illustrates a three-dimensional human-readable plot graph representative of a latent space distribution of one or more network packets that have been classified via a variational autoencoder according to one or more embodiments of the disclosure;

[0024] FIG. 9 is an example graphical user interface (GUI) that may be used to present information to a user via a display device according to one or more embodiments of the disclosure;

[0025] FIG. 10A shows an example three-dimensional plot graph according to one or more embodiments of the disclosure;

[0026] FIG. 10B shows an example three-dimensional plot graph showing true labels of a plurality of data plots representative of a plurality of packets that have been encoded in latent space, according to one or more embodiments of the disclosure; FIG. 11 is a block diagram of circuitry that, in some embodiments, may be used to implement various functions, operations, acts, processes, and / or methods disclosed herein.

[0027] MODE(S) FOR CARRYING OUT THE INVENTION

[0028] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which are shown, by way of illustration, specific examples of embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable a person of ordinary skill in the art to practice the disclosure. However, other embodiments enabled herein may be utilized, and structural, material, and process changes may be made without departing from the scope of the disclosure.

[0029] The illustrations presented herein are not meant to be actual views of any particular method, system, device, or structure, but are merely idealized representations that are employed to describe the embodiments of the disclosure. In some instances similar structures or components in the various drawings may retain the same or similar numbering for the convenience of the reader; however, the similarity in numbering does not necessarily mean that the structures or components are identical in size, composition, configuration, or any other property.

[0030] The following description may include examples to help enable one of ordinary skill in the art to practice the disclosed embodiments. The use of the terms “exemplary,” “by example,” and “for example,” means that the related description is explanatory, and though the scope of the disclosure is intended to encompass the examples and legal equivalents, the use of such terms is not intended to limit the scope of an embodiment or this disclosure to the specified components, steps, features, functions, or the like.

[0031] It will be readily understood that the components of the embodiments as generally described herein and illustrated in the drawings could be arranged and designed in a wide variety7of different configurations. Thus, the following description of various embodiments is not intended to limit the scope of the disclosure, but is merely representative of various embodiments. While the various aspects of the embodiments may’ be presented in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0032] Furthermore, specific implementations shown and described are only examples and should not be construed as the only way to implement the disclosure unless specified otherwise herein. Elements, circuits, and functions may be shown in block diagram form in order not to obscure the disclosure in unnecessary detail. Conversely, specific implementations shown and described are exemplary only and should not be construed as the only way to implement the disclosure unless specified otherwise herein. Additionally, block definitions and partitioning of logic between various blocks is exemplar}' of a specific implementation. It will be readily apparent to one of ordinary' skill in the art that the disclosure may be practiced by numerous other partitioning solutions. For the most part, details concerning timing considerations and the like have been omitted where such details are not necessary to obtain a complete understanding of the disclosure and are within the abilities of persons of ordinary skill in the relevant art.

[0033] Those of ordinary- skill in the art will understand that infomiation and signals maybe represented using any of a variety- of different technologies and techniques. Some drawings may illustrate signals as a single signal for clarity of presentation and description. It will be understood by a person of ordinary skill in the art that the signal may represent a bus of signals, wherein the bus may have a variety- of bit widths and the disclosure may be implemented on any number of data signals including a single data signal.

[0034] The various illustrative logical blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a special purpose processor, a digital signal processor (DSP), an Integrated Circuit (IC), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor (may also be referred to herein as a host processor or simply a host) may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. A general-purpose computer including a processor is considered a special-purpose computer while the general-purpose computer is configured to execute computing instructions (e.g., software code) related to embodiments of the disclosure.

[0035] The embodiments may be described in terms of a process that is depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe operational acts as a sequential process, many of these acts can be performed in another sequence, in parallel, or substantially concurrently. In addition, the order of the acts may be re-arranged. A process may correspond to a method, a thread, a function, a procedure, a subroutine, a subprogram, other structure, or combinations thereof. Furthermore, the methods disclosed herein may be implemented in hardware, software, or both. If implemented in software, the functions may be stored or transmitted as one or more instructions or code on computer-readable media. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.

[0036] Any reference to an element herein using a designation such as “first,” “second,” and so forth does not limit the quantity' or order of those elements, unless such limitation is explicitly stated. Rather, these designations may be used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not mean that only two elements may be employed there or that the first element must precede the second element in some manner. In addition, unless stated otherwise, a set of elements may include one or more elements.

[0037] As used herein, the term “substantially” in reference to a given parameter, property, or condition means and includes to a degree that one of ordinary skill in the art would understand that the given parameter, property7, or condition is met with a small degree of variance, such as, for example, within acceptable manufacturing tolerances. By way of example, depending on the particular parameter, property, or condition that is substantially met, the parameter, property, or condition may be at least 90% met, at least 95% met, or even at least 99% met.

[0038] As used herein, the term “benign,” when used in conjunction with terms such as “packet(s),” “packet flow,” “stream of packets,” and “transfer entropy,” refers to electronic communications that are free of threats (e.g., cyber threats, security threats). For example, a “benign stream of packets” refers to a stream of packets that is free of such threats. Accordingly, a “benign packet flow” is a packet flow7corresponding to a benign stream of packets. Also, a “benign transfer entropy” refers to a transfer entropy of a benign packet flow.

[0039] Network packet metadata show significant distribution variability with multiple attack signatures ranging from malicious download, brute force attempts, vulnerability7scan, and malicious command execution. Machine-learning based classification models for attack detection are difficult to achieve with high classification speed and remain vulnerable to zero-days attacks while conventional rule-based classification systems are not scalable for high accuracy.

[0040] A cyberattack surface (e.g., a total of the different points where an unauthorized user can try to enter data or extract data from an environment) is expected to increase by an order of magnitude by the year 2025, while network attacks have increased by over fifty7times between the years 2015 and 2020. While rule-based network packet security systems have been the gold standard for protections, auxiliary machine-learning based systems, such as embodiments disclosed herein, may be useful to secure future systems. These auxiliary7rule-based learning based systems may bear some similarities to systems currently used for credit card fraud detection, but may be adapted for network packet characteristics by incorporating reconstructed probabilities.

[0041] In contrast to using binary classifications to detect attacks, disclosed in some embodiments is a mechanism for detecting network packet anomalies that are indicative of a network attack using semi-supervised learning (e.g., using a hybrid “ensemble” model) by providing a human-readable latent space representation of a plurality of network packets that have been input into a classifier machine-learning model such as, for example, a variational autoencoder with classifiers. The variational autoencoder may be trained using a benign packet flow representation of packets in a network (e.g., using benign packets to train on expected packet characteristics). The variational autoencoder may then take as input one or more packets transmitted within a network and encode the one or more packets into a latent space representation of the one or more packets. The variational autoencoder may then classify (e.g., using a classifier such as a classifier model taking the latent space representation of data input into the variational autoencoder as input) the packets, for example, as benign (e.g., in line with the data used to train the variational autoencoder) or anomalous based, at least in part, on the packet pay load of each packet input into the variational autoencoder and represented in the latent space representation of the plurality7of network packets.

[0042] In some embodiments, the variational autoencoder may automatically extract features from benign packets used to train the variational autoencoder and represent those features in latent space. After the variational autoencoder has been trained, one or more packets transmitted in a network may be input into the variational autoencoder to generate a latent space representation of the one or more packets transmitted in the network. Latent space representation of the packets may then be used to classify the packets (e.g., by a classifier model taking the latent space generated by the variational autoencoder as input). For example, the variational autoencoder with classifiers (or other type of unsupervised machine-learning model configured to classify data or including classifiers) may predict a probability of each network packet input into the variational autoencoder that the network packet is benign. By utilizing a variational autoencoder with classifiers, a plurality7of network packets may be classified (e.g., as anomalous or benign) and also encoded into a latent space representation of the plurality of network packets.

[0043] After the plurality of network packets are classified and encoded into a latent space representation of the plurality of network packets, a human-readable plot graph (e.g., a three-dimensional plot graph) based, at least in part, on the latent space representation of the one or more network packets and the classification of each of the plurality of network packets, may be generated and presented to a user via a display of a user device. For example, the human-readable plot graph may represent the latent space representation of the plurality of network packets with a reduced dimensionality (e.g., reduced to three- dimensions). Moreover, each data point represented in the human-readable plot graph may represent a packet payload of each of the plurality of network packets. Each data point may include a visual indicator (e.g., may be colored) based upon the classification of the corresponding packet payload to help a user to visually identify anomalous packets to report or further analyze one or more packets. For example, a data point in the human- readable plot graph may have the color green if the corresponding packet pay load was classified as benign and may be colored red if classified as anomalous. Furthermore, each data point represented in the human-readable plot graph may be selectable such that information corresponding the selected data point (e.g., a packet represented in the human- readable plot graph) is presented to the user via the display of the user device.

[0044] Furthermore, in addition to the variational autoencoder, a supervised machinelearning model may be trained to classify a network packet input into the supervised machine-learning model as benign or as anomalous. For example, the supervised machinelearning model may be in the form of a convolutional neural network trained using labeled (e.g.. ground truth) benign or anomalous (e.g., malicious) packet data. The supervised machine-learning model may be trained to output a probability (separate from the probability output via the variational autoencoder) of a network packet being anomalous based on the network packet input into the supervised machine-learning model. The resulting probability output from the supervised machine-learning model may then be input into a logic regression model along with the probability output from the variational autoencoder to generate a final probability’ that a given network packet is benign.

[0045] By combining the outputs of the variational autoencoder (or other type of unsupervised machine-learning model) with the output of a supervised machine-learning model into a logic regression model, a combined hybrid model may be formed such that the hybrid model may generate a probability that a packet input into the hybrid model (e.g., into both the supervised machine-learning model and the variational autoencoder) is anomalous or benign thereby leveraging the advantages and mitigating drawbacks of both supervised and unsupervised machine-learning techniques.

[0046] For example, machine-learning models trained using supervised learning typically have tendencies toward overfitting and false positives from biased training data. Machinelearning models trained using unsupervised learning, on the other hand, generally are less accurate compared to supervised learning methods. However, by integrating output by machine-learning models trained using both supervised and unsupervised methods using logistic regression, an ’ensemble" semi-supervised model may be generated that integrates the strengths of both supervised and unsupervised methods. For example, the unsupervised machine-learning model may refine feature representation for better clustering while the supervised machine-learning model may provide a trusted baseline of know n malicious and benign network traffic payloads against which probability generated by the unsupervised machine-learning model is compared. This integration may enable the ensemble model to achieve more accurate results than either the unsupervised machine-learning model or the supervised machine-learning model may achieve alone. Moreover, the hybrid model may exhibit fewer quantization errors and may reduce loss associated with quantized models, such as when the hybrid model is implemented in an FPGA. Furthermore, FPGA implementation of the hybrid model may reduce per-packet latency compared to implementation in a GPU, which may reduce power consumption for a device implementing the hybrid model.

[0047] Furthermore, by presenting the three-dimensional plot representative of a latent space representation of a network packet flow, a user may more easily visually distinguish between classifications of each netw ork packet of the network packet flow to aid in identifying anomalous packets.

[0048] FIG. 1 is a block diagram of a security system 100, according to some embodiments. The security’ system 100 includes a network interface 112 configured to receive a stream of packets 114 from a network 106 and processing circuitry 116 implementing a machine-learning model 122 trained to encode one or more packets of the stream of packets into a latent space representation of the one or more packets and to classify (e.g., as anomalous or benign) the one or more packets of the stream of packets based, at least in part, on the payload of each packet input into the machine-learning model 122. The processing circuitry may also implement a human-readable plot graph generator 104 configured to generate a human-readable plot graph 120 based on the payload of each packet and / or the output (e.g., a latent space distribution of the input data) of the machine-learning model 122. The security system 100 may also include a destination device 108, which may include an electronic display to display the human-readable plot graph 120.

[0049] Each packet of the stream of packets 1 14 provided to the machine-learning model 122 may be encoded into a latent space representation of each packet based, at least in part, on a payload data of each packet of the stream of packets 114. For example, each packet of the stream of packets 114 may be encoded into latent space via a variational autoencoder where each packet is represented in latent space as a distribution of data points. An input terminal 118 to the processing circuitry 116 is configured to receive the stream of packets 114 from the network interface 112. Using the machine-learning model 122, the processing circuitry 116 may be configured to generate, via the human- readable plot graph generator 104. a human-readable plot graph 120 based, at least in part, on a latent space representation of the stream of packets 114. The latent space representation may then be input into a decoder and / or classifier of the machine-learning model 122 to generate a reconstructed packet flow representation and may classify each packet as either anomalous or benign based on the latent space distribution of the input data. For example, the machine-learning model 122 may be configured to generate a probability (e.g., as a percentage) that a network packet input into the machine-learning model 122 is anomalous or benign. If the probability that the network packet is anomalous is above a pre-determined threshold, the machine-learning model 122 may classify the network packet as anomalous. Similarly, in some embodiments, if the probability that the network packet is benign is above a pre-determined threshold, the machine-learning model 122 may classify (e.g., label) the network packet as benign.

[0050] In some embodiments, the latent space representation may, before being input into the decoder, be input into a clustering algorithm (e.g.. a machine-learning model configured to cluster a latent space representation input into the model) to cluster data points with like features within the latent space. For example, the clustering algorithm may be used for the purposes of training the machine-learning model 122 to classify the input data as either anomalous or benign based on the clustered latent space distribution of the input data. The clustering algorithm may include one or more of k-means clustering, hierarchical clustering, mean shift clustering, agglomerative clustering. Ordering Points to Identify the Clustering Structure (OPTICS), spectral clustering, affinity propagation, fuzzy C-means clustering, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Expectation-Maximization (EM), or Gaussian Mixture Models (GMM), without limitation. After the machine-learning model 122 has been trained, the cluster layer and the decoder may be removed and the machine-learning model 122 may be configured to output a latent space representation of an input (e.g., the stream of packets 1 14) with or without a clustering algorithm. The latent space representation of the input data may be provided to the human-readable plot graph generator 104, and a human-readable plot graph 120 may be generated based, at least in part, on the latent space representation of the input data and the classification of each of the packets to visually represent the latent space representation of the stream of packets 114 as well as reflect the classification performed by the machine-learning model 122 to enable a user to visually identify one or more packets that are anomalous in comparison with metrics known to be associated with a benign packet flow.

[0051] The security system 100 also includes a destination device 108 to which the stream of packets 114 and / or the human-readable plot graph 120 are directed for delivery. Responsive to generating the human-readable plot graph 120, the processing circuitry 116 may be configured to deliver the human-readable plot graph 120 to the destination device 108. The latent space representation of the stream of packets 114 may be processed by the processing circuitry 116 to cluster or visually mark the latent space representation of the stream of packets 114 to make the data represented in the latent space representation more easily understood visually by the human eye. For example, in some embodiments, the latent space representation of the stream of packets 114 may have the dimensionality further reduced (e.g., to three dimensions) and some data points may be removed or condensed to make the resulting human-readable plot graph 120 more easily understood by a user. Furthermore, each data point may be colored or otherwise marked or labeled based on the classification of the corresponding packet by the machine-learning model 122. In some embodiments, the human-readable plot graph 120 may be a visual representation of the latent space representation of the stream of packets 114 itself without additional processing.

[0052] In some embodiments, anomalous data capable of being detected by security system 100 may include Nonstop_Virus, Backdoor_payload, Lokibot, Putingods, Sandworm, Vawtrak, Trickbot, without limitation. The security system 100 may also be configured to handle large amounts of incoming data through parallelism. For example, artificial intelligence (Al) may be used to detect anomalous data and visualize it with very little latency or delay. Possible FPGAs that may be used to implement machine-learning models discussed in the disclosure (e.g., variational autoencoder 110) include Zilinx ZCU104 or the Nvidia Al 00.

[0053] Various embodiments disclosed herein may be implemented orthogonally to rulebased network protection systems including firewall rule implementations and exceptions, and may be deployed in conjunction with such technologies. In contrast to such rule-based systems, however, packets may be manually assessed herein responsive to a human- readable latent space representation of a plurality of packets (e.g., a stream of packets).

[0054] In some embodiments, the security system 100 may be implemented in a field programmable gate array (FPGA), a computer processing unit (CPU), and / or a graphics processing unit (GPU) to support both conventional networks and 5G wireless networks with low latency.

[0055] FIG. 2 is a block diagram of a processing circuitry 200, which is one embodiment of the processing circuitry 116 of FIG. 1. The processing circuitry 200 may include a packet flow representation generator 202 and also includes a variational autoencoder 210 and a human-readable graph generator 214. The packet flow representation generator 202 is configured to receive the stream of packets 212 and generate a packet flow representation 206 responsive to the stream of packets 212. The packet flow representation generator 202 is configured to deliver the packet flow representation 206 to the variational autoencoder 210.

[0056] In some embodiments, the variational autoencoder 210 may encode and classify the stream of packets 212 based, at least in part, on payload data included in each of the packets of the stream of packets 212. In some embodiments, the packet flow representation 206 may include a transfer entropy corresponding to the packet flow of the stream of packets 212. In such embodiments, the packet flow representation generator 202 may be configured to determine the transfer entropy responsive to the stream of packets 212. In some embodiments, the packet flow representation 206 may include flow derived metadata, which may be metadata carried by the stream of packets 212, and which may include summarized data indicating the packet flow of the stream of packets 212. In such embodiments, the packet flow representation generator 202 may be configured to extract the flow-derived metadata from the stream of packets 212. In some embodiments, the packet flow representation 206 may include a Granger causality corresponding to the packet flow of the stream of packets 212. A Granger causality corresponding to the packet flow of the stream of packets 212 may include a parameter that may be used to predict the packet flow' of the stream of packets 212 based, at least in part, on a statistical causal relationship between the parameter and the packet flow. In such embodiments, the packet flow representation generator 202 may be configured to determine the Granger causality responsive to the stream of packets 212.

[0057] The variational autoencoder 210 is configured to receive the packet flow representation 206 and generate a latent space representation of packets 216 responsive to the packet flow representation 206 or the stream of packets themselves, as the case may be. The variational autoencoder 210 may be trained to encode one or more packets of the stream of packets 212 or the packet flow' representation 206 into a latent space representation having a plurality of dimensions. Additionally, the variational autoencoder 210 may be configured to classify the stream of packets 212 or the packet flow representation 206 such that each packet of the stream of packets represented in the packet flow representation 206 is classified. For example, the variational autoencoder 210 may classify one or more packets of the stream of packets 212 and determine whether the packet is anomalous or benign (e.g., expected). The classified representation of packets 208 (e.g., classified packets of the stream of packets or representations of packets of the packet flow representation) may then be combined with the latent space representation of packets 216 generated by the variational autoencoder 210 and input into the human-readable graph generator 214. In some embodiments, a human-readable graph may be generated by the human-readable graph generator 214 based on the latent space representation of the stream of packets, the packet flow' representation 206 and / or the variational autoencoder 210.

[0058] In some embodiments, the stream of packets 212 may be input directly into the variational autoencoder 210 without being input into the packet flow representation generator. In this case, the packet flow representation 206 would be the stream of packets 212 including pay load data for each packet of the stream of packets 212. In this embodiment, no metadata would need to be extracted or used by the variational autoencoder 210 as the variational autoencoder 210 will encode and classify packets of the stream of packets based on the pay load data of each packet and not based on the metadata. In some embodiments, the variational autoencoder 210 may utilize the metadata and / or the packet payload data.

[0059] FIG. 3 is a block diagram of a variational autoencoder 300, which is an example of the machine-learning model 122 of FIG. 1 and / or the variational autoencoder 210 of FIG. 2. The variational autoencoder 300 includes an input layer 302, an encoder layer 304, a latent layer 306, a decoder layer 310, and an output layer 312 layer. The encoder layer 304 may be a separate layer based on latent dimension selection. The encoder layer 304 is configured to compress a packet flow representation (e.g., the packet flow representation 206 of FIG. 2) received from the input layer 302 into a latent space 314 corresponding to the latent layer 306. In some embodiments, the variational autoencoder 300 may also include a clustering algorithm 308 that is configured to organize one or more data points represented in the latent space into disparate "clusters’7or groups. For example, the cluster layer may use one or more clustering algorithms (e g., k-means clustering, hierarchical clustering, mean shift clustering, agglomerative clustering, Ordering Points to Identify the Clustering Structure (OPTICS), spectral clustering, affinity propagation, fuzzy C-means clustering, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Expectation-Maximization (EM), or gaussian mixture models, without limitation). During training of the variational autoencoder 300, a directed acyclic graph hierarchy may have been built. The latent space 314 corresponds to a random multivariable normal distribution. The random multivariable normal distribution may be based on user-specified latent dimensions (e.g., 32 dimensions, without limitation).

[0060] The decoder layer 310 is configured to generate a reconstructed packet flow representation, which may include examples from the latent distribution of the latent space 314, responsive to information sampled from the latent space 314. For example, the decoder layer may be configured to classify one or more packets as anomalous or benign based on the latent space representation of a packet pay load of each of the one or more packets.

[0061] Once the variational autoencoder 300 is trained, separate packet metadata of the packet flow representation 206 and / or pay load data from the stream of packets 212 may be passed to the human-readable graph generator 214 classifications of each packet of the stream of packets 212 to generate a human-readable plot graph configured to visually identify classified anomalous packets as well as each packet’s location in latent space (e.g., in a reduced dimensionality graph representative of the latent space representation of packets 216) to help an operator to see if a packet was mistakenly identified as anomalous.

[0062] FIG. 4 is a flowchart of method 400 illustrating an operation of a security system (e.g.. the security system 100 of FIG. 1) performed by a processor executing instructions stored on a computer-readable storage medium. For instance, FIG. 4 shows one or more embodiments of a sequence flow diagram that the security system 100 may use to generate a human-readable plot graph showing anomalous characteristics of a plurality of network packets. At operation 402, the security system 100 provides a plurality of network packets to a variational autoencoder to classify each packet of the plurality of network packets as anomalous or benign and to generate a latent space representation of the plurality of network packets. For example, the machine-learning model 122 of the security system 100 may include a variational autoencoder configured to encode each packet of the plurality of network packets into a latent space representation of each packet based, at least in part, on a packet payload of each packet of the plurality of network packets. The latent space representation of each packet may be a distribution (e.g., a gaussian distribution or a probability distribution) representative of the data (e.g., packet payload) of each packet of the plurality of network packets. This distribution may then be randomly sampled (e.g., by a decoder) for the purposes of reconstruction based on how the decoder is trained to map samples from the latent space to an output.

[0063] In some embodiments, the variational autoencoder may be trained to reduce dimensionality of input data to a latent space representation having three dimensions. Additionally, in some embodiments, the variational autoencoder may generate a latent space having more than three dimensions and the dimensionality of the latent space representation may be reduced after the latent space representation has been generated to generate a three-dimensional latent space representation.

[0064] At operation 404, the security system 100 may provide the latent space representation of the plurality of network packets to a clustering model to classify each packet of the plurality of network packets based on the latent space representation of the plurality of network packets. For example, the latent space of the plurality of network packets may be input into a Gaussian Mixture Model (GMM), which may define boundaries and clusters (e.g., to cluster and classify) the latent space representation of the plurality of network packets.

[0065] At operation 406, the security system 100 may generate a three-dimensional plot graph based, at least in part, on the classification of each of the one or more network packets and the latent space representation of the one or more packets. For example, the three-dimensional plot graph may include a plurality of data plots, each plot representative of a network packet represented in the latent space representation of each packet. Each plot may also include information pertaining to the classification of the network packet by the machine-learning model 122 which may visually distinguish between network packets classified as anomalous and network packets classified as benign. For example, a network packet represented in the three-dimensional plot graph may be colored red if the network packet was classified as anomalous (e.g., a probability that the network packet is anomalous is above a pre-determined threshold). Likewise, a network packet represented in the three-dimensional plot graph may be colored green if the network packet was classified as benign (e.g., a probability that the network packet is benign is above a pre-determined threshold).

[0066] Furthermore, the three-dimensional plot graph may be configured to be manipulated and selected by a user via a user input device (e.g., a mouse, keyboard, touch pad, touch screen, controller, etc.). For example, the three-dimensional plot graph may be presented to a user via an electronic display device of a destination device which has received the three- dimensional plot graph. The destination device may also be in connection with the user input device, which may allow a user to manipulate a view of the three-dimensional plot graph. For example, a user may rotate or scale the three-dimensional plot graph to view the three-dimensional plot graph from different angles as well as see a full view of plots of the three-dimensional plot graph which may be hidden in certain perspective views.

[0067] Furthermore, the three-dimensional plot graph may also include selectable elements. For example, each plot in the three-dimensional plot graph may be able to be selected by a user using the user input device. Once a plot of the three-dimensional plot graph is selected, information pertaining to a network packet represented by the selected plot may be presented to the user via the electronic display device. For example, the information may include one or more of an IP source, IP destination, source port, destination port, hostname, packet pay load data, or a probability that the network packet represented by the selected plot is anomalous or benign. The information may also include a network topology map indicating one or more nodes in the network topology that may be potentially affected by the network packet (e.g.. the packet pay load of the network packet) associated with the selected plot.

[0068] FIG. 5 shows a hybrid model 500 trained using both unsupervised and supervised learning methods. The hybrid model 500 may include an unsupervised machine-learning model 514, a supervised machine-learning model 520, and an ensemble model 542. One or more packets or packet payloads may be provided to models included in the hybrid model 500 that have been trained using both supervised (e g., using known labels for the packet payloads) and unsupervised learning, respectively, and provide the output of the supervised and unsupervised models to an ensemble model configured to combine the outputs into a final output. The unsupervised machine-learning model 514 (i.e., a machinelearning model trained using unsupervised training methods) includes an encoder machinelearning model (e.g., a variational autoencoder) configured to automatically extract one or more features from a packet payload 502 and encode the packet payload 502 into latent space 510 to generate a latent space representation 522 of the packet payload 502 based on the extracted features. For example, the encoder may. based on the packet payload 502, output a mean vector 518 and a variance vector 516 to map a Gaussian distribution in latent space representative of the packet payload 502. The unsupervised machine-learning model 514 also includes a Gaussian Mixture Model (GMM) 528 configured to generate a clustered latent space representation 524 and to detect network traffic represented in the latent space representation 522 that falls outside expected or learned patterns.

[0069] The unsupervised machine-learning model 514 may use the latent space representation 522 or the clustered latent space representation 524 to classify packet payloads 502 that have been input into the unsupervised model. For example, the unsupervised machine-learning model 514 may be trained using only packet payloads 502 that are known to be benign such that, after the unsupervised model is trained, a packet payload 502 input into the unsupervised machine-learning model 514 may be classified as anomalous based on its latent space representation compared to the latent space representation of known benign packet payloads.

[0070] The GMM 528 may be trained on the encoder’s latent space representation of one or more packets provided to the encoder, which may enable the GMM 528 to detect network traffic that falls outside learned patterns (e.g., falling outside of data known to be associated with benign network packets). For example, the GMM 528 may detect potentially anomalous data if, within latent space, the data is a distance away from data known to be associated with benign data in latent space that is above a pre-determined threshold.

[0071] Furthermore, based on the classification as well as a position in latent space of each packet payload 502, the model trained using unsupervised learning may generate a probability 530 for each packet payload 502 represented in the latent space representation 522 or the clustered latent space representation 524 where the probability indicates a probability that a packet payload 502 is anomalous. In some embodiments, the probability may be based, at least in part, on a distance in latent space of the packet payload 502 currently being input into the model from previously input packets known to be benign. In some embodiments, the encoder of the unsupervised machine-learning model 514 may use maximum mean discrepancy (MMD) or Kullback-Leibler (KL) Divergence to measure a difference between tw o probability distributions in latent space.

[0072] The supervised machine-learning model 520 (i.e., a machine-learning model trained using supervised training methods) may include a convolutional neural network (CNN) that may have been trained to classify network traffic (e.g., packet payload 502) using labeled datasets. For example, to train the supervised machine-learning model 520, a packet payload that has been pre-labeled as anomalous or benign may be input into the supervised machine-learning model 520 (e.g.. a deep neural network (DNN) or the CNN) to generate a classification of the packet payload (e.g., as anomalous or benign). The resulting classification is then compared to the label (e.g., ground truth) and the machine-learning model may then automatically make one or more adjustments to one or more aspects (e.g., one or more parameters in one or more nodes of each of the layers 534) of the supervised machine-learning model 520 responsive to the comparison of the predicted classification result and the provided label for a given packet payload.

[0073] After the unsupervised model and the supervised model have been trained, data (e.g., packet payloads 502 outside of a training data set) may be provided simultaneously to both the unsupervised model and the supervised model. Each of the unsupervised model and the supervised model may then generate respective probabilities 530, 532, each probability indicative of a probability that the packet payload 502 is anomalous. For example, the CNN may utilize a Softmax function configured to generate a confidence score representative of a confidence that the predicted classification of the CNN is correct. Each respective probability may then be provided to the logistic regression model 538 of the ensemble model 542, where the logistic regression model 538 may be configured to generate a final predicted probability 540 that the packet payload 502 is anomalous based on the probability 530 and the probability 532 of the unsupervised model and the supervised model, respectively.

[0074] Unsupervised dimensionality reduction and clustering can be used in tandem with one of the plurality of network packet payload supervised classification approaches to reduce false positives. For example, clustering in the latent space may be combined with a model trained using supervised learning to generate an ensemble model for network packet payload identification at low false positives. Accordingly, the hybrid model 500 effectively harnesses the strengths of both supervised and unsupervised learning techniques to minimize false positives. For example, the ensemble approach implemented by the hybrid model 500 combines a supervised learning algorithm, and an unsupervised autoencoder. The hybrid model 500 tackles known issues in anomaly detection algorithms. Supervised models may effectively recognize known patterns but may also suffer from higher rates of false positives, which refer to the incorrect flagging of "‘normal” or expected behavior as anomalous. In contrast, unsupervised models do not require labeled data and are effective at capturing a wide range of data distributions, but they may lack the pinpoint accuracy offered by supervised models. By combining these two machine-learning approaches into a single hybrid model 500, the hybrid model 500 may leverage the advantages of both supervised and unsupervised learning while mitigating their respective drawbacks. Furthermore, the unsupervised machine-learning model may output a latent space representation of the one or more packets that has fewer dimensions compared to conventional. The lower dimensional latent space may lead to a larger variance, but also reduces model complexity and may reduce or simplify calculations done for anomaly detection, which may faster processing of data using fewer clock cycles of processing circuitry to enable real-time anomaly detection. Furthermore, by combining the lower dimensional latent space with the supervised model component using the logical regression model 538 may reduce the variance generated by a lower dimensionality latent space.

[0075] In some embodiments, the encoder of the unsupervised machine-learning model 514 as well as the supervised machine-learning model 520 may be implemented in an FPGA, and the GMM of the supervised machine-learning model 520 as well as the logistic regression model 538 may be implemented in a CPU. Though discussed as being implemented on specific hardware, any part of the hybrid model 500 may be implemented in an FPGA, GPU. and / or CPU.

[0076] The hybrid model 500 may be configured for real-time anomaly detection in a network (e.g., a 5G network) and, in some embodiments, optimized for use in an FPGA. By balancing throughput and accuracy, the hybrid model 500 may enable live (i.e., realtime) operation on an FPGA and may comply with 5G's speed requirements. The variational autoencoder architecture may be adapted for network traffic analysis, its latent space trained with maximum mean discrepancy (MMD) for interpretability and identification of malicious traffic.

[0077] FIG. 6 is a flowchart of method 600 illustrating an operation of a security system (e.g.. security’ system 100 of FIG. 1) performed by a processor executing instructions stored on a computer-readable storage medium. At operation 602, the security system 100 receives first network data including one or more benign network packets. For example, the first network data may be a training data set used in an unsupervised machine-learning algorithm to generate a machine-learning model, such as an autoencoder or a variational autoencoder.

[0078] At operation 604, the security system 100 may train a variational autoencoder for classifying network traffic as anomalous or benign based, at least in part, on the first network data. For example, a variational autoencoder may automatically extract one or more features from data input into the variational autoencoder and encode the one or more features into a latent space representation. Accordingly, training data including similar characteristics (e.g., known benign network packets), may enable the autoencoder to leam a feature space w ithin the latent space known to be associated with benign packets.

[0079] At operation 606, the security system 100 may provide second network data including one or more network packets as input to the variational autoencoder to encode the second network data into a latent space representation of the network data and to classiy’ each of the one or more packets as anomalous or benign. For example, after the variational autoencoder is trained using the first network data, the variational autoencoder may encode the second network data into a latent space representation of the network data. The variational autoencoder may then classify’ the second network data based on the latent space representation. For example, the variational autoencoder may include classifiers (e.g., models or algorithms configured to classify) to classify the second network data as anomalous responsive to the second network data being a distance away, in latent space. from the data know n to be benign where the distance is greater than a pre-determined threshold. In some embodiments, the classification may be done by another algorithm or model, such as a GMM. Furthermore, in some embodiments, the variational autoencoder may generate a probability that a packet included in the second network data is anomalous. For example, the probability may be based, at least in part, on a distance, w ithin latent space, of a latent space representation of the packet from a latent space representation of packets known to be benign. The packet may be classified based on the probability that the packet is anomalous. For example, the packet may be classified as anomalous if the probability that the packet is anomalous is above a pre-determined threshold. At operation 608, the security system 100 may generate one or more three-dimensional plot graphs based, at least in part, on each of the classified network packets of the second network data and the latent space representation of the second netw ork data.

[0080] FIG. 7 A shows a latent space representation 700a of network packet pay loads represented in a three-dimensional plot graph, according to one or more embodiments of the disclosure. For example, the latent space representation 700a may be a result of a packet flow being input into the variational autoencoder 110 and encoded into latent space. In some embodiments, the latent space representations of network packet payloads may be modified using a clustering algorithm. For example, K-means clustering may be used on the network packet payloads represented in latent space representation 700b to cluster “normal” or benign packets closer together to visually separate the normal or benign packets from anomalous packet pay loads as shown in FIG. 7B.

[0081] FIG. 8 shows a three-dimensional plot graph 800 based, at least in part, on latent space representation of a stream of packets input into an autoencoder (e.g., variational autoencoder 210). For example, the black plots 808 may indicate anomalous packet payloads (e.g., packets represented in the three-dimensional plot graph with a high probability of being anomalous) while the white plots 804 may represent “normal” or benign packet payloads (e.g., packets represented in the three-dimensional plot graph with a low probability of being anomalous). The grey plots 806 may indicate packets represented in the three-dimensional plot graph 800 having a moderate (e.g., from about a 30% to about a 70% chance) probability of being anomalous.

[0082] By providing a three-dimensional plot graph 800 for display to a user, a user may identify and mitigate anomalous packet pay loads as part of a security network in a way not provided by conventional systems or methods. For example, a user (e.g., a network security operator) may be enabled to visually see how anomalous a given network packet is. Moreover, in some embodiments, the display device used to present the three-dimensional plot graph 800 may also enable the user to select (e.g., using a mouse or touch screen) on a network packet represented in the three-dimensional plot graph 800 to display further information pertaining to that packet. For example, a user may select one of the white plots 804 of the three-dimensional plot graph 800 and may be provided information 810 that includes IP source, IP destination, source port, destination port, hostname of the sender of the packet, and a percentage of probability that the selected plot is anomalous.

[0083] FIG. 9 illustrates an example graphical user interface (GUI) 900 that may be used to present information to a user via a display device. For example, the GUI 900 may show a three-dimensional plot graph (e.g., the three-dimensional plot graph 800) in window 904, data relating to packets represented in the three-dimensional plot graph in information window 906, and a network topology in window 908. The information window^ 906 may include all of the information 810 in FIG. 8 and any other information that may be related to a network packet. For example, as shown in FIG. 9, the information window 906 may include data corresponding to a packet payload for one or more packets represented in the three-dimensional plot graph.

[0084] The network topology in window 908 may include one or more nodes of a netw ork and may indicate nodes that may be affected by a selected plot in the three-dimensional plot graph. For example, affected nodes in the network topology that may be affected by one or more packets represented in the three-dimensional plot graph may be colored based on susceptibility, such as colored red to indicate that the node may be affected.

[0085] FIG. 10A show s an example three-dimensional plot graph 1008 of a latent space representation of a plurality of network packets encoded into latent space and predicted to be benign 1014 or anomalous 1016 according to one or more embodiments of the disclosure. FIG. 10B show s an example three-dimensional plot graph 1012 of a latent space representation of a plurality of netw ork packets encoded into latent space and known to be (e.g., pre-labeled as) benign 1018 and anomalous 1020.

[0086] Referring to both FIGS. 10A and 10B together, the predicted benign 1014 and predicted anomalous 1016 data plots representative of network packets input into a CNN may be compared to the known or true benign 1018 and true anomalous 1020 plots. The CNN may then automatically update one or more parameters of the CNN, based on the comparison, to train the CNN. It will be appreciated by those of ordinary skill in the art that functional elements of embodiments disclosed herein (e.g., functions, operations, acts, processes, and / or methods) may be implemented in any suitable hardware, software, firmware, or combinations thereof. FIG. 11 illustrates non-limiting examples of implementations of functional elements disclosed herein. In some embodiments, some or all portions of the functional elements disclosed herein may be performed by hardware specially configured for carrying out the functional elements.

[0087] FIG. 1 1 is a block diagram of circuitry 1 100 that, in some embodiments, may be used to implement various functions, operations, acts, processes, and / or methods disclosed herein. The circuitry 1100 includes one or more processors 1102 operably coupled to one or more data storage devices (sometimes referred to herein as "storage 1 104"). The storage 1104 includes machine-executable code 1 106 stored thereon, and the processors 1102 include logic circuitry 1108. The machine-executable code 1106 includes information describing functional elements that may be implemented by (e g., performed by) the logic circuitry 1108. The logic circuitry 1108 is adapted to implement (e.g., perform) the functional elements described by the machine-executable code 1106. The circuitry 1100, when executing the functional elements described by the machineexecutable code 1106, should be considered as special purpose hardware configured for carrying out functional elements disclosed herein. In some embodiments, the processors 1102 may be configured to perform the functional elements described by the machine-executable code 1106 sequentially, concurrently (e.g., on one or more different hardware platforms), or in one or more parallel process streams.

[0088] When implemented by logic circuitry71108 of the processors 1102, the machineexecutable code 1106 is configured to adapt the processors 1102 to perform operations of embodiments disclosed herein. For example, the machine-executable code 1106 may be configured to adapt the processors 1102 to perform at least a portion or a totality of the method 400 of FIG. 4 and / or the method 600 of FIG. 6.

[0089] The processors 1102 may include a general purpose processor, a special purpose processor, a central processing unit (CPU), a microcontroller, a programmable logic controller (PLC), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, other programmable device, or any combination thereof designed to perform the functions disclosed herein. A general-purpose computer including a processor is considered a special-purpose computer while the general-purpose computer is configured to execute functional elements corresponding to the machine-executable code 1106 (e.g., software code, firmware code, hardware descriptions) related to embodiments of the disclosure. It is noted that a general- purpose processor (may also be referred to herein as a host processor or simply a host) may be a microprocessor, but in the alternative, the processors 1102 may include any conventional processor, controller, microcontroller, or state machine. The processors 1102 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0090] In some embodiments, the storage 1104 includes volatile data storage (e.g.. random-access memory (RAM)), non-volatile data storage (e.g.. Flash memory, a hard disc drive, a solid state drive, erasable programmable read-only memory (EPROM), etc.). In some embodiments, the processors 1102 and the storage 1104 may be implemented into a single device (e.g.. a semiconductor device product, a system on chip (SOC). etc ). In some embodiments, the processors 11 2 and the storage 11 4 may be implemented into separate devices.

[0091] In some embodiments, the machine-executable code 1106 may include computer- readable instructions (e.g., software code, firmware code). By way of non-limiting example, the computer-readable instructions may be stored by the storage 1104, accessed directly by the processors 1102, and executed by the processors 1102 using at least the logic circuitry 1108. Also by way of non-limiting example, the computer-readable instructions may be stored on the storage 1104, transferred to a memory device (not shown) for execution, and executed by the processors 1102 using at least the logic circuitry 1108. Accordingly, in some embodiments, the logic circuitry 1108 includes electrically configurable logic circuitry 1108.

[0092] In some embodiments, the machine-executable code 1106 may describe hardware (e.g., circuitry) to be implemented in the logic circuitry 1108 to perform the functional elements. This hardware may be described at any of a variety of levels of abstraction, from low-level transistor layouts to high-level description languages. At a high-level of abstraction, a hardware description language (HDL) such as an IEEE Standard hardware description language (HDL) may be used. By way of non-limiting examples, VERILOG®, SYSTEMVERILOG™ or very large scale integration (VLSI) hardware description language (VHDL™) may be used.

[0093] HDL descriptions may be converted into descriptions at any of numerous other levels of abstraction as desired. As a non-limiting example, a high-level description can be converted to a logic-level description such as a register-transfer language (RTL), a gatelevel (GL) description, a layout-level description, or a mask-level description. As a non- limiting example, micro-operations to be performed by hardware logic circuits (e.g.. gates, flip-flops, registers, without limitation) of the logic circuitry 1108 may be described in an RTL and then converted by a synthesis tool into a GL description, and the GL description may be converted by a placement and routing tool into a layout-level description that corresponds to a physical layout of an integrated circuit of a programmable logic device, discrete gate or transistor logic, discrete hardware components, or combinations thereof. Accordingly, in some embodiments, the machine-executable code 1106 may include an HDL, an RTL, a GL description, a mask level description, other hardware description, or any combination thereof.

[0094] In embodiments where the machine-executable code 1106 includes a hardware description (at any level of abstraction), a system (not shown, but including the storage 1104) may be configured to implement the hardware description described by the machine-executable code 1106. By way of non-limiting example, the processors 1102 may include a programmable logic device (e.g., an FPGA or a PLC), and the logic circuitry 1108 may be electrically controlled to implement circuitry corresponding to the hardware description into the logic circuitry 1108. Also by way of non-limiting example, the logic circuitry 1108 may include hard-wired logic manufactured by a manufacturing system (not shown, but including the storage 1104) according to the hardware description of the machine-executable code 1106.

[0095] Regardless of whether the machine-executable code 1106 includes computer- readable instructions or a hardware description, the logic circuitry 1108 is adapted to perform the functional elements described by the machine-executable code 1106 when implementing the functional elements of the machine-executable code 1106. It is noted that although a hardware description may not directly describe functional elements, a hardware description indirectly describes functional elements that the hardware elements described by the hardware description are capable of performing. As used in the disclosure, the terms ‘‘module” or “component” may refer to specific hardware implementations configured to perform the actions of the module or component and / or software objects or software routines that may be stored on and / or executed by general purpose hardware (e.g., computer-readable media, processing devices, etc.) of the computing system. In some embodiments, the different components, modules, engines, and services described in the disclosure may be implemented as objects or processes that execute on the computing system (e.g., as separate threads). While some of the system and methods described in the disclosure are generally described as being implemented in software (stored on and / or executed by general purpose hardware), specific hardware implementations or a combination of software and specific hardware implementations are also possible and contemplated.

[0096] As used in the disclosure, the term “combination” with reference to a plurality of elements may include a combination of all the elements or any of various different subcombinations of some of the elements. For example, the phrase “A, B, C, D, or combinations thereof may refer to any one of A, B, C, or D; the combination of each of A, B, C. and D; and any subcombination of A. B, C. or D such as A. B, and C; A. B, and D; A, C, and D; B, C, and D; A and B; A and C; A and D; B and C; B and D; or C and D.

[0097] Terms used in the disclosure and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including, but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes, but is not limited to,” etc.).

[0098] Additionally, if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and / or “an” should be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,’’ without other modifiers, means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” or “one or more of A, B, and C, etc.” is used, in general such a construction is intended to include A alone, B alone. C alone, A and B together, A and C together, B and C together, or A. B, and C together, etc.

[0099] Further, any disjunctive word or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” should be understood to include the possibilities of “A” or “B” or “A and B.”

[0100] Additional, non-limiting example embodiments of the disclosure are set forth below.

[0101] Embodiment 1 : An apparatus comprising: an input terminal configured to receive a plurality of network packets; and processing circuitry implementing a variational autoencoder trained to encode the plurality7of network packets into a latent space representation of the plurality of network packets, the processing circuitry configured to: provide the plurality of network packets to the variational autoencoder to generate a latent space representation of the plurality of network packets; provide the latent space representation of the plurality of network packets to a clustering model to classify each packet of the plurality of netw ork packets based on the latent space representation of the plurality of network packets; and generate a three-dimensional plot graph based, at least in part, on the classification of each of the plurality of network packets and the latent space representation of one or more network packets.

[0102] Embodiment 2: The apparatus of Embodiment 1, wherein the processing circuitry7is further configured to: cluster, via the clustering model, two or more data points of the latent space representation of the one or more netyvork packets based, at least in part, on like characteristics of the two or more data points.

[0103] Embodiment 3: The apparatus of Embodiment 2, wherein the two or more data points are clustered via one or more clustering algorithms including one or more of k- means clustering, hierarchical clustering, mean shift clustering, agglomerative clustering, Ordering Points to Identify the Clustering Structure (OPTICS), spectral clustering, affinity propagation, fuzzy C-means clustering, DBSCAN, Expectation-Maximization (EM), or Gaussian mixture models.

[0104] Embodiment 4: The apparatus of Embodiments 1-3, wherein the processing circuitry is further configured to provide the three-dimensional plot graph to a user via a display of a display device.

[0105] Embodiment 5: The apparatus of Embodiments 1-4. wherein the three-dimensional plot graph presents three-dimensional data points representative of the plurality of network packets, the three-dimensional data points visually distinguishing between network packets classified as anomalous and network packets classified as benign.

[0106] Embodiment 6: The apparatus of Embodiments 1-5. wherein the plurality of network packets are classified based, at least in part, on a pay load of each network packet of the plurality of network packets.

[0107] Embodiment 7 : The apparatus of Embodiments 1 -6, wherein the plurality of network packets are encoded into the latent space representation of the plurality of network packets based, at least in part, on a payload of each network packet of the plurality of network packets.

[0108] Embodiment 8: The apparatus of Embodiments 1-7, wherein the processing circuitry is configured to perform dimension reduction on the latent space representation to reduce the dimensions of the latent space representation to three dimensions.

[0109] Embodiment 9: The apparatus of Embodiments 1-8, wherein the processing circuitry is configured to associate payload data or classification data with each plot point of the three-dimensional plot graph.

[0110] Embodiment 10: The apparatus of Embodiment 9. wherein each plot point of the three-dimensional plot graph receives one or more visually distinguishing characteristics based, at least in part, on the payload data or classification data associated with each plot point.

[0111] Embodiment 11 : The apparatus of Embodiment 10, wherein each plot point of the three-dimensional plot graph associated with a classification of benign is colored green and each plot point of the three-dimensional plot graph associated with a classification of anomalous is colored red.

[0112] Embodiment 12: The apparatus of Embodiments 1-10, wherein the processing circuitry is in a Field Programmable Gate Array (FPGA). Embodiment 13: A method of operating a security system, the method comprising: receiving first network data including one or more benign network packets; training a variational autoencoder for classifying network traffic as anomalous or benign based, at least in part, on the first network data; providing second network data including one or more netw ork packets as input to the variational autoencoder to encode the second network data into a latent space representation of the second network data and to classify each of the one or more network packets as anomalous or benign; and generating one or more three- dimensional plot graphs based, at least in part, on each of the classified network packets of the second network data and the latent space representation of the second network data.

[0113] Embodiment 14: The method of Embodiment 13, comprising clustering two or more data points represented in the latent space representation of the second network data based, at least in part, on one or more characteristics of one or more of the two or more data points.

[0114] Embodiment 15: The method of Embodiment 13 or Embodiment 14, further comprising presenting the one or more three-dimensional plot graphs to a user using an electronic display device; selecting a plot point of the one or more three-dimensional plot graphs responsive to a user selection of the plot point via a user input device; and presenting, via the electronic display device, information corresponding to the selected plot point.

[0115] Embodiment 16: The method of Embodiment 15, wherein the information corresponding to the selected plot point includes a probability that a packet payload associated with the selected plot point is anomalous.

[0116] Embodiment 17: The method of Embodiment 15 or Embodiment 16, wherein the information corresponding to the selected plot point further includes one or more of IP source, IP destination, and the payload contents.

[0117] Embodiment 18: The method of Embodiment 15, wherein the information corresponding to the selected plot point includes a network topology7map indicating one or more nodes in the network topology that is potentially affected by a packet payload associated with the selected plot point.

[0118] Embodiment 19: A security system comprising: a destination device configured to receive one or more user inputs via a user input device in connection with the destination device; an electronic display in connection with the destination device; a network interface configured to receive a stream of packets from a network; and processing circuitry configured to: provide the received stream of packets to a variational autoencoder model and a convolutional neural network model to generate a first probability and a second probability, respectively, each of the first probability and the second probability indicative of a probability that a packet payload associated with the selected plot point is anomalous; provide the first probability and the second probability to a logical regression model to generate a third probability indicative of a probability for each packet of the stream of packets that the packet is benign; and generate a three-dimensional plot graph representative of a latent space representation of the received stream of packets via the variational autoencoder model wherein each plot point of the three-dimensional plot graph is representative of a packet of the received stream of packets.

[0119] Embodiment 20: The security system of Embodiment 19, wherein the processing circuitry is configured to: present the three-dimensional plot graph to a user using the electronic display; select a plot point of the three-dimensional plot graph responsive to a user selection of the plot point via a user input device; and present, via the electronic display, information corresponding to a packet payload associated with the selected plot point.

[0120] Embodiment 21 : The security system of Embodiment 19 or Embodiment 20, wherein the information corresponding to the packet payload includes a probability that a packet payload associated with the selected plot point is anomalous.

[0121] While the disclosure has been described herein with respect to certain illustrated embodiments, those of ordinary skill in the art will recognize and appreciate that the present invention is not so limited. Rather, many additions, deletions, and modifications to the illustrated and described embodiments may be made without departing from the scope of the invention as hereinafter claimed along with their legal equivalents. In addition, features from one embodiment may be combined with features of another embodiment while still being encompassed within the scope of the invention as contemplated by the inventor.

Claims

CLAIMSWhat is claimed is:1 . An apparatus comprising: an input terminal configured to receive a plurality of network packets; and processing circuitry implementing a variational autoencoder trained to encode the plurality of network packets into a latent space representation of the plurality’ of network packets, the processing circuitry configured to: provide the plurality’ of network packets to the variational autoencoder to generate a latent space representation of the plurality of network packets; and provide the latent space representation of the plurality of network packets to a clustering model to classify each packet of the plurality of network packets based on the latent space representation of the plurality of network packets; generate a three-dimensional plot graph based, at least in part, on the classification of each of the plurality of network packets and the latent space representation of one or more network packets.

2. The apparatus of claim 1, wherein the processing circuitry' is further configured to: cluster, via the clustering model, two or more data points of the latent space representation of the one or more network packets based, at least in part, on like characteristics of the two or more data points.

3. The apparatus of claim 2, wherein the two or more data points are clustered via one or more clustering algorithms including one or more of k-means clustering, hierarchical clustering, mean shift clustering, agglomerative clustering, Ordering Points to Identify the Clustering Structure (OPTICS), spectral clustering, affinity propagation, fuzzy C-means clustering, DBSCAN, Expectation-Maximization (EM), or Gaussian mixture models.

4. The apparatus of claim 1, wherein the processing circuitry' is further configured to provide the three-dimensional plot graph to a user via a display of a display device.

5. The apparatus of claim 4, wherein the three-dimensional plot graph presents three-dimensional data points representative of the plurality of network packets, the three- dimensional data points visually distinguishing between network packets classified as anomalous and network packets classified as benign.

6. The apparatus of claim 1, wherein the plurality of network packets are classified based, at least in part, on a payload of each network packet of the plurality of network packets.

7. The apparatus of claim 1, wherein the plurality of network packets are encoded into the latent space representation of the plurality of network packets based, at least in part, on a payload of each network packet of the plurality of network packets.

8. The apparatus of claim 1, wherein the processing circuitry is configured to perform dimension reduction on the latent space representation to reduce the dimensions of the latent space representation to three dimensions.

9. The apparatus of claim 1, wherein the processing circuitry is configured to associate payload data or classification data with each plot point of the three-dimensional plot graph.

10. The apparatus of claim 9, w herein each plot point of the three-dimensional plot graph receives one or more visually distinguishing characteristics based, at least in part, on the payload data or classification data associated with each plot point.

11. The apparatus of claim 10, wherein each plot point of the three-dimensional plot graph associated with a classification of benign is colored green and each plot point of the three-dimensional plot graph associated with a classification of anomalous is colored red.

12. The apparatus of claim 1, wherein the processing circuitry is in a Field Programmable Gate Array (FPGA).

13. A method of operating a security system, the method comprising: receiving first network data including one or more benign network packets; training a variational autoencoder for classifying network traffic as anomalous or benign based, at least in part, on the first network data; providing second network data including one or more network packets as input to the variational autoencoder to encode the second network data into a latent space representation of the second network data and to classify each of the one or more network packets as anomalous or benign; and generating one or more three-dimensional plot graphs based, at least in part, on each of the classified network packets of the second network data and the latent space representation of the second network data.

14. The method of claim 13, further comprising: clustering two or more data points represented in the latent space representation of the second network data based, at least in part, on one or more characteristics of one or more of the two or more data points.

15. The method of claim 13, further comprising: presenting the one or more three-dimensional plot graphs to a user using an electronic display device; selecting a plot point of the one or more three-dimensional plot graphs responsive to a user selection of the plot point via a user input device; and presenting, via the electronic display device, information corresponding to the selected plot point.

16. The method of claim 15, wherein the information corresponding to the selected plot point includes a probability that a packet payload associated with the selected plot point is anomalous.

17. The method of claim 1 , wherein the information corresponding to the selected plot point further includes one or more of IP source, IP destination, and the payload contents.

18. The method of claim 15, wherein the information corresponding to the selected plot point includes a network topology map indicating one or more nodes in the network topology that is potentially affected by a packet payload associated with the selected plot point.

19. A security system comprising: a destination device configured to receive one or more user inputs via a user input device in connection with the destination device; an electronic display in connection with the destination device; a network interface configured to receive a stream of packets from a network; and processing circuitry configured to: provide the received stream of packets to a variational autoencoder model and a convolutional neural network model to generate a first probability and a second probability, respectively, each of the first probability and the second probability indicative of a probability that a packet payload associated with a packet of the stream of packets is anomalous; provide the first probability7and the second probability7to a logical regression model to generate a third probability' indicative of a probability for each packet of the stream of packets that the packet is benign; and generate a three-dimensional plot graph representative of a latent space representation of the received stream of packets via the variational autoencoder model wherein each plot point of the three-dimensional plot graph is representative of a packet of the received stream of packets.

20. The security system of claim 19, yvherein the processing circuitry is configured to: present the three-dimensional plot graph to a user using the electronic display; select a plot point of the three-dimensional plot graph responsive to a user selection of the plot point via a user input device; and present, via the electronic display, information corresponding to a packet payload associated with the selected plot point.

21. The security system of claim 20, wherein the information corresponding to the packet pay load includes a probability that a packet payload associated with the selected plot point is anomalous.

Images