Treating data flows differently based on level of interest

Abstract

To provide a cyber threat defense platform which analyzes a cyber security threat using artificial intelligence.SOLUTION: A traffic manager module of a cyber threat defense platform 100 that can differentiate between data flows to a client device. The traffic manager module includes: a registration module that registers a connection between devices within a client network to transmit a series of data packets; a classifier module that executes a comparison of features of the connection to a set of interest criteria to determine an interest level for the cyber threat defense platform in the connection, and applies an interest classifier describing the interest level to the connection based on the comparison; and a deep packet inspection engine that examines the data packets of the connection for cyber threats if the interest classifier indicates interest.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Copyright Notice A portion of this disclosure contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by others of the patent file or records as it appears in the Patent and Trademark Office of the United States, but otherwise reserves all copyrights whatsoever.

[0002] Related Applications This application claims priority and the benefit under 35 U.S.C. § 119 to U.S. Provisional Patent Application No. 62 / 983,307, filed on February 28, 2020, entitled "An Artificial Intelligence Based Cyber Security System", and U.S. Provisional Patent Application No. 63 / 078,092, filed on September 14, 2020, entitled "An Intelligent Cyber Security System", the entire contents of which are incorporated herein by reference.

[0003] The embodiments of the designs provided herein generally relate to a cyber threat defense platform. In one embodiment, the cyber threat defense platform may selectively perform deep packet inspection at a client device to identify irregular data connections.

Background Art

[0004] In a cybersecurity environment, firewalls, endpoint security laws, and other tools such as security information and event management systems (SIEMs), and restricted environments such as sandboxes are deployed to enforce certain policies and provide protection against certain threats. These tools currently form an important part of an organization's cyber defense strategy, but they are insufficient in the face of new-era cyber threats.

[0005] Cyber threats, including e-mail threats, viruses, Trojan horses, and worms, are sophisticated and can quickly harm a network. In addition, human users can cause further damage to the system through malicious acts. A cyber security system must identify each of these cyber threats as they unfold.

Summary of the Invention

[0006] A traffic manager module of a cyber threat defense platform that can distinguish data flows to client devices. The registration module is configured to register connections between one or more devices within a client network and transmit a series of one or more data packets. The classifier module is configured to perform a comparison between the characteristics of the connection and a set of criteria of interest to determine the degree of interest of the cyber threat defense platform in the connection. The classifier module is further configured to apply an interest classifier that explains the degree of interest in the connection based on the comparison. The deep packet inspection engine is configured to test one or more data packets of the connection for cyber threats when the interest classifier indicates interest. The diverting device is configured to divert one or more data packets of the connection away from the deep packet inspection engine when the interest classifier indicates no interest.

[0007] These and other features of the design provided herein can be better understood with reference to the drawings, the specification, and the claims, all of which form the disclosure of this patent application.

Brief Description of the Drawings

[0008] The drawings refer to some embodiments of the design provided herein.

[0009]

Figure 1

[0010]

Figure 2

[0011]

Figure 3

[0012]

Figure 4

[0013]

Figure 5

[0014]

Figure 6

[0015]

Figure 7

[0016]

Figure 8

[0017]

Figure 9

[0018]

Figure 10

[0019]

Figure 11

[0020]

Figure 12

[0021]

Figure 13

[0022]

Figure 14

[0023]

Figure 15

[0024]

Figure 16

[0025]

Figure 17

[0026]

Figure 18

[0027]

Figure 19

[0028]

Figure 20

[0029]

Figure 21

[0030]

Figure 22

[0031]

Figure 23

[0032]

Figure 24

[0033] This design is subject to various modifications, equivalents, and alternative forms, but those specific embodiments are shown by examples in the drawings and will be described in more detail hereinafter. It should be understood that this design is not limited to the specific embodiments disclosed, but rather is intended to cover all modifications, equivalents, and alternative forms using those specific embodiments.

Mode for Carrying Out the Invention

[0034] In the following description, to provide a thorough understanding of the present design, numerous specific details are set forth, such as specific data signals, named components, the number of servers within the system, and the like. However, it will be apparent to those skilled in the art that the present design may be practiced without these specific details. In other instances, well-known components or methods are not described in detail and are instead described in block diagrams to avoid unnecessarily obscuring the present design. Further, references may be made by specific numbers, such as the first server. However, references by specific numbers should not be construed literally as an order, but rather, the first server should be construed as being different from the second server. Accordingly, the specific details set forth are merely examples. Also, features implemented in one embodiment may be implemented in another embodiment if logically possible. The specific details may vary and are still intended to be within the spirit and scope of the present design. The term "coupled" is defined to mean connected either directly to a component or indirectly to a component through another component.

[0035] Generally, a cyber threat defense platform may use artificial intelligence to analyze cyber security threats. FIG. 1 illustrates a block diagram of one embodiment of a cyber threat defense platform having a cyber threat module that references a machine learning model trained on the normal behavior of network activities and user activities associated with the network. The cyber threat module determines threat risk parameters that take into account "what is the likelihood of one or more abnormal behaviors in a series of e-mail activities, network activities, and user activities under analysis" that are "deviations from normal harmless behavior" and thus are likely malicious behavior.

[0036] The cyber threat defense platform 100 can protect against cyber security threats from the e-mail system and its network. The cyber threat defense platform 100 includes i) a trigger module, ii) a collection module, iii) a data store, iv) a network module, v) an e-mail module, vi) a coordinator module, vii) a comparison module, viii) a cyber threat module, ix) a traffic manager module, x) a user interface module, xi) an analyzer module, xii) an autonomous response module, xiii) at least one input or output (I / O) port for securely connecting to other ports as needed, xiv) a first artificial intelligence model trained on the characteristics of vectors for malicious activities and related data, a second artificial intelligence model trained on e-mail, a third artificial intelligence model trained on potential cyber threats, a fourth artificial intelligence model trained on normal life patterns, and one or more artificial intelligence models each trained on different users, devices, system activities and interactions between entities within the system, and other aspects of the system, etc., one or more machine learning models, and xv) other similar components within the cyber threat defense platform, etc.

[0037] The trigger module can detect timestamped data indicating that one or more i) events and / or ii) alerts have occurred from I) abnormal or II) suspicious behavior / activities, and then trigger that something abnormal is happening. Thus, the collection module is triggered by i) abnormal behavior, ii) suspicious activities, and iii) specific events and / or alerts of any combination of both. In-line data can be collected onto the deployment from the data store when traffic is observed. The range and wide variety of data available at this location results in good quality data for analysis. The collected data is passed to the comparison module and the cyber threat module.

[0038] The collection module can consist of multiple automatic data collection devices, each looking at different aspects of the data, depending on specific hypotheses formed about the events and / or alerts to be analyzed. Data related to each type of possible hypothesis is automatically drawn from additional external and internal sources. Some data is drawn or obtained by the collection module for each possible hypothesis. A collaborative feedback loop occurs between the collection module, a probe module that monitors network and e-mail activities, a comparison module for comparing one or more models trained on different aspects of this process, and a cyber threat module for identifying cyber threats based on the comparison by the comparison module. The e-mail module is monitored, but similar modules can be applied to other communication systems, such as text messages and other potential vectors for malicious activities. Each hypothesis of typical threats, such as insider attacks by human users, inappropriate network behavior, or e-mail behavior, or malicious software or malware attacks, inappropriate network behavior, or e-mail behavior, can have various fulcrums of data and other metrics associated with its possible threat. Machine learning algorithms look at the relevant points of the data and support or refute specific hypotheses about what suspicious activities or abnormal behavior are related to each hypothesis about what suspicious activities or abnormal behavior are related. The network can have rich data and metrics that can be collected. The collection devices can then filter or aggregate large amounts of data into important or prominent features of the data. In one embodiment, the network module, the e-mail module, and the coordinator module can be part of the cyber threat module.

[0039] The probe module can be configured to collect probe data from probes deployed on a client device. The client device is a device operated by a user to interact with a network. The probe data describes any activity executed by the client device and managed by a network administrator associated with the network. The network-managed activity can be a network activity or an email activity. Further, the probe module can be divided into an email module and a network module. The probe module can be configured to collect input data from one or more probes deployed on one or more network devices that describes network-managed activities executed by the client device.

[0040] A network module that monitors network-managed activities and an email module that monitors emails can each feed their data to a coordinator module to correlate the causal relationships between these activities, and supply this input to a cyber threat module. The coordinator module can be configured to contextualize network data and email data to create a combined data set for analysis.

[0041] The cyber threat module may also use one or more machine learning models trained on cyber threats within the network. The cyber threat module may refer to models trained on the normal behavior of user activities, network activities, and email activities associated with the network. The cyber threat module can refer to these various trained machine learning models, as well as data from the network module, email module, and trigger module. The cyber threat module can determine threat risk parameters that take into account how a series of abnormal behaviors correlate to potential cyber threats and what the likelihood is of this series of one or more abnormal behaviors of the network activities and user activities under analysis being "outside of normal harmless behavior" and thus malicious behavior.

[0042] One or more machine learning models can be self-learning models that use unsupervised learning and are trained on the normal behavior of different aspects of the system, such as device activities and user activities associated with email. The self-learning model of normal behavior is updated regularly. The self-learning model of normal behavior is updated when new input data is received and is considered within the range of normal behavior. The normal behavior threshold is used by the model as a varying benchmark for the parameters corresponding to the normal life patterns of the computing system. The normal behavior threshold varies according to the updated changes within the computer system and enables the model to distinguish behaviors on the computing system that deviate from the parameters set by the varying benchmark.

[0043] The comparison module can compare the analyzed metrics with respect to user activities and network activities, and the corresponding potential cyber threats, that are compared to their respective varying benchmarks of the parameters for the normal life patterns of the computing system used by the self-learning machine learning model.

[0044] The comparison module is configured to perform a comparison between input data and at least one machine learning model to identify behavior on the network that deviates from the normal benign behavior of network entities. The comparison module receives a combined data set from the coordinator module. The at least one machine learning model is trained on the normal benign behavior of network entities. The at least one machine uses a normal behavior benchmark that describes parameters corresponding to a normal pattern of activity of the network entity. The comparison module can use the comparison to identify whether the network entity is in a state of violating the normal behavior benchmark.

[0045] The comparison module can be integrated with the cyber threat module. The cyber threat defense platform 100 can also include one or more machine learning models trained to obtain an understanding of multiple features of the transmitted data and related data, including classifying the properties of the transmitted data and its metadata. The cyber threat module can then determine a cyber threat risk parameter indicating the likelihood of a cyber threat, according to the analyzed metrics and a varying benchmark of what is considered normal behavior.

[0046] The cyber threat module can also determine whether the network event or series of network events under analysis potentially have malicious characteristics, with reference to a machine learning model trained on network events and related data. The cyber threat module can also take this network event feature analysis into account in its determination of threat risk parameters. The cyber threat module can generate a set of event data here that describes irregular events by an entity representing a user or a device participating in the network. The cyber threat module can use this event data to determine whether the irregular event indicates a violation state representing a malicious event or a confidential data exposure. To do this, the cyber threat module can present the event data to the user analyst for verification using the user interface and display module. Alternatively, the cyber threat module can perform an autonomous analysis to determine whether an entity has entered a violation state using machine learning.

[0047] The cyber threat defense platform 100 can also include one or more machine learning models trained to obtain an understanding of multiple characteristics of related data, including classifying SaaS management events and properties of SaaS management events and their metadata.

[0048] Alternatively, the cyber threat module can perform an autonomous analysis to determine whether a network entity in a violation state is a cyber threat using machine learning. The cyber threat module is configured to identify whether a series of related behavior parameters deviating from the normal harmless behavior of the violation state and its network entity identified by the comparison module correspond to a cyber threat.

[0049] The cyber threat defense platform 100 may use multiple machine learning models. Each machine learning model may be trained on specific aspects of the normal life patterns of the system, such as the output from one or more cyber security analysis tools that analyze devices, users, network traffic flows, and systems. One or more machine learning models may also be trained on the characteristics and aspects of all patterns of types of cyber threats. One or more machine learning models may also be trained by observing vectors for malicious activities, such as network activity or e-mail.

[0050] The cyber threat defense platform 100 may have a traffic manager module to distinguish data flows and determine which data flows should be tested. A data flow is a connection to one or more devices within a physical or virtualized client network that transfers a series of one or more data packets. The traffic manager module may be within a host-based agent within the client network, such as on a laptop used within the client network. Alternatively, the traffic manager module may be within a virtualized sensor installed as a stand-alone virtual machine or on a hypervisor that receives packets by span or traffic mirroring. Further, the traffic manager module may be within a centralized cyber security device that may be physical or virtual. The cyber threat defense platform can receive the packet data itself by span or traffic mirroring while in communication with any host-based agent and virtual sensor probe.

[0051] The cyber threat defense platform 100 can supplement data provided to users and cyber experts who use analyzer modules to monitor various connections between client devices within a network. The analyzer module can flag client devices that are locations of irregular events. The analyzer module can be configured to flag client devices for host-based traffic decryption. In host-based traffic decryption, a host-based agent can decrypt one or more data packets for a connection at the client device. The analyzer module can be configured to determine that a client device permits host-based traffic decryption based on at least one of endpoint rarity, timing rarity, domain rarity, and environment. The host-based agent can perform decryption by at least one of receiving a private key from a third-party agent, uploading a public / private key pair from the client network to a configuration page, or obtaining a private key from the client device.

[0052] The cyber threat defense platform 100 can then take actions to counter the detected potential cyber threats.

[0053] The self - response module is configured to take actions pre - approved by a human user in order to autonomously attempt to counter malicious threats. Here too, rather than the human taking the action, the self - response module is configured to trigger one or more self - actions to be taken to block cyber threats when potential cyber threats are detected. A cyber security device may have a self - response module with a user - programmable interface. The user - programmable interface hosted on a cyber security device having any of i) a field, ii) a menu, and iii) an icon is described to enable the user to pre - approve the self - response module to take actions to block cyber threats. The user - programmable field / menu enables the user to pre - approve the module to take actions such as killing individual processes, revoking specific permissions, preventing the download of specific files, allowing only processes observed in the life pattern of peer devices that should be active over a certain set period, requesting to quarantine files suspected by other endpoint protection platforms (EPPs), etc., without interfering with the operation of other processes proceeding inside the device. The user interface has granularity in the options available for the user to program the self - response module to take very specific actions such as killing individual processes, revoking specific permissions while still allowing other permissions for that user, obtaining live terminal access, preventing the download of specific files, allowing only processes observed in the life pattern of peer devices that should be active over a certain set period, requesting to quarantine files suspected by other EPPs, etc., without shutting down the entire device or blocking all external communications, or revoking one or more but not all of the user's privileges.By actions such as only depriving a user of some user privileges or forcing a peer lifestyle pattern, while the user is performing normal activities such as typing a document or entering data into a program without being aware of malicious software using the user's authentication information at all, it is very likely that malicious software has started, such as accessing and downloading a confidential file, while preventing the execution of a specific connection or a specific process, but the user can continue working.

[0054] Examples of autonomous actions available for pre - approval by a human user for the autonomous response module are actions such as preventing or delaying activities related to threats; isolating or semi - isolating people, processes, devices; feeding threat intelligence to the EPP as well as the endpoint detection and response (EDR) processes and devices to take third - party or vendor - specific actions such as isolation or firewall blocking; terminating irregular processes on client devices, etc.; and, in many cases, including a general prompt to the user on the display screen of the endpoint computing device, along with actions such as not interfering with the user's normal daily activities or other processes on the endpoint computing device.

[0055] The self - response module can be configured to trigger one or more rapid self - actions to be taken to prevent a cyber - threat when the threat - risk parameters from the cyber - threat module, which are not a human taking an action, are equal to or greater than a threshold value that can serve as a guideline for the action. The cyber - threat module, configured in cooperation with the self - response module to trigger one or more self - actions to be taken to prevent a cyber - threat, improves by responding to the cyber - threat without waiting for any human intervention, thereby limiting the impact of the cyber - threat on computing devices in an e - mail system from consuming unapproved CPU cycles, memory space, and power consumption in the computing devices.

[0056] The self - response module can tag a specific user to have a lower threshold for self - response according to the cyber - threat situation. For example, the highest financial responsible person can cause great damage to the company by conducting financial transactions to embezzle funds. When the cyber - threat module identifies a cyber - threat to the financial function, the self - response module can lower the threshold for self - response when it identifies the tagged user associated with the cyber - threat. The self - response module can simultaneously employ several different clustering methods, including matrix - based clustering, density - based clustering, and hierarchical clustering techniques, to identify which users should be tagged with which threat types.

[0057] The cyber - threat defense platform 100 can be hosted on a device, on one or more servers, or within its own cyber - threat appliance platform.

[0058] FIG. 2 illustrates a block diagram of an example of a series of embodiments related to other networks under analysis of abnormal behavior for a network entity.

[0059] The user interface can display a graph 200 of a series of abnormal behaviors that serve as an example for the SaaS application in relation to other networks under analysis.

[0060] The cyber threat module collaborates with one or more machine learning models. The one or more machine learning models infer, for cyber threat analysis, "what could happen due to a series of entirely different alerts and / or events resulting from abnormal patterns", and then are trained using a mathematical algorithm and configured separately therewith to assign threat risks associated with those entirely different items of a series of alerts and / or events that form the abnormal pattern.

[0061] This is a "behavior pattern analysis" of what abnormal behavior is in a network, system, device, user, or network entity such as an e-mail under the analysis by a cyber threat module and a machine learning model. The cyber defense system uses abnormal behavior that deviates from normal behavior, and then constructs a series of abnormal behaviors and the causal relationships between this series of abnormal behaviors to detect cyber threats. The behavior pattern analysis, which is an example of what abnormal behavior is, can be as follows. The abnormal pattern can be determined by filtering out activities, events, or alerts that fall within the window of what is a normal life pattern for that network entity under analysis. Then, the remaining behavior patterns of activities, events, or alerts can be analyzed after filtering to determine whether the pattern indicates the behavior of a malicious party such as a human, program, e-mail, or other threat. The defense system can go back to some of the normal activities that were filtered out and draw them out in order to help support or refute the hypothesis of whether the pattern might indicate the behavior of a malicious party. The example behavior patterns included in the chain are shown in a graph over a time frame of, for example, 7 days. The defense system detects three irregular behaviors in a series of abnormal data transfers, and three abnormal features in e-mails within the monitored system that appear to have some causal relationship to the abnormal data transfer. Similarly, two abnormal authentication information attempts at abnormal behavior of trying to access an information focus area, or malicious IP addresses and users associated with abnormal authentication information that tries abnormal behavior, have a causal relationship with at least one of those three e-mails with abnormal features. When the behavior pattern analysis of individual behaviors or the behavior pattern analysis of the chain as a group is considered to indicate a malicious threat, a score is created of how confident the defense system is in the assessment of whether the abnormal pattern was caused by a malicious party.Next, what is further assigned is a threat level parameter (e.g., a score or probability) indicating the level of threat that this malicious party poses to the system. Finally, for different types of cyber threats that are equal to or greater than the configurable levels of threats, the cyber threat defense platform can be configured within its user interface regarding what types of automated response actions, if any, the defense system can take when posed by this malicious party.

[0062] The cyber threat module can connect individual alerts and events that form different items for the cyber threat analysis of a series of completely different alerts or events from the abnormal patterns. The cyber threat module can identify similar features from individual alerts or events that form different items composed of a series of alerts or events forming abnormal patterns by referring to one or more machine learning models trained for email threats.

[0063] One or more machine learning models can also be trained for all styles of features and aspects of the types of cyber threats to analyze the threat risks associated with a series or group of alerts or events forming abnormal patterns. Machine learning techniques using advanced mathematics can detect previously unidentified threats without relying on predetermined rules and automatically defend the network.

[0064] This model can be implemented by detecting probabilistic changes in normal behavior through the application of a Bayesian mathematical model without a teacher to detect behavioral changes in computers and computer networks. The core threat detection system is called "Bayesian probability". The Bayesian probability approach can determine periodicity in multiple time series data and identify changes across single or multiple time series data for the purpose of detecting irregular behavior. A large number of metrics can be obtained from emails and network sources of data, each resulting in time series data for a given metric.

[0065] Detectors within the cyber threat module, including the probe module and any SaaS module components, can be discrete mathematical models that perform specific mathematical methods on a set of different variables using a target. Thus, each model is targeted, for example, at i) its cyber security analysis tool, ii) analyzing various aspects of third-party SaaS interactions, iii) the life patterns of alerts and / or events brought about by specific devices and / or users within the system.

[0066] Fundamentally, the cyber threat defense platform mathematically characterizes what constitutes "normal" behavior based on the analysis of a large number of different measures of the network behavior of devices / a set of different measures. The cyber threat defense platform can build a sophisticated "life pattern" that understands what represents normalcy for all people, devices, email activities, and network activities within the system being protected by the cyber threat defense platform.

[0067] As discussed, each machine learning model can be trained on specific aspects of the system's normal life pattern, such as a device, a user, network traffic flow, the output from one or more cybersecurity analysis tools that analyze the system, the e-mail contact relevance of each user, e-mail characteristics, and others. One or more machine learning models can use at least an unsupervised learning algorithm to establish what the system's normal life pattern is. The machine learning model can be trained on both i) the past normal distribution of alerts and events for that system, and ii) the normal distribution information from a similar peer system to establish the normal life pattern of the behavior of alerts or events for that system. Another set of machine learning models is trained on the characteristics of the SaaS application and the activities and behaviors of the SaaS application users to establish normalcy for these.

[0068] The model can detect anomalies by leveraging at least two different approaches: for example, comparing the behavior of each system to its own history and comparing that system to the history of its peers, or comparing an e-mail to both the e-mail characteristics and the activities and behaviors of its e-mail user. This comparison of multiple sources enables the model to avoid learning existing bad behaviors as "normal behaviors" since entities subject to security breaches, such as devices, users, components, e-mails, etc., exhibit behaviors different from their neighboring peers.

[0069] In addition, one or more machine learning models can use a comparison between i) a normal behavior pattern for the system corresponding to past normal delivery of alerts and events for the system mapped within the same multi-dimensional space, and ii) the current set of individual alert and event behaviors under analysis. This comparison can result in the detection of one or more abnormal patterns of behavior within the plotted individual alerts or events, enabling the detection of previously unidentified cyber threats as compared to simply finding cyber threats using predefined descriptive objects or signatures. Thus, even increasingly sophisticated malicious cyber threats that choose when to act to generate low-level alerts and events will still be detected even if they have not yet been identified by other methods of cyber analysis. These sophisticated malicious cyber threats can include malware, spyware, keyloggers, malicious links in emails, malicious attachments in emails, and others, as well as rogue internal information technology staff who are well aware of ways to not trigger any high-level alerts or events.

[0070] Plotting and comparison are ways that can filter out what is normal for the system and then focus the analysis on what is abnormal or non-normal for the system. Thereafter, for each hypothesis about what could happen with a series of abnormal events or alerts, the collection module gathers additional metrics from a data store that includes a pool of metrics originally considered "normal behavior" to support or refute each possible hypothesis about what could happen with this series of abnormal behavior under analysis.

[0071] Note that each individual alert or event within a series of alerts or events forming an abnormal pattern can exhibit sophisticated abnormal behavior. Thus, each alert or event can have a low threat risk associated with that individual alert or event. However, when analyzed as a different series or group of alert or event behaviors forming a series of abnormal patterns by one or more machine learning models, that different series of alerts or events can be determined to have a much higher threat risk here than any of the individuals and / or events within that chain.

[0072] In addition, modern cyberattacks can be of such severity and speed that a human response cannot occur quickly enough. Thanks to these advances in self-learning, machines can detect these emerging threats and deploy appropriate real-time responses to counter the most serious cyber threats.

[0073] The threat detection system has the ability to self-learn and detect normality in order to distinguish true anomalies, enabling organizations of all sizes to understand the behavior of users and machines on the organization's network at both the individual and group levels. Instead of using predefined descriptive objects and / or signatures, monitoring behavior means that more attacks can be detected in advance and very subtle signs of wrongdoing can be detected. Different from traditional legacy defenses, a particular attack type or new malware does not have to be seen first before it can be detected. The behavior-based defense approach behaviorally and mathematically models both machine, email, and human activities during and after a security breach in order to predict and understand today's increasingly sophisticated cyber attack vectors. Therefore, it is possible to computationally establish what is normal and then detect what is abnormal. In addition, machine learning uses probabilistic mathematics to constantly revisit assumptions about behavior. The unsupervised machine learning method of the cyber threat defense platform does not require training data with predefined labels. Instead, the unsupervised machine learning method can identify the main patterns and trends in the data without the need for human input.

[0074] The user interface and output module can also project individual alerts and / or events that form a chain onto a user interface having at least three dimensions: i) the horizontal axis of a time window, ii) the vertical axis of a scale indicating the threat risk assigned to each alert and / or event in the chain, and iii) a third dimension of different colors for similar features shared between the individual alerts and events that form the very different items of the chain. The different colors can be red, blue, yellow, or others. In the case of grayscale, the user interface can use different shades of gray, black, and white with potentially different hash patterns. These similarities between events or alerts in the chain can be, for example, the same device, the same user authentication information, the same group, the same source identifier, the same destination Internet protocol address, the same type of data transfer, the same type of abnormal activity, the same type of alert, the same unusual connection being made, the same type of event, or alerts or events resulting from others. It should be noted that when a human intellect visually views the projected pattern and the corresponding data, the human can ultimately determine whether a cyber threat is posed. Here too, the at least three-dimensional projection helps the human to more easily synthesize this information. The visualization on the user interface enables the human to see data that supports or disproves why these aggregated alerts or events may potentially be malicious for a cyber threat defense platform. Also, instead of generating a simple binary output of "malicious" or "harmless", the mathematical algorithms of the cyber threat defense platform result in an output indicating different degrees of potential security breaches.

[0075] Defense System Figure 3 illustrates an example cyber threat defense platform for protecting an example network. The example network of Figure 3 shows an example of a network of computer system 50 that uses a threat detection system. The system depicted by Figure 3 shows a simplified example provided to facilitate the description of the present invention. System 50 includes a first computer system 10 within a building, and the first computer system 10 uses a threat detection system for detection, thereby attempting to prevent threats to computing devices within its scope. The first computer system 10 includes three computers 1, 2, 3, a local server 4, and a multifunction device (MFD) 5 that provides printing, scanning, and fax capabilities to each of the computers 1, 2, 3. All of the devices within the first computer system 10 are communicatively coupled via a local area network (LAN) 6. As a result, all of the computers 1, 2, 3 can access the local server 4 via the LAN 6 and use the functions of the MFD 5 via the LAN 6.

[0076] The LAN 6 of the first computer system 10 is connected to the Internet 20, where the Internet 20 provides access to a number of other computing devices including a server 30 and a second computer system 40 to the computers 1, 2, 3. The second computer system 40 also includes two computers 41, 42 connected by a second LAN 43.

[0077] In this exemplary embodiment of the present invention, computer 1 on the first computer system 10 has a threat detection system and thus executes a threat detection method for detecting threats to the first computer system. Thus, computer 1 comprises a processor configured to execute the steps of the process described herein, a memory required to store information regarding the execution of this process, and a network interface for collecting the required information. This method will be described in detail hereinafter with reference to FIG. 3.

[0078] Computer 1 constructs and maintains a dynamic, ever-changing model of the "normal behavior" of each user and machine within system 10. This approach is based on Bayesian mathematics and monitors all interactions, events, and communications within system 10 - which computers are talking to which computers, files created, networks accessed.

[0079] For example, computer 2 is operated by a marketing employee based at the company's San Francisco office who regularly accesses the marketing network. Computer 2 is active approximately from 8:30 a.m. to 6:00 p.m. and typically communicates with machines within the company's U.K. office within the second computer system 40 from 9:30 a.m. to noon. This same employee virtually never accesses the employee time sheet and rarely connects to the company's Atlanta network and has no transactions in Southeast Asia. The threat detection system extracts all available information about this employee to establish the person's "life pattern", which is dynamically updated as further information is collected. The "normal" model is used as a fluctuating benchmark to enable the system to identify behavior on the system that appears to deviate from this normal life pattern, flag this behavior as an anomaly, and request further investigation.

[0080] A threat detection system is built to address the fact that today's attackers are becoming increasingly sophisticated. Attackers may "hide" within the system using normal software protocols to ensure that they avoid raising suspicion with end users, such as by slowing down the user's machine. Thus, any attack process will stop or "fall back" when the mouse or keyboard is used. However, more sophisticated attacks still attempt the opposite, hiding in memory by masquerading as normal processes and stealing CPU cycles only when the machine is active in an attempt to break through relatively simple security processes. These sophisticated attackers look for activities not directly associated with user input. Advanced persistent threat (APT) attacks typically have a very long mission window of weeks, months, or years, and such processor cycles may not be stolen so frequently as to affect machine performance. However, no matter how hidden and sophisticated an attack may be, it leaves a measurable difference in typical machine behavior, even if extremely small, during and around a security breach. This difference in behavior can be observed and acted upon in the form of Bayesian mathematical analysis used by a threat detection system installed on Computer 1.

[0081] Figure 4 illustrates in a block diagram the integration of a threat detection system with other network protections. The network generally has a firewall 402 as the first line of defense. The firewall 402 analyzes the packet headers on incoming network data packets to enforce network policies. The firewall 402 may be integrated with an intrusion prevention system (IPS) to analyze packet headers and payloads for the entire event. Internally, an identity management module 404 controls access to the network's users.

[0082] The network security module 406 can enforce practices and policies for the network as determined by a network administrator. The encryption module 408 can not only encrypt communications within the network, but also encrypt and decrypt communications between network entities and external entities. The antivirus or malware countermeasure module 410 can search packets for known viruses and malware. The patch management module 412 can ensure that the latest patches are applied to security applications within the network. The centralized logging module 414 can track both internal communications within the network and interactive communications with the network. The threat detection system can act as real-time threat intelligence 416 for the network. The real-time threat intelligence can interact with other defense components to protect the network.

[0083] The cyber defense self-learning platform uses machine learning technology. Machine learning technology that uses advanced mathematics can detect previously unidentified threats without rules and automatically defend the network. Note that today's attacks can be of such severity and speed that a human response cannot occur quickly enough. Thanks to these self-learning advancements, machines can now discover emerging threats and deploy appropriate real-time responses to counter the most serious cyber threats.

[0084] The cyber threat defense platform constructs a sophisticated "life pattern" that understands what represents normality for all people, devices, and network activities within the system protected by the cyber threat defense platform.

[0085] The threat detection system can self-learn to detect normality in order to distinguish true anomalies, enabling organizations of all sizes to understand the behavior of users and machines on the organization's network at both the individual and group levels. Instead of using predefined descriptive objects and / or signatures, monitoring behavior means that more attacks can be detected in advance and very subtle signs of wrongdoing can be detected. Different from traditional legacy defenses, a specific attack type or new malware does not have to be seen first before it can be detected. The behavior-based defense approach behaviorally and mathematically models both machine and human activities during and after a security breach in order to predict and understand today's increasingly sophisticated cyber-attack vectors. This approach can, therefore, computationally establish what is normal and then detect what is abnormal.

[0086] This intelligent system can make value judgments and perform higher-value, more considerate tasks. Machine learning requires the development of complex algorithms and a general framework for interpreting the results generated. However, when these approaches are applied correctly, they can facilitate machines making logical probability-based decisions and taking on considerate tasks.

[0087] Advanced machine learning is at the forefront of the battle against automated and human - caused cyber threats, overcoming the limitations of rule - and signature - based approaches. For example, machine learning learns what is normal within a network without relying on knowledge of previous attacks. Machine learning evolves at a scale of modern business complexity and diversity where every device and person is slightly different. Machine learning turns the attacker's innovation against the attacker by making any abnormal activity visible. Machine learning constantly revisits assumptions about behavior using probabilistic mathematics. Machine learning is always up - to - date and does not rely on human input. Utilizing machine learning in cyber security technology is difficult, but when implemented correctly, it is very powerful. Machine learning means that previously undetected threats can be detected even when their occurrence fails to trigger any rule set or signature. Instead, machine learning enables the system to analyze large data sets and learn the "life patterns" of what it sees.

[0088] Figure 5 illustrates the application of a cyber - threat defense platform that uses advanced machine learning to detect anomalous behavior. A normal pattern of behavior 510 can describe a set of user or device behaviors within a threshold level of occurrence, such as a 98% probability of occurrence based on previous behavior. Anomalous activity 520 can describe a set of user or device behaviors that exceed the threshold level of occurrence. The cyber - threat defense platform can initiate an autonomous response 530 to disrupt the anomalous activity without affecting normal behavior.

[0089] Machine learning can approximate some human capabilities to a machine. Machine learning can approximate thinking by forming judgments using past information and insights. Machine learning can act in real - time so that the system processes information immediately. Machine learning can self - improve by constantly challenging and adapting its machine - learned understanding of the model based on new information.

[0090] Therefore, unsupervised machine learning enables computers to recognize emerging threats without prior warnings or supervision.

[0091] Unsupervised Machine Learning Unsupervised learning solves problems without predefined labels. This enables the system to handle unexpected situations and accept uncertainty. Although the system does not always know the characteristics of the search target, it can independently classify data and detect persuasive patterns.

[0092] The unsupervised machine learning method of the cyber threat defense platform does not require training data with predefined labels. Instead, the unsupervised machine learning method can identify the main patterns and trends in the data without the need for human input. Unsupervised learning offers the advantage of enabling computers to discover previously unknown relationships beyond what the programmer already knows.

[0093] The cyber threat defense platform uses a unique implementation form of unsupervised machine learning algorithms to analyze network data on a large scale, intelligently handle the unexpected, and accept uncertainty. Instead of relying on the knowledge of past threats to know what to look for, the cyber threat defense platform can independently classify data and detect persuasive patterns that define what can be considered normal behavior. Any new behavior that deviates from this notion of "normality" may indicate a threat or security breach. The impact of unsupervised machine learning on cyber security by the cyber threat defense platform is transformative. Threats that would otherwise go undetected are identified, highlighted, prioritized in context, and can be isolated using these algorithms. The application of machine learning has the potential to provide full network visibility and a much higher detection level, ensuring that the network has internal defense mechanisms. Machine learning has the ability to learn when to execute an automated response to the most serious cyber threats, crushing ongoing attacks before they become a crisis for the organization.

[0094] This new mathematics not only identifies important relationships in the data but also quantifies the uncertainty associated with such inferences. Knowing and understanding this uncertainty allows many results to be grouped within a consistent framework based on Bayesian probability analysis. The mathematics behind machine learning is very complex and difficult to fully understand. Robust and reliable algorithms are developed with the scalability that enables their successful application to real-world environments.

[0095] Overview In one embodiment, the probabilistic approach of the cyber security cyber threat defense platform is based on the Bayesian framework. This enables the cyber threat defense platform to integrate a large number of weak signatures of potentially irregular network behavior to produce a single clear measure of how likely a network device is to be compromised. This probabilistic mathematical approach provides the ability to understand important information even when the target of the search is unknown among the network noise.

[0096] Threat Ranking Importantly, the approach of the cyber threat defense platform accounts for the inevitable ambiguity present in the data and differentiates between slightly different levels of evidence that different data may contain. Instead of generating a simple binary output of "malicious" or "harmless", the mathematical algorithm of the cyber threat defense platform results in an output indicating different degrees of potential security breaches. This output enables the system user to rank different alerts in a rigorous manner, prioritize those that require immediate action, and at the same time eliminate the numerous false positive problems associated with a rule-based approach.

[0097] At a fundamental level, the cyber threat defense platform mathematically characterizes what constitutes "normal" behavior based on the analysis of a large number of different measures of network behavior by devices. Such network behavior can include server access, data access, event timing, authentication information use, domain name server (DNS) requests, and other similar parameters. Each measure of network behavior is then monitored in real time to detect irregular behavior.

[0098] Clustering The behavior of a device must be analyzed within the context of other similar devices on the network so that it can accurately model what should be considered normal for the device. To achieve this, the cyber threat defense platform algorithmically identifies the natural groupings of devices, a task that is impossible to perform manually even on moderately sized networks, by leveraging the strengths of unsupervised learning.

[0099] To achieve as holistic a view as possible of the relationships within the network, the cyber threat defense platform simultaneously employs several different clustering methods, including matrix-based clustering, density-based clustering, and hierarchical clustering techniques. The resulting clusters are then used to inform the modeling of exemplary behavior for individual devices. Clustering analyzes behavior within the context of other similar devices on the network. The clustering algorithm identifies natural groupings of devices that are impossible to perform manually. Additionally, the cyber threat defense platform runs multiple different clustering methods simultaneously to inform the model.

[0100] Network Topology Also, any cyber threat detection system must recognize that the network is much more than the sum of its individual parts and that most of its meaning is contained in the relationships between its different entities. Additionally, any cyber threat defense platform must further recognize that complex threats can often induce minor changes within this network structure. To capture such threats, the cyber threat defense platform employs several different mathematical methods so that it can model multiple aspects of the network topology.

[0101] One approach is based on the iterative matrix method that reveals important connection structures within the network. In parallel with these, the cyber threat defense platform has developed an innovative application of models from the field of statistical physics that enables it to reveal the underlying irregular structure hidden within by modeling the "energy landscape" of the network.

[0102] Network Structure A further important issue in modeling the behavior of network devices and of the network itself is the high-dimensional structure of the problem, which involves a large number of potential predictor variables. Observing packet traffic and host activities within local area networks (LANs), wide area networks (WANs), and the cloud is difficult because both the inputs and outputs can include many interrelated features such as protocols, source and destination machines, log changes, rule triggers, and others. Learning a predictive function with a sparse and consistent structure is important for avoiding overfitting.

[0103] In this context, the cyber threat defense platform has adopted state-of-the-art large-scale computing approaches to learn the sparse structure within models of network behavior and connectivity based on applying L1-regularization techniques such as the Least Absolute Shrinkage and Selection Operator (LASSO) method. This assigns a convex optimization problem that can be efficiently solved and enables the discovery of the true relationships between different network components and events that can result in a simple model.

[0104] Recursive Bayesian Estimation By combining these multiple analyses of different scales of network behavior, the cyber threat defense platform takes advantage of the strength of recursive Bayesian estimation (RBE) through the implementation of Bayesian filters to generate a single comprehensive description of the state of each device.

[0105] Using RBE, the mathematical models of the cyber threat defense platform can continuously adapt in a computationally efficient manner as new information becomes available to the system. They continuously recalculate the threat level in light of new evidence to identify changing attack behaviors where traditional signature-based methods fail.

[0106] The innovative approach of the cyber threat defense platform to cybersecurity has pioneered the use of Bayesian methods for tracking changing device behaviors and computer network structures. The core of the mathematical modeling of the cyber threat defense platform is the determination of exemplary behavior enabled by a sophisticated software platform that enables its mathematical models to be applied in real time to new network data. The result is a system that can identify subtle variations in machine events within the computer network behavior history that may indicate a cyber threat or security breach.

[0107] The cyber threat defense platform uses mathematical analysis and machine learning to detect potential threats and enable the system to anticipate emerging risks. The cyber threat defense platform approach means that detection no longer relies on archives of previous attacks. Instead, attacks can be distinguished against the backdrop of an understanding of what represents normality within the network. No predefined rules are required, which enables the most likely insights and defenses against today's threats. In addition to its detection capabilities, the cyber threat defense platform can automatically create digital antibodies as an immediate response to the most threatening cyber incursions. The cyber threat defense platform approach performs both detection and protection against cyber threats. Pure unsupervised machine learning removes the reliance on signature-based approaches to cyber security that are not working. The technology of the cyber threat defense platform can become an essential tool for security teams seeking to understand the scale of their own networks, observe the level of activity, and detect potentially weak areas. These no longer need to be manually hunted out but are rather flagged by an automated system and ranked in terms of their significance.

[0108] Machine learning techniques are fundamental allies in defending systems from today's hacker and insider threats and in formulating responses to unknown ways of cyberattacks. It is an epoch-making step change in cyber security. Defense must start from within.

[0109] Example Method The threat detection system will be further described in more detail with reference to the process flow implemented by a threat detection system for the automatic detection of cyber threats through probabilistic changes in normal behavior by applying an unsupervised Bayesian mathematical model for detecting behavioral changes in computers and computer networks.

[0110] The core threat detection system is called "Bayesian probabilistic". Bayesian probability is a Bayesian system that automatically determines the periodicity in multiple time series data and identifies changes across single or multiple time series data for the purpose of detecting irregular behavior.

[0111] Figure 6 illustrates a flowchart of one embodiment of a method for modeling human, machine, or other activities. The cyber threat defense platform first ingests data from multiple sources (block 602). Raw data sources include raw network IP traffic obtained from Internet Protocol (IP) or other network test access points (TAPs) or switched port analyzers (SPANs); machine-generated log files; building access ("swipe card") systems; IP or non-IP data flowing through industrial control system (ICS) distributed networks; individual machine, peripheral, or component power usage; telecommunications signal strength; or machine-level performance data obtained from on-host sources such as central processing unit (CPU) usage, memory usage, disk usage, disk free space, network usage, and others, but are not limited to these.

[0112] The cyber threat defense platform obtains secondary metrics from raw data (block 604). From these raw data sources, multiple metrics can be obtained for a given metric, each generating time series data. The data is dumped into individual time slices. For example, the observed count can be counted per second, per 10 seconds, or per 60 seconds. These buckets can be combined at a later stage where it is necessary to provide a longer range of values for any multiple of the selected internal size. For example, if the selected fundamental time slice is 60 seconds in length, each metric time series stores a single value for that metric every 60 seconds, and then any new time series data for a fixed multiple of 60 seconds (120 seconds, 180 seconds, 600 seconds, etc.) can be calculated without loss of accuracy. The metrics are directly selected and fed into Bayesian probability by a low-order model, which reflects some inherent fundamental part of the data and can be obtained from the raw data using specific domain knowledge. The metrics obtained depend on the threats the system is looking for. To provide a secure system, the cyber threat defense platform generally obtains multiple metrics regarding a wide range of potential threats. Communications from components within a network that come into contact with known suspicious domains.

[0113] The actual specific metrics used are mostly inappropriate for a Bayesian probabilistic system as long as the metrics are selected. Metrics obtained from network traffic can include data such as the number of bytes of data entering or leaving a networked device per time interval, file access, commonality or rarity of communication processes, invalid Secure-Sockets Layer (SSL) certificates, failed authentication attempts, or e-mail access patterns.

[0114] When a transmission control protocol (TCP), user datagram protocol (UDP), or other transport layer IP protocol is used across an IP network, and when alternative internet layer protocols such as the internet control message protocol (ICMP) or internet group message protocol (IGMP) are used, knowledge of the structure of the protocol in use and basic packet header analysis can be utilized to generate additional metrics. Such additional metrics may originate from networked devices and include the number of multicasts per time interval, the number of internal link-local IP broadcast requests originating from networked devices, the size of the packet payload data, or the number of individual TCP connections made by a device, or data transferred by a device either as a combined total across all destinations or to any definable network range such as a single target machine or a specific network range.

[0115] In the case of IP traffic where the application layer protocol can be determined and analyzed, additional types of time series metrics can be defined, for example, as follows. These time series metrics can be, for example, the number of DNS requests generated per time interval by a networked device, either for any definable network range or in total; the number of Simple Mail Transfer Protocol (SMTP), Post Office Protocol (POP), or Internet Message Access Protocol (IMAP) logins or login failures generated by a machine per time interval; the number of Lightweight Directory Access Protocol (LDAP) logins or login failures generated; the data transferred via file sharing protocols such as Server Message Block (SMB), SMB2, File Transfer Protocol (FTP), or others; or may include logins to Microsoft Windows Active Directory, Secure Shell (SSH) or local logins to Linux or Unix-like systems, or other authentication systems such as Kerberos.

[0116] The raw data required to obtain these metrics can be collected from virtual switch implementations, cloud-based systems, or the communication devices themselves via passive fiber or copper connections to network internal switch devices. Ideally, the system receives copies of all communication packets to provide full coverage of the organization.

[0117] In the case of other sources, some domain-specific time series data is obtained, and each of these data is selected to reflect a completely different and distinguishable aspect of the underlying source of that data, which in a sense reflects the use or behavior of that system over time.

[0118] Many of these time series data are extremely sparse, with most data points equal to 0. Examples are an employee using a swipe card to access a building or part of a building, or a user logging into their workstation authenticated by a Microsoft Windows Active Directory server, which typically occurs a small number of times per day. Other time series data sets, such as the size of data moving in and out of an always-on web server, web server CPU utilization, or the power consumption of a copier, are more dense.

[0119] Regardless of the type of data, such time series data sets tend to have various patterns within the data that repeat at approximately regular intervals, whether originally generated as a result of distinct human behavior so as to exhibit periodicity, or originally generated as a result of automated computers or other systems. Further, such data can have many quite different but independent regular time periods that are evident within the time series.

[0120] The detector performs an analysis of secondary metrics (block 606). The detector is a discrete mathematical model that implements a specific mathematical method on different variable sets having a target network. For example, a Hidden Markov Model (HMM) may specifically look at the size and transmission time of packets between nodes. The detector is provided within a hierarchy that is a pyramid of models arranged roughly. Each detector model acts efficiently as a filter and passes its output to another model higher up in the pyramid. At the top of the pyramid is the Bayesian probability which is the final threat determination model. Each of the lower order detectors monitors different global attributes or "features" of the underlying network and / or computer. These attributes can be values over time for all internal computational features such as packet speed and shape, endpoint file system values, and TCP / IP protocol timing and events. Each detector is specialized to record different environmental factors based on a detector having an internal mathematical model such as an HMM and make a determination about them.

[0121] While a threat detection system can be configured to look for any potential threats, in practice, the system may continue to look for one or more specific threats depending on the network in which the threat detection system is being used. For example, the threat detection system can provide a way that known characteristics of the network, such as desired compliance and human resource policies, trigger when cooperating with a set of probabilities of anomalies resulting from the probabilistic decision output or a varying threshold. The heuristic is constructed using a complex series of weighted logical expressions that represent a regular expression with atomic objects obtained at runtime from the output of detectors that measure / tokenize data and local context information. These series of logical expressions are then stored in an online library and parsed in real time against the output from the scale / tokenize detector. An example policy can take the form of "alert when any employee (context information) subject to HR disciplinary situations accesses confidential information (heuristic definition) in an irregular manner (Bayesian probability output) when compared to previous behavior." In other words, different arrays of pyramids of detectors are provided to detect specific types of threats.

[0122] The analysis performed by the detector on the secondary metric then outputs the data in a form suitable for use with a model of normal behavior. As can be seen, the data is in a form suitable for comparison with the model of normal behavior and for updating the model of normal behavior.

[0123] The threat detection system calculates threat risk parameters (block 608) that indicate the likelihood of a threat using automated adaptive periodicity detection mapped to the analysis of the observed behavior's life pattern. This infers that a threat exists from a collected set of attributes that the threat itself has shown as a deviation from exemplary collective or individual behavior over time. The automated adaptive periodicity detection uses a period of time calculated such that the Bayesian probability is most appropriate within the observed network or machine. Further, the life pattern analysis identifies how a human or machine behaves over time, for example, when they typically start and end work. Since these models continuously adapt automatically, they are inherently more difficult to break than known systems. The threat risk parameter is the probability that a threat exists in a particular configuration. Alternatively, the threat risk parameter is a value indicating the presence of a threat, which is compared to one or more thresholds indicating the likelihood of the threat.

[0124] In practice, the step of calculating a threat involves comparing current data collected in relation to a user to a model of normal behavior of the user and system being analyzed. The current data collected can relate to a period in time, which can relate to a particular influx of new data or a particular time period from seconds to days. In some configurations, the system is configured to predict the expected behavior of the system. The predicted behavior is then compared to the actual behavior to determine whether a threat exists.

[0125] The system uses machine learning or artificial intelligence to understand what is normal inside a company's network and when something is not normal. The system then calls an automated response to crush cyberattacks until a human team can step in. This can include disconnecting connections, blocking the sending of malicious emails, preventing file access, preventing communication outside the organization, etc. This approach starts in the most surgical and controlled way possible to interrupt the attack without affecting the normal behavior of, for example, a laptop. As the attack escalates, the cyber threat defense platform can ultimately isolate devices to prevent broader harm to the organization.

[0126] To improve the accuracy of the system, the checks are performed to compare the current behavior of the user with that of related users, e.g., users within a single office. For example, if there is unexpectedly low-level activity from a user, this may not be due to abnormal activity from the user but rather a factor affecting the entire office. Various other factors can be considered to evaluate whether the abnormal behavior actually indicates a threat.

[0127] Finally, the cyber threat defense platform decides (block 610) based on threat risk parameters whether further action needs to be taken regarding the threat. After the probability of the threat existing is presented, a human operator can make this decision. Alternatively, an algorithm can make the decision, e.g., by comparing the determined probability with a threshold.

[0128] In one configuration, assuming a unique global input of Bayesian probabilities, a form of threat visualization is provided, which allows the user to view the threat landscape across all internal traffic, and do so without the need to know how those internal networks are structured or dense, and in such a way that a "universal" view is presented within a single compartment regardless of the size of the network. The topology of the network under scrutiny is automatically projected as a graph based on device communication relationships via an interactive 3D user interface. The projection can scale linearly to any node scale without prior seeding or skeleton definition.

[0129] Accordingly, the threat detection system described above implements a valid form of recursive Bayesian estimation to maintain a distribution over probabilistic state variables. This distribution is constructed from a complex set of low-level host, network, and traffic observations or "features". These features are recorded iteratively and processed in real-time on the platform. A proper representation of the relational information between entities within a generally dynamic system, such as an enterprise network, a living cell or social community, or indeed the entire Internet, is a probabilistic network that is topologically rewiring and semantically evolving over time. Learning a predictive function with a sparse and consistent structure in many high-order structured input / output problems, such as the observation of packet traffic and host activity within a distributed digital enterprise where both inputs and outputs can include tens of thousands to millions of related features (such as data transport, host-web-client dialogs, log changes, and rule triggers), poses the problem of the lack of a normal distribution. To overcome this, the threat detection system comprises a data structure that determines a rotating continuum rather than a step-wise method where iterative time cycles, such as work days, shift patterns, and other routines, are dynamically assigned. In this way, a non-frequentist architecture is provided for inferring and testing causal relationships between explanatory variables, observations, and feature sets. This enables an efficiently solvable convex optimization problem and results in a parsimonious model. In such a configuration, the threat detection process can be triggered by the input of new data. Alternatively, the threat detection process can be triggered by the absence of predictive data. In some configurations, the process can be triggered by the presence of an event that can serve as a pointer to a particular course of action.

[0130] The method and system are configured to be implemented by one or more processing components having any portion of software stored in a form executable on a computer-readable medium. The computer-readable medium can be non-transitory and can exclude wireless or other carrier waves. The computer-readable medium can be a physical computer-readable medium such as, for example, semiconductor or solid state memory, magnetic tape, removable computer floppy disk, random access memory (RAM), read only memory (ROM), rigid magnetic disk, and optical disks such as CD-ROM, CD-R / W, or DVD.

[0131] The various methods described above can be implemented by a computer program product. The computer program product can include computer code configured to instruct a computer to perform one or more of the various functions described above. A computer program and / or code for implementing such methods can be provided to an apparatus such as a computer on a computer-readable medium or a computer program product. In the case of a computer program product, a transitory computer-readable medium can include wireless or other carrier waves.

[0132] An apparatus such as a computer can be configured according to such code for performing one or more processes according to the various methods discussed herein.

[0133] FIG. 7 illustrates a flowchart of an embodiment of a method for identifying irregular events from network event data. A cyber threat defense platform can use a probe module configured to collect probe data from one or more probes deployed on one or more client devices (block 702). A network entity represents at least one of a user and a network device. The probe data can describe network managed activities SaaS activities by the network entity.

[0134] The cyber threat defense platform can use an email module configured to collect email data from an email service (block 704). The cyber threat defense platform uses a coordinator module to contextualize the email data from the email module using probe data from the probe module to create a combined data set for analysis (block 706). The cyber threat defense platform uses a cyber threat module configured to analyze the combined data set using at least one machine learning model to distinguish behavior on the network that deviates from normal harmless behavior. The at least one machine learning model is trained on normal harmless behavior of network entities. The at least one machine learning model uses a normal behavior benchmark as a benchmark for at least one parameter corresponding to a normal pattern of network activity to distinguish deviant behavior.

[0135] The cyber threat defense platform has a comparison module that compares a combined data set including third-party event data with at least one machine learning model to distinguish behavior on the network that deviates from normal harmless behavior of the network entity (block 710). The comparison module can identify whether the network entity is in a state of violation of the normal behavior benchmark (block 712). The cyber threat module can identify whether a series of related behavior parameters that deviate from the violation state and the normal harmless behavior of the network entity correspond to a cyber threat (block 714).

[0136] The cyber threat defense platform can use a user interface module configured to present a graphical representation of a cyber threat within a graphical user interface (block 716). The cyber threat defense platform can use an autonomous response module configured to select an autonomous response to take in response to a cyber threat (block 718). The autonomous response can be, for example, reducing the permissions of a network entity or disabling the user account of a network entity. The autonomous response module can send an alert of the cyber threat to an internal system administrator or a third-party operator along with a proposed response to the cyber threat (block 720). The autonomous response module can execute an autonomous response in response to a cyber threat (block 722).

[0137] Figure 8 illustrates third-party event data. The network event data represents various management events. The management event can be a login event 802 that describes a user logged in to a user account of an online application or service. The management event can be a failed login event 804 that describes a user's failure to log in to a user account of an online application or service. The management event can be a resource creation event 806 that describes the creation of a virtual instance of an online application. The management event can be a resource view event 808 that describes the viewing of a virtual instance of an online application. The management event can be a resource modification event 810 that describes the modification of a virtual instance of an online application. The management event can be a resource deletion event 812 that describes the deletion of a virtual instance of an online application. The management event can be a file upload event 814 that describes the upload of a file to an online application. The management event can be a file download event 816 that describes the download of a file from an online application. The management event can be a management action event 818 that describes an action at the management level for an online application.

[0138] The cyber threat defense platform can obtain management events using various methods. The network module can extract management events from client devices for each event. FIG. 9 illustrates a flowchart of one embodiment of a method for extracting data from a client device. The network module is configured to instruct one or more connectors to send a Hypertext Transfer Protocol Secure (HTTPS) event request to the client network (block 902). The HTTPS event request requests management events from the audit log of the client network. One or more connectors generate the HTTPS event request (block 904). One or more connectors send the HTTPS event request to the client network to request management events (block 906). The network module is configured to receive management events from one or more connectors in response to the event request (block 908). The network module is configured to obtain the metadata of the management events (block 910).

[0139] The self - response module can autonomously determine a response using the threat risk parameters generated by the cyber - threat module. FIG. 10 illustrates a flowchart of one embodiment of a method for identifying a self - response. A cyber - threat defense platform may have a cyber - threat module configured to generate threat risk parameters that enumerate a set of values describing aspects of a cyber - threat (block 1002). A cyber - threat defense platform may have a self - response module configured to generate a benchmark matrix having a set of benchmark scores (block 1004). The self - response module can identify tagged users associated with the cyber - threat (block 1006). The self - response module can lower a threshold for self - response when identifying tagged users associated with the cyber - threat (block 1008). The self - response module can compare the threat risk parameters with the benchmark matrix to determine a self - response (block 1010). The self - response module can determine a self - response based on the comparison (block 1012).

[0140] A cyber - threat defense platform can generate threat risk parameters to explain the relative risk of an irregular event. FIG. 11 illustrates a block diagram of threat risk parameters. The threat risk parameters may have a threat type 1102 that describes the type of threat identified, such as finance, management, information technology, production, or others. The threat risk parameters may have a confidence score 1104 indicating the likelihood of a violation, which explains the probability that a template entity is in a violated state. The threat risk parameters may have a severity score 1106 indicating the percentage by which a template entity in a violated state deviates from normal behavior, as represented by at least one model. The threat risk parameters may have a result score indicating the severity of the damage caused by the violation.

[0141] FIG. 12 illustrates a flowchart of one embodiment of a method for generating threat risk parameters. A cyber threat module can generate threat risk parameters that enumerate a set of values that describe aspects of a violation state (block 1202). The cyber threat module can identify threat types of cyber threats by using various clustering techniques and group the threats with other identified cyber threats (block 1204). The cyber threat module can generate a confidence score (block 1206). The cyber threat module can generate a severity score (block 1208). The cyber threat module can generate an outcome score (block 1210). The cyber threat module can input at least one of the confidence score, the severity score, and the outcome score into the threat risk parameters (block 1212).

[0142] FIG. 13 illustrates a block diagram of a benchmark matrix. A self-regulating response module, in conjunction with the cyber threat module, can input a varying benchmark into the benchmark matrix that is adaptable to the changing nature of both the network and threats to the network. The benchmark matrix can have a confidence benchmark 1302 indicating the likelihood of a violation, which, if exceeded, describes the probability that a template entity is in a violated state. The benchmark matrix can have a severity benchmark 1304 indicating the percentage, which, if exceeded, indicates that a template entity is in a violated state. The benchmark matrix can have an outcome benchmark 1306 indicating the severity of damage resulting from a violation state, which, if exceeded, indicates that immediate action should be taken. The self-regulating response module can adjust these benchmarks as more data is added and larger user inputs are received.

[0143] The self-response module can assign weights to each benchmark score to assign relative importance to each benchmark score and take into account the decision to send an inoculation notice. Similar to the benchmarks, these weights can evolve over time. For example, the benchmark matrix can have a reliability weight 1308 indicating the importance of the reliability benchmark, a severity weight 1310 indicating the importance of the severity benchmark, and a result weight 1312 indicating the importance of the result benchmark. Using these assigned weights, different deviations from the benchmark can have better results for the final decision to be sent and the inoculation notice.

[0144] Figure 14 illustrates a flowchart of an embodiment of a method for triggering an inoculation notice by comparing the analyzed input data with a benchmark. The self-response module can generate a benchmark matrix having a set of benchmark scores to determine a self-response (block 1402). The self-response module can input the benchmark scores into the benchmark matrix based on the data collected during the violation identification process (block 1404). The self-response module can assign weights to each benchmark score to assign relative importance to each benchmark score (block 1406).

[0145] Interest Classifier The cyber threat detection platform is configured to perform packet inspection by analyzing a subset of each possible connection. One approach to monitoring connections is to process and inspect all connection traffic to the client device. This approach may not be computationally wise because not all connections are "interesting" or can be parsed in their entirety. An alternative approach is to perform connection-specific deep packet inspection and processing. In this approach, a traffic manager module, such as within a host-based agent, virtualized sensor, centralized physical device, or centralized cloud device, can process connections differently based on how interested they are, how much information the cyber threat detection platform can parse from the protocol, and whether the security team wishes to view the connection. This approach branches or filters uninteresting connections to save computation, automatically decrypts or generates a packet capture (PCAP) for interesting connections, and parses only the connection metadata that does not present a very large value. The traffic manager module can "shunt" connections. In other words, the network card can stop processing the actual content of the connection and supply only metadata such as the number of packets or bytes, greatly reducing the computation time.

[0146] Connection-specific approaches can collaborate network cards that can not only process traffic but also be aware of the flow passing through the network cards such as volume, connection type, or protocol. Connection-specific approaches can divert connections based only on the usefulness of metadata, such as those with some encryption protocols, or based on the connection being a large connection where the connection is thought to be of no interest. Connection-specific deep packet inspection and processing approaches can also "undivert" connections when the connection becomes of interest. For example, a cyber threat defense platform can instruct the network card to start processing data again because the data connection is thought to be of interest. The cyber threat defense platform can identify connections of interest based on the connection being irregular within the context of the device's past behavior or device peer group behavior, multiple connections with similar characteristics across several devices increasing the overall irregularity, or subsequent actions changing the "interestingness" state of a connection that was initially of no interest. The autonomous action module can then end the connection, check if the connection is successfully blocked, or supply that information to a third-party firewall by obtaining data regarding the connection to a spoof reset (RST) packet. Connections of interest can also be irregular enough that a security system, decryption method, or team may desire to decrypt the data at the packet level within the packet capture. Computationally expensive and slow decryption is ensured for connections with the top investigative values. The deep packet inspection engine can then collect packet-level data so that the security team can do so.

[0147] FIG. 15 illustrates a block diagram of a physical traffic manager module. A centralized physical device may have a traffic manager module that uses a network card. The network card can register a connection from the network and transmit a series of one or more than one data packets. The network card can analyze one or more than one data packets to identify potential cyber threats. After the analysis by the network card, the network card can pass the data packets to a processor for decryption and processing. The processor can transmit any data to the network by passing the data to an offload module. The offload module can packetize the data and transmit the new data packets to the network.

[0148] The network card may have a registration module to register a connection between one or more than one devices within a client network and transmit a series of one or more than one data packets. The network card may have a classifier module configured to perform a comparison between the characteristics of the connection and a set of interest criteria to determine the degree of interest of the cyber threat defense platform in the connection. The classifier module can adjust the set of interest criteria based on a set of host parameters for the client network. The set of host parameters can be at least one of storage capacity, processing power, and network bandwidth. The classifier module can be configured to apply an interest classifier that describes the degree of interest in the connection based on the comparison. If the interest classifier indicates interest, the network card may have a deep packet inspection (DPI) module to test one or more than one data packets of the connection for cyber threats. If the interest classifier indicates no interest, the network card may have a diverting device configured to divert one or more than one data packets of the connection away from the deep packet inspection engine.

[0149] FIG. 16 illustrates a block diagram of a virtual traffic manager module. A host-based agent, hypervisor, or centralized cloud device may have a traffic manager module that uses a virtual network map module. The network map module can register connections from the network and transmit a series of one or more data packets. The network map module can analyze one or more data packets to identify potential cyber threats. After the analysis by the network map module, the network map module can pass the data packets to a processor for decoding and processing. The processor can transmit any data to the network by passing the data to an offload module. The offload module can packetize the data and transmit new data packets to the network.

[0150] The virtual network mapping module may have a registration module to register connections between one or more devices in the client network and to send a series of one or more data packets. The virtual network mapping module may have a classifier module configured to perform a comparison between the characteristics of the connection and a set of interest criteria to determine the degree of interest of the cyber threat defense platform in the connection. The classifier module can adjust the set of interest criteria based on a set of host parameters for the client network. The set of host parameters can be at least one of storage capacity, processing power, and network bandwidth. The classifier may be configured to apply an interest classifier that describes the degree of interest in the connection based on the comparison. If the interest classifier indicates interest, the virtual network mapping module may have a deep packet inspection (DPI) module to test one or more data packets of the connection for cyber threats. If the interest classifier indicates no interest, the virtual network mapping module may have a diverting device configured to divert one or more data packets of the connection away from the deep packet inspection engine.

[0151] Figure 17 illustrates a flowchart of an embodiment of a method for establishing interest criteria. The traffic manager module can receive a set of interest criteria from the analyzer module of the cyber threat defense platform (block 1702). The classifier module of the traffic manager module can determine a set of host parameters for the client device (block 1704). The classifier module can adjust the set of interest criteria based on the set of host parameters (block 1706). The classifier module can store the set of interest criteria for future use (block 1708).

[0152] FIG. 18 illustrates a flowchart of an embodiment of a method for handling a data connection with a deep packet inspection engine. A registration module of a traffic manager module can register a connection between one or more devices within a client network and transfer a series of one or more data packets (block 1802). A classifier module of the traffic manager module can perform a comparison between the characteristics of the connection and a set of criteria of interest to determine the degree of interest of the cyber threat defense platform in the connection (block 1804). The classifier module can determine that the connection is of interest based on at least one of the connection being a short-term connection, being decodable within a parameter set for the client device, or being outside of a normal connection pattern (block 1806). The classifier module can apply an interest classifier that describes the degree of interest as being of interest to the connection based on the comparison (block 1808). A diverta of the traffic manager module can pass one or more data packets of the connection to the deep packet inspection engine for further testing for cyber threats if the interest classifier indicates that it is of interest (block 1810). The deep packet inspection engine can collect a set of packet metadata for one or more data packets of the pass-through connection (block 1812). An offload module of the traffic manager module can send the set of packet metadata to an analyzer module of the centralized cyber threat defense platform when processing is performed outside of the centralized cyber threat defense platform (block 1814).

[0153] Figure 19 illustrates a flowchart of one embodiment of a method for diverting data connections beyond a deep packet inspection engine. A registration module of a traffic manager module can register a connection between one or more devices within a client network to transfer a series of one or more data packets (block 1902). A classifier module of the traffic manager module can perform a comparison between the characteristics of the connection and a set of criteria of interest to determine the degree of interest of the cyber threat defense platform in the connection (block 1904). The classifier module can determine that the connection is of no interest based on at least one of the connection being a long-term connection, not being decryptable within a parameter set for the client device, and being within a normal connection pattern (block 1906). The classifier module can apply an interest classifier that describes the degree of interest as being of no interest to the connection based on the comparison (block 1908). A diverting device of the traffic manager module can divert one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates no interest (block 1910). A processing module of the client device can collect a set of packet metadata for one or more data packets of the diverted connection (block 1912). An offloading module of the traffic manager module can send the set of packet metadata to an analyzer module of the centralized cyber threat defense platform when processing is performed outside the centralized cyber threat defense platform (block 1914). The classifier module can monitor at least one of the connection length and the payload size for the diverted connection (block 1916).

[0154] Figure 20 illustrates a flowchart of an embodiment of a method for handling a data connection with data packet drops. A traffic manager module can receive a passthrough connection within a deep packet inspection engine (block 2002). The deep packet inspection engine can identify packet drops within the passthrough connection (block 2004). A classifier module can apply an updated interest classifier that explains different degrees of interest based on the packet drops (block 2006). A diverter can divert a passthrough connection with packet drops away from the deep packet inspection engine (block 2008).

[0155] Figure 21 illustrates a flowchart of an embodiment of a method for handling a data connection during irregular events. A cyber threat module of a cyber threat defense platform can detect an irregular event at a client device (block 2102). A diverter of a traffic manager module can reconnect a diverted connection to a deep packet inspection engine upon detection of an irregular event at the client device (block 2104). An autonomous action module can obtain data regarding a connection to a spoof reset (RST) packet and check whether to terminate the connection or whether the connection is successfully blocked (block 2104). The autonomous action module can disconnect the connection upon detection by an analyzer module of an irregular event at the client device (block 2108). A classifier module of the traffic manager module can adjust a set of interest criteria based on the irregular event (block 2110).

[0156] Host-Based Decryption The cyber threat detection platform is configured to data - mine communication protocols using decryption in order to protect the network from cyber threats within this network using encrypted communication protocols. Encryption of Domain Name System (DNS) traffic and other protocols is increasing in demand as the security risks of plain - text protocols become prominent. Host - based traffic decryption approaches decryption in a Deep Packet Inspection (DPI) engine in three ways. First, a host - based agent can receive a private key from a third - party proxy or agency. Second, a host - based agent can upload a public / private key pair to a centralized device associated with the host - based agent via a Secure Shell console or other interface. Third, a host - based agent can obtain keys from client devices. Third - party proxies or agencies cooperate with a device - host - based cyber threat detection and response platform that uses a Universal Translator to instruct third - party systems and obtain data from them. Host - based key acquisition consists of two possible approaches: process memory acquisition or personal firewall proxy.

[0157] In the process memory acquisition approach, the host-based agent module observes when a new connection is opened on port 443 and notifies another module of the process that a connection has been opened. This process memory acquisition approach is not limited to this port; rather, secure transfers are made using port 443, which is the standard port for HTTPS traffic. Another module locates the memory for the process that opened the connection and scans the memory for patterns that could be encryption keys. The other module then passes this key through a secure system to the cyber threat detection platform, and this secure system sends the key and process information to the correct device and correct module, such as a DPI engine, where the private key can be matched to the observed traffic and decrypted. The personal firewall proxy mode is when the host-based agent acts as a personal firewall, and traffic is controlled per device. The host-based agent acts as a "man-in-the-middle" for the traffic to view the traffic in its decrypted form or to obtain the key before the key exits the device. The personal firewall proxy mode can use an endpoint agent that performs process analysis and acts as a man-in-the-middle proxy.

[0158] FIG. 22 illustrates a flowchart of one embodiment of a method for using host - based decoding for data connection on a client device. The analyzer module of the cyber threat defense platform can determine that the client device permits host - based traffic decoding (block 2202). When making this determination, the analyzer module may consider the rarity of the endpoint, the rarity of the timing, the rarity of the domain, or the environment. The analyzer module can flag the client device for host - based traffic decoding (block 2204). The deep packet inspection engine can perform decoding in a host - based agent (block 2206). The deep packet inspection engine can receive a private key from a third - party agent. Alternatively, the deep packet inspection engine on the host - based agent can transfer the encrypted traffic to a centralized cyber - security defense platform supplied with a public / private key pair, enabling the deep packet inspection engine located in that cyber - security defense platform to perform decoding and processing instead. Otherwise, the deep packet inspection engine can obtain a private key from the client network.

[0159] External Storage for Packet Data of Interest The cyber security defense platform can cooperate with storage devices such as databases on the Internet or private databases in the cloud to store packet data of interest for longer and more thorough analysis, and as a result, the amount of storage is minimized as a concern. The cyber security defense platform can store a large number of packets in storage compared to just the storage of the cyber security platform itself. Furthermore, since storage is not limited in storage size, storage can store packets over a longer duration. Additionally, storage can perform more computational artificial intelligence on the data due to external storage. The cyber security defense platform can write packet data processed by a deep packet inspection (DPI) engine to limited-duration storage for acquisition by an operator who wishes to conduct a detailed investigation. Due to the vast amount of data and deep packet inspection across the monitored network, only a limited amount of data tends to be stored. Therefore, packet data is retained based on relevance. Data can be configured for stage-based expiration based on relevance, where data of interest is stored in long-term storage and connections with a lower level of "relevance" expire sooner. Relevance can be derived from protocol irregularities, irregularities in connection factors (source, destination, timing, etc.), or any number of additional metrics. A user can identify packets having x-type metrics to be sent to external storage through the user interface. A classifier module can increase user input using a classifier for default identification of data packets of interest.

[0160] The performance of artificial intelligence-based inspection of individual byte-level data is too computationally expensive and slow when packet data is held within a centralized cybersecurity platform or on a virtualized probe, and requires performance by processes existing on the platform or probe. When stored externally in secure extended storage, the external infrastructure can be utilized to perform artificial intelligence analysis on the packet data without sharing computing processing power with the probe or cybersecurity platform. The external infrastructure can be a locally located virtual machine or machine learning microservice such as Amazon AWS Machine Learning. The analysis can include inspection of byte-level character strings for irregular use of a Transformer or other deep learning model.

[0161] Secure expansion of storage capacity, and some data types maintained within cybersecurity devices, can in most cases directly cooperate and communicate with a cybersecurity defense platform. The cybersecurity defense platform has a user interface on a display for interfacing with an end user. The cybersecurity defense platform securely connects to and communicates with separate external storage, whether physical or virtualized. For example, external storage, a cloud-based Simple Storage System (S3) bucket within the same Virtual Private Cloud (VPC) or within a virtual private cloud managed by the organization supplying the cybersecurity defense platform. A virtualized probe performing deep packet inspection directly writes connections of interest to this outside of the probe storage to be accessed by the connected cybersecurity defense platform via application programming interface calls, tunnels with multiple factor authentication, connections via connected probes, or additional communication methods. This virtualization can enable the probe to automatically scale as the traffic volume increases or decreases without data disruption.

[0162] An additional benefit of this approach is larger and longer-term storage. Additional storage clusters or blobs can be created to address demand. Options for standard cyber security defense platform deployment can utilize external packet storage within a managed VPC as an additional service. Packet data can also be queried via an application programming interface from compatible services such as intelligence tools within the customer network or can be searchable. Finally, smaller subsets of such data enable potential machine learning approaches that may not be feasible for larger data sets due to compute costs.

[0163] FIG. 23 illustrates a flowchart of one embodiment of a method for off-site storage of packet captures from a data connection to a client device. A deep packet inspection engine can collect a packet capture of one or more data packets for the connection (block 2302). The deep packet inspection engine can set an expiration time for the packet capture indicating when the packet capture can be overwritten within a cloud simple storage system (block 2304). The deep packet inspection engine can send the packet capture to the cloud simple storage system for storage (block 2306).

[0164] Website The website is configured as a browser-based tool or a direct collaboration app tool for configuring, analyzing, and communicating with a cyber threat defense platform.

[0165] Network Some electronic systems and devices can communicate with each other within a network environment. The network can include at least one firewall, at least one network switch, a plurality of computing devices operable by users of the network, a cyber threat coordinator component, and a host-based agent. FIG. 24 illustrates a networked environment in a schematic diagram. The network environment has a communication network. The network can include one or more networks selected from an optical network, a cellular network, the Internet, a local area network (“LAN”), a wide area network (“WAN”), a satellite network, a third-party “cloud” environment, a fiber network, a cable network, and combinations thereof. In some embodiments, the communication network is the Internet. There can be a number of server computing systems and a number of client computing systems connected to each other via the communication network.

[0166] The communication network can connect one or more server computing systems selected from at least a first server computing system and a second server computing system to each other and also to at least one or more client computing systems. Each server computing system can optionally include an organized data structure such as a database. Each of the one or more server computing systems can have one or more virtual server computing systems, and a plurality of virtual server computing systems can be implemented by design. Each of the one or more server computing systems can have one or more firewalls and similar defenses to protect data integrity.

[0167] At least one or two or more client computing systems, such as mobile computing devices (e.g., smartphones having an Android-based operating system), can communicate with a server. The client computing system can include, for example, a software application or a hardware-based system that can communicate with the first electric personal transportation vehicle and / or the second electric personal transportation vehicle in some cases. Each of the one or two or more client computing systems can have one or two or more firewalls and similar defenses to protect data integrity.

[0168] The cloud provider platform can include one or two or more of the server computing systems. The cloud provider can install and operate application software within the cloud (e.g., a network such as the Internet), and cloud users can access the application software from one or two or more of the client computing systems. Generally, a cloud user having a cloud-based site within the cloud cannot manage alone the cloud infrastructure or platform where the application software runs. Thus, the server computing system and its systematic data structure can be shared resources, and each cloud user is given exclusive use of a specific amount of the shared resources. The cloud-based site of each cloud user can be given an exclusive amount of virtual space and bandwidth within the cloud. Cloud applications can differ from other applications in scalability, which can be achieved by mimicking tasks on multiple virtual machines at runtime to meet varying workload requirements. The load balancer distributes the workload across a set of virtual machines. This process is transparent to cloud users who see only a single access point.

[0169] Cloud-based remote access can be coded to engage in request and response cycles by applications on a client computing system, such as a web browser application within a client computing system, using protocols such as the Hypertext Transfer Protocol ("HTTP"). Cloud-based remote access can be accessed at any time and / or from anywhere by a smartphone, desktop computer, tablet, or any other client computing system. Cloud-based remote access is coded to engage in 1) request and response cycles from all web browser-based applications, 3) request and response cycles from dedicated online servers, 4) request and response cycles directly between native applications within a client device and cloud-based remote access to another client computing system, and 5) combinations thereof.

[0170] In an embodiment, the server computing system can include a server engine, a web page management component, a content management component, and a database management component. The server engine can perform basic processing and operating system level tasks. The web page management component can handle the creation, display, or routing of web pages or screens associated with receiving and providing digital content and digital advertisements. A user (e.g., a cloud user) can access one or more of the server computing systems by their associated Uniform Resource Locator ("URL"). The content management component can handle most of the functions within the embodiments described herein. The database management component can include storage and retrieval tasks related to databases, queries to databases, and storage of data.

[0171] In some embodiments, the server computing system may be configured to display information in a window, web page, etc. For example, any program module, application, service, process, and other similar software including those executable when running on the server computing system can cause the server computing system to display a window and a user interface screen within a portion of the display screen space. With respect to a web page, for example, a user via a browser on a client computing system can interact with the web page and then supply input to queries / fields and / or services presented on the user interface screen. The web page can be served on a hypertext markup language ("HTML") or wireless access protocol ("WAP") compliant client computing system (e.g., client computing system 802B) or any equivalent thereof by a web server, e.g., the server computing system. The client computing system can host a browser and / or a specific application to interact with the server computing system. Each application has code described to implement functions such that software components perform presentation fields and the like to extract details of desired information. For example, algorithms, routines, and engines within the server computing system can extract information from presentation fields and place that information in a suitable storage medium such as a database (e.g., a database). A comparison wizard can refer to the database and be described to use such data. The application can be hosted, for example, on the server computing system and served to a specific application or browser of the client computing system, for example. The application then serves a window or page that enables entry of details.

[0172] Computing System The computing system can be, in whole or in part, one or more portions of a server or client computing device according to some embodiments. The components of the computing system can include, but are not limited to, a processing unit having one or more processing cores, a system memory, and a system bus that couples various system components including the system memory to the processing unit. The system bus can be any of several types of bus structures selected from a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.

[0173] The computing system typically includes various computer machine-readable media. The computer machine-readable media can be accessed by the computing system and can be any available media including both volatile and nonvolatile media, as well as removable and non-removable media. By way of example and not limitation, the use of computer machine-readable media includes storage of information such as computer-readable instructions, data structures, other executable software, or other data. Computer-storage media can be used to store desired information and can be accessed by the computing device 900, and includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic disk storage devices, or any other tangible medium. Transitory media such as a wireless channel are not included in machine-readable media. Communication media typically embodies computer-readable instructions, data structures, other executable software, or other transport mechanisms and includes any information delivery media.

[0174] The system memory includes computer storage media in the form of volatile and / or non-volatile memory such as read only memory (ROM) and random access memory (RAM). The basic input / output system (BIOS), which contains basic routines that help transfer information between elements within a computing system, such as during startup, is typically stored in the ROM. The RAM is typically immediately accessible and / or contains data and / or software currently being operated on by the processing unit. By way of example and not limitation, the RAM may contain an operating system, application programs, other executable software, and a portion of program data.

[0175] The drives and those associated computer storage media described above provide storage of computer readable instructions, data structures, other executable software, and other data of the computing system.

[0176] A user may input commands and information into the computing system via input devices such as a keyboard, touch screen, or software or hardware input buttons, a microphone, pointing device, and / or scrolling input components such as a mouse, trackball, or touchpad. The microphone can work in conjunction with speech recognition software. These and other input devices are often connected to the processing unit through a user input interface coupled to the system bus, but may be connected by other interfaces and bus structures such as a parallel bus, game port, or universal system bus (USB). A display monitor or other type of display screen device is also connected to the system bus via an interface such as a display interface. In addition to the monitor, the computing device may also include other peripheral device output devices such as speakers, vibrators, lighting, and other output devices that may be connected through an output peripheral device interface.

[0177] A computing system can operate in a networked environment using logical connections to one or more remote computers / client devices such as a remote computing system. The logical connections can include a personal area network ("PAN") (e.g., Bluetooth (R)), a local area network ("LAN") (e.g., Wi-Fi), and a wide area network ("WAN") (e.g., a cellular network), but can also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet. A browser application or a direct application that interacts with a cloud platform can be within a computing device and can be stored in memory.

[0178] Note that this design can be executed on a single computing system and / or on a distributed system where different parts of this design are executed on different parts of a distributed computing system.

[0179] The applications described in this specification include, but are not limited to, software applications, mobile apps, and programs that are part of operating system applications. Some portions of this description are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing art to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, considered to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, and otherwise manipulated. Sometimes, mainly for reasons of convenience, it has proven useful to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc. These algorithms can be written in several different software programming languages such as Python, C, C+, or other similar languages. Also, the algorithms can be implemented with lines of code within software, configured logic gates within software, or a combination of both. In embodiments, the logic consists of an electrical circuit that follows the rules of Boolean logic, software that includes a pattern of instructions, or any combination of both.

[0180] However, it should be noted that all of these terms and similar terms will be associated with appropriate physical quantities and are nothing more than convenient labels applied to these quantities. As is apparent from the above description, unless explicitly indicated otherwise, throughout the description, descriptions using terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" refer to the action and process of a computer system that operates on and transforms data represented as physical (electronic) quantities in the registers and memories of the computer system into other data similarly represented as physical quantities in the computer system memory or registers, or in other such information storage, transmission, or display devices, or a similar electronic computing device.

[0181] Many functions implemented by electronic hardware components can be replicated by software emulation. Thus, software programs written to achieve those same functions can emulate the functionality of the hardware components in the input-output circuit.

[0182] Although the foregoing design and its embodiments have been provided in considerable detail, it is not the applicant's intention that the designs and embodiments provided herein be limiting. Additional conforming and / or integrating is possible and, in a broad sense, these conforming and / or integrating are also encompassed. Thus, the foregoing design and embodiments can be prepared without departing from the scope resulting from the following claims, which scope is limited only by the claims when appropriately construed.

Claims

1. A method for a cyber threat defense system for distinguishing data flows, comprising: In a traffic manager module of the cyber threat defense system, registering a connection between one or more devices within a client network and transferring a series of one or more data packets; Performing a comparison between characteristics of the connection and a set of interest criteria to determine a degree of interest of the cyber threat defense system in the connection; Applying an interest classifier that explains the degree of interest in the connection based on the comparison; If the interest classifier indicates interest, passing the one or more data packets of the connection to a deep packet inspection engine for further testing for cyber threats; If the interest classifier indicates no interest, diverting the one or more data packets of the connection away from the deep packet inspection engine. A method for a cyber threat defense system.

2. The method for a cyber threat defense system according to claim 1, further comprising determining that the connection is uninteresting based on at least one of the connection being a long-term connection, not being decryptable within a parameter set for the client device, and being within a normal connection pattern.

3. The method for a cyber threat defense system according to claim 1, further comprising monitoring at least one of connection length and payload size for diverted connections.

4. The method for a cyber threat defense system according to claim 1, further comprising collecting a set of packet metadata for diverted connections.

5. Identifying packet drops within a pass-through connection being processed by the deep packet inspection engine; Diverting the pass-through connection with the packet drops away from the deep packet inspection engine; The method for a cyber threat defense system according to claim 1, further comprising.

6. The method for a cyber threat defense system according to claim 1, further comprising the step of disconnecting the connection when detected by an analyzer module for irregular events in the client device.

7. The method for a cyber threat defense system according to claim 6, further comprising the step of obtaining data regarding a connection to a spoof reset packet.

8. The method for a cyber threat defense system according to claim 1, further comprising the step of reconnecting a diverted connection to the deep packet inspection engine when detected by an analyzer module for irregular events in the client device.

9. The method for a cyber threat defense system according to claim 8, further comprising the step of adjusting the set of the criteria of interest based on the irregular events.

10. A non-transitory computer-readable medium comprising computer-readable code operable to instruct a computing device to perform the method according to claim 1 when executed by one or more processing devices in the cyber threat defense system.

11. A traffic manager module for a cyber threat defense system, a registration module configured to register a connection between one or more devices in a client network and transmit a series of one or more data packets; a classifier module configured to perform a comparison between characteristics of the connection and a set of criteria of interest to determine a degree of interest of the cyber threat defense system in the connection, and to apply an interest classifier explaining the degree of interest in the connection based on the comparison; a deep packet inspection engine configured to test the one or more data packets of the connection for cyber threats when the interest classifier indicates interest; and a diverting device configured to divert the one or more data packets of the connection away from the deep packet inspection engine when the interest classifier indicates no interest comprising a traffic manager module.

12. The traffic manager module according to claim 11, further configured such that the classifier module adjusts the set of interest criteria based on a set of host parameters for the client network.

13. The traffic manager module according to claim 12, wherein the set of host parameters is at least one of memory capacity, processing power, and network bandwidth.

14. The traffic manager module according to claim 11, wherein the deep packet inspection engine is configured to collect a packet capture of the one or more data packets for the connection.

15. The traffic manager module according to claim 14, further comprising an offload module configured to send the packet capture to a cloud storage system.

16. The traffic manager module according to claim 15, wherein the offload module is configured to set an expiration period for the packet capture within the cloud storage system indicating when the packet capture can be overwritten.

17. The traffic manager module according to claim 11, wherein the traffic manager module is located in at least one of a host-based agent, a virtualization sensor installed on a hypervisor, a centralized physical device, and a centralized cloud device.

18. A network comprising: at least one firewall; at least one network switch; a plurality of computing devices operable by a user of the network; a cyber threat coordinator component comprising: a probe module configured to collect input data describing network-administered activities performed by a network device from one or more probes deployed on the one or more network devices; a cyber threat module configured to identify whether the input data corresponds to a cyber threat to the network. A cyber threat coordinator component including an analyzer module configured to flag a host-based agent for host-based traffic decryption, A traffic manager module, A registration module configured to register a connection between one or more devices on the network and transfer a series of one or more data packets, A classifier module configured to perform a comparison between the characteristics of the connection and a set of interest criteria to determine the degree of interest of the cyber threat defense system in the connection and, based on the comparison, apply an interest classifier that explains the degree of interest in the connection, A deep packet inspection engine configured to test the one or more data packets of the connection for cyber threats and perform host-based traffic decryption of the one or more data packets for the connection when the interest classifier indicates interest, A traffic manager module including a diverting device configured to divert the one or more data packets of the connection away from the deep packet inspection engine when the interest classifier indicates no interest, A network comprising the same.

19. The network according to claim 18, wherein the analyzer module of the cyber threat coordinator component is configured to determine that the host-based agent permits host-based traffic decryption based on at least one of the rarity of the endpoint, the rarity of the timing, the rarity of the domain, and the environment.

20. The network according to claim 18, wherein the deep packet inspection engine of the host-based agent is configured to perform decryption by at least one of receiving a private key from a third-party agent, uploading a public / private key pair to a centralized device associated with the host-based agent, and obtaining a private key from the client network.

Images