Treat data flows differently based on their level of interest

Abstract

A traffic manager module of a cyber threat protection platform can differentiate data flows to a client device. A registration module can register connections between devices in a client network and transmit a series of data packets. A classifier module can perform a comparison of characteristics of the connection with a set of interest criteria to determine a level of interest to the cyber threat protection platform in the connection. The classifier module can apply an interest classifier that describes a level of interest to the connection based on the comparison. A deep packet inspection engine can test data packets of the connection for cyber threats if the interest classifier indicates interest. A diverter can divert data packets of the connection away from the deep packet inspection engine if the interest classifier indicates no interest.

Description

[Technical field]

[0001] Copyright Notice A portion of this disclosure contains material that is subject to copyright protection. The copyright owner has no objection to others copying of the copyrighted material, as it appears in the U.S. Patent and Trademark Office patent files or records, but otherwise reserves all copyrights whatsoever.

[0002] Related Applications This application claims priority under 35 U.S.C. § 119 to and the benefit of U.S. Provisional Patent Application No. 62 / 983,307, filed February 28, 2020, entitled "An Artificial Intelligence Based Cyber ​​Security System," and U.S. Provisional Patent Application No. 63 / 078,092, filed September 14, 2020, entitled "An Intelligent Cyber ​​Security System," each of which is incorporated by reference in its entirety herein.

[0003] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0002] Embodiments of the designs provided herein generally relate to a cyber threat protection platform. In one embodiment, the cyber threat protection platform can selectively perform deep packet inspection on client devices to identify anomalous data connections. [Background technology]

[0004] In the cybersecurity environment, firewalls, endpoint security methods, and other tools such as security information and event management systems (SIEMs), and restricted environments such as sandboxes are deployed to enforce specific policies and provide protection against specific threats. While these tools currently form a vital part of an organization's cyber defense strategy, they are inadequate in the new age of cyber threats.

[0005] Cyber ​​threats, including email threats, viruses, Trojans, and worms, are sophisticated and can quickly compromise networks. In addition, human users can cause further damage to systems through malicious actions. Cybersecurity systems must identify each of these cyber threats as they unfold. Summary of the Invention

[0006] A traffic manager module of the cyber threat protection platform capable of differentiating data flows to a client device. The registration module is configured to register a connection between one or more devices in the client network and transmit a sequence of one or more data packets. The classifier module is configured to perform a comparison of characteristics of the connection to a set of interest criteria to determine a degree of interest of the cyber threat protection platform in the connection. The classifier module is further configured to apply an interest classifier describing a degree of interest in the connection based on the comparison. The deep packet inspection engine is configured to test one or more data packets of the connection for cyber threats if the interest classifier indicates interest. The diverter is configured to divert one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates no interest.

[0007] These and other features of the designs provided herein can be better understood with reference to the drawings, specification, and claims, all of which form the disclosure of this patent application. [Brief description of the drawings]

[0008] The drawings refer to several embodiments of the designs provided herein.

[0009] [Figure 1]FIG. 1 is a block diagram of one embodiment of a cyber threat protection platform having a cyber threat module that references a machine learning model that is trained on normal behavior of network entities to identify cyber threats by identifying deviations from normal behavior.

[0010] [Diagram 2] FIG. 2 is a block diagram of one embodiment of an example series of non-normal behaviors of a network entity activity relative to other networks under analysis.

[0011] [Diagram 3] FIG. 3 is a diagram illustrating an example cyber threat defense platform for protecting an example network.

[0012] [Figure 4] FIG. 4 is a block diagram illustrating the integration of the threat detection system with other network protections.

[0013] [Diagram 5] FIG. 5 is a diagram illustrating the application of a cyber threat protection platform that uses advanced machine learning to detect anomalous behavior.

[0014] [Figure 6] FIG. 6 is a flow chart of one embodiment of a method for modeling human, machine, or other activity.

[0015] [Figure 7] FIG. 7 is a flow chart of one embodiment of a method for identifying cyber threats.

[0016] [Figure 8] FIG. 8 is a diagram illustrating third party event data.

[0017] [Figure 9]FIG. 9 is a flow chart of one embodiment of a method for extracting data from an online application.

[0018] [Figure 10] FIG. 10 is a flow chart of one embodiment of a method for identifying an autonomous response.

[0019] [Figure 11] Figure 11 is a block diagram of the threat risk parameters.

[0020] [Figure 12] FIG. 12 is a flow chart of one embodiment of a method for generating a threat risk parameter.

[0021] [Figure 13] Figure 13 is a block diagram of the benchmark matrix.

[0022] [Figure 14] FIG. 14 is a flow chart of one embodiment of a method for generating a benchmark matrix.

[0023] [Figure 15] Figure 15 is a block diagram of the Physical Traffic Manager module.

[0024] [Figure 16] Figure 16 is a block diagram of the virtual traffic manager module.

[0025] [Figure 17] FIG. 17 is a flow chart of one embodiment of a method for establishing interest criteria.

[0026] [Figure 18] FIG. 18 is a flow chart of one embodiment of a method for processing a data connection with a deep packet inspection engine.

[0027] [Figure 19] FIG. 19 is a flow chart of one embodiment of a method for diverting a data connection across a deep packet inspection engine.

[0028] [Figure 20] FIG. 20 is a flow chart of one embodiment of a method for handling a data connection with dropped data packets.

[0029] [Figure 21] FIG. 21 is a flow chart of one embodiment of a method for handling data connectivity during an anomalous event.

[0030] [Figure 22] FIG. 22 is a flow chart of one embodiment of a method for using host-based decryption for a data connection on a client device.

[0031] [Diagram 23] FIG. 23 is a flow chart of one embodiment of a method for off-site storage of packet captures from a data connection to a client device.

[0032] [Figure 24] FIG. 24 is a diagram illustrating an example network to be protected by a cyber threat protection platform.

[0033] While the present design is subject to various modifications, equivalents, and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will now be described in detail. It is to be understood that the present design is not limited to the specific embodiments disclosed, but rather is intended to cover all modifications, equivalents, and alternative forms that utilize the specific embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] In the following description, numerous specific details are set forth, such as specific data signals, named components, the number of servers in a system, etc., to provide a thorough understanding of the design. However, it will be apparent to one of ordinary skill in the art that the design may be practiced without these specific details. In other instances, well-known components or methods have not been described in detail, but rather in block diagram form, to avoid unnecessarily obscuring the design. Furthermore, specific numerical references may be made, such as a first server. However, the specific numerical references should not be construed as a literal sequence, but rather that a first server is different from a second server. Thus, the specific details set forth are merely examples. Also, features implemented in one embodiment may be implemented in another embodiment where logically possible. It may be contemplated that the specific details may be varied and still be within the spirit and scope of the design. The term coupled is defined to mean connected either directly to a component, or indirectly to a component through another component.

[0035] In general, a cyber threat protection platform may use artificial intelligence to analyze cyber security threats. Figure 1 illustrates a block diagram of an embodiment of a cyber threat protection platform having a cyber threat module that references a machine learning model trained on normal behavior of network activity and user activity associated with the network. The cyber threat module determines a threat risk parameter that takes into account "what is the likelihood of one or more non-normal behaviors of the set of email activity, network activity, and user activity under analysis that deviate from being normal, harmless behavior" and thus are likely to be malicious behavior.

[0036] The cyber threat protection platform 100 can protect against cyber security threats from email systems as well as their networks. The cyber threat protection platform 100 may include components such as: i) a trigger module, ii) a collection module, iii) a data store, iv) a network module, v) an email module, vi) a coordinator module, vii) a comparison module, viii) a cyber threat module, ix) a traffic manager module, x) a user interface module, xi) an analyzer module, xii) an autonomous response module, xiii) at least one input or output (I / O) port for securely connecting to other ports as needed, xiv) one or more machine learning models, such as a first artificial intelligence model trained on vector features for malicious activity and associated data, a second artificial intelligence model trained on email, a third artificial intelligence model trained on potential cyber threats, a fourth artificial intelligence model trained on normal life patterns, and one or more artificial intelligence models each trained on different users, devices, system activities and interactions between entities within the system, and other aspects of the system, and xv) other similar components within the cyber threat protection platform.

[0037] The trigger module may detect time-stamped data indicating one or more i) events and / or ii) alerts occurring from I) non-normal or II) suspicious behavior / activity and then trigger that something non-normal is happening. Thus, the collection module is triggered by specific events and / or alerts of i) anomalous behavior, ii) suspicious activity, and iii) any combination of both. Inline data may be collected on the deployment from a data store as traffic is observed. The range and wide variation of data available at this location results in good quality data for analysis. The collected data is passed to the comparison module and the cyber threat module.

[0038] The collection module may consist of multiple automated data collectors, each looking at different aspects of the data depending on the particular hypothesis formed about the event and / or alert being analyzed. Data related to each type of possible hypothesis is automatically pulled from additional external and internal sources. Some data is pulled or acquired by the collection module for each possible hypothesis. A feedback loop of cooperation occurs between the collection module, a probe module that monitors network and email activity, a comparison module for comparing one or more models trained on different aspects of this process, and a cyber threat module for identifying cyber threats based on the comparison by the comparison module. Although the email module is monitored, similar modules may be applied to other communication systems, such as text messages and other potential vectors for malicious activity. Each hypothesis of a typical threat, such as insider attacks of human users, improper network behavior, or email behavior, or malicious software or malware attacks, improper network behavior, or email behavior, may have various fulcrums of data and other metrics associated with that possible threat. The machine learning algorithm looks at relevant points in the data and for each hypothesis about what suspicious activity or anomalous behavior is associated, supports or disproves the particular hypothesis of what suspicious activity or anomalous behavior is associated with. The network may have a wealth of data and metrics that may be collected. The collector may then filter or aggregate the large amount of data into important or salient features of the data. In one embodiment, the network module, email module, and coordinator module may be part of a cyber threat module.

[0039] The probe module may be configured to collect probe data from a probe deployed on a client device. The client device is a device operated by a user to interact with a network. The probe data describes any activity performed by the client device and managed by a network administrator associated with the network. The network managed activity may be network activity or email activity. Further, the probe module may be divided into an email module and a network module. The probe module may be configured to collect input data describing the network managed activity performed by the client device from one or more probes deployed on one or more network devices.

[0040] The network module, which monitors network managed activity, and the email module, which monitors email, may each feed their data to a coordinator module to correlate causal relationships between these activities and provide this input to the cyber threat module. The coordinator module may be configured to contextualize the network data and the email data to create a combined data set for analysis.

[0041] The cyber threat module may also use one or more machine learning models that are trained on cyber threats within the network. The cyber threat module may reference models that are trained on normal behavior of user activity, network activity, and email activity associated with the network. The cyber threat module may reference these various trained machine learning models as well as data from the network module, email module, and trigger module. The cyber threat module may determine a threat risk parameter that takes into account how a set of non-normal behaviors correlate to potential cyber threats, and what the likelihood is of this set of one or more non-normal behaviors of the network activity and user activity under analysis that deviates from being normal, harmless behavior, and thus is malicious behavior.

[0042] The one or more machine learning models may be self-learning models that use unsupervised learning and are trained on normal behavior for different aspects of the system, e.g., device activity and user activity associated with email. The self-learning model of normal behavior is updated periodically. The self-learning model of normal behavior is updated when new input data is received and deemed within the range of normal behavior. Normal behavior thresholds are used by the model as a moving benchmark of parameters that correspond to normal life patterns for the computing system. The normal behavior thresholds move according to updated changes in the computer system, allowing the model to discern behavior on the computing system that falls outside the parameters set by the moving benchmark.

[0043] The comparison module can compare the analyzed metrics with respect to user activity and network activity, as well as corresponding potential cyber threats, compared to their respective varying benchmarks of parameters against normal life patterns for the computing system used by the self-learning machine learning model.

[0044] The comparison module is configured to perform a comparison between the input data and at least one machine learning model to discern behavior on the network that deviates from normal, harmless behavior of the network entity. The comparison module receives the combined data set from the coordinator module. The at least one machine learning model is trained on normal, harmless behavior of the network entity. The at least one machine learning model uses a normal behavior benchmark that describes parameters that correspond to normal patterns of activity for the network entity. The comparison module can use the comparison to identify whether the network entity is in violation of the normal behavior benchmark.

[0045] The comparison module may be integrated with a cyber threat module. The cyber threat protection platform 100 may also include one or more machine learning models trained on gaining an understanding of multiple characteristics of the transmitted data and associated data, including classifying properties of the transmitted data and its metadata. The cyber threat module may then determine cyber threat risk parameters indicative of a possible cyber threat according to the analyzed metrics and varying benchmarks of what is considered normal behavior.

[0046] The cyber threat module may also refer to a machine learning model trained on the network events and related data to determine whether the network event or series of network events under analysis has potentially malicious characteristics. The cyber threat module may also take this network event characteristic analysis into account in its determination of the threat risk parameter. The cyber threat module may generate a set of event data describing anomalous events by entities, here representing users or devices participating in the network. The cyber threat module may use this event data to determine whether the anomalous events indicate a malicious event or a breach state representing sensitive data exposure. To do this, the cyber threat module may use the user interface and display module to present the event data to a user analyst for review. Alternatively, the cyber threat module may execute an autonomous analyst to use machine learning to determine whether an entity has entered a breach state.

[0047] The cyber threat defense platform 100 may also include one or more machine learning models trained to gain an understanding of multiple features of SaaS management events and associated data, including classifying properties of the SaaS management events and their metadata.

[0048] Alternatively, the cyber threat module can execute an autonomous analyst to use machine learning to determine if a network entity in a breach state is a cyber threat. The cyber threat module is configured to identify whether the breach state identified by the comparison module and a set of associated behavioral parameters that deviate from normal, harmless behavior of the network entity correspond to a cyber threat.

[0049] The cyber threat protection platform 100 may use multiple machine learning models. Each machine learning model may be trained on a particular aspect of the normal life patterns of the system, such as devices, users, network traffic flows, and output from one or more cybersecurity analysis tools that analyze the system. The one or more machine learning models may also be trained on the characteristics and aspects of all manner of cyber threat types. The one or more machine learning models may also be trained by observing vectors for malicious activity, such as network activity or emails.

[0050] The cyber threat protection platform 100 may have a traffic manager module to distinguish data flows and determine which data flows should be tested. A data flow is a connection to one or more devices in a physical or virtualized client network that transfers a series of one or more data packets. The traffic manager module may be in a host-based agent in the client network, such as on a laptop used in the client network. Alternatively, the traffic manager module may be in a virtualized sensor installed as a standalone virtual machine or on a hypervisor that receives packets by span or traffic mirroring. Additionally, the traffic manager module may be in a centralized cyber security appliance, which may be physical or virtual. The cyber threat protection platform may receive packet data itself by span or traffic mirroring while in communication with any host-based agent and virtual sensor probes.

[0051] The cyber threat protection platform 100 may supplement the data provided to users and cyber professionals who use an analyzer module to monitor various connections between client devices in the network. The analyzer module may flag client devices that are the location of anomalous events. The analyzer module may be configured to flag the client device for host-based traffic decryption. In host-based traffic decryption, a host-based agent may decrypt one or more data packets for a connection at the client device. The analyzer module may be configured to determine that the client device allows host-based traffic decryption based on at least one of endpoint rarity, timing rarity, domain rarity, and environment. The host-based agent may perform the decryption by at least one of receiving a private key from a third-party agent, uploading a public / private key pair to a configuration page from the client network, or retrieving the private key from the client device.

[0052] The cyber threat protection platform 100 can then take action to combat the detected potential cyber threats.

[0053] The autonomous response module is configured to take actions that are pre-approved by a human user to autonomously attempt to combat a malicious threat. Again, rather than a human taking the action, the autonomous response module is configured to trigger one or more autonomous actions to be taken to thwart a potential cyber threat when it is detected. The cybersecurity device may have an autonomous response module with a user programmable interface. A user programmable interface hosted on the cybersecurity device having either i) fields, ii) menus, and iii) icons is described to allow a user to pre-authorize the autonomous response module to take actions to thwart a cyber threat. The user programmable fields / menus allow a user to pre-authorize the module to take actions such as killing individual processes without disturbing the operation of other processes going on inside the device, revoking certain privileges, preventing downloading of certain files, allowing only processes observed in the pattern of life of peer devices to be active for a set period of time, requesting other endpoint protection platforms (EPPs) to quarantine suspicious files, etc. The user interface has granularity in the options available to the user to program the autonomous response module to take very specific actions, such as killing individual processes, revoking certain permissions while still allowing other permissions for that user, gaining live terminal access, preventing the downloading of certain files, allowing only processes observed in the pattern of life of peer devices to be active for a set period of time, asking other EPPs to quarantine suspicious files etc. without shutting down the entire device or blocking all external communications, or revoking one or more but not all of that user's privileges.Actions such as revoking only some user privileges or enforcing peer life patterns may make it impossible for a user to make certain connections or execute certain processes that malicious software would most likely have initiated, such as accessing and downloading confidential files, while the user is performing normal activities for that user, such as typing a document or entering data into a program, entirely unaware of the malicious software using the user's credentials, but allowing the user to continue working.

[0054] Example autonomous actions available for pre-approval by a human user for the autonomous response module may include the following actions: prevent or delay threat-related activity; isolate or semi-isolate people, processes, or devices; feed threat intelligence to EPP and endpoint detection and response (EDR) processes and devices to take third-party or vendor-specific actions such as isolation or firewall blocking; terminate anomalous processes on client devices, etc.; and, in many cases, general prompts to the user on the display screen of the endpoint computing device along with actions that do not interfere with the user's normal day-to-day activities or other processes on the endpoint computing device.

[0055] The autonomous response module may be configured to trigger one or more rapid autonomous actions to be taken to thwart a cyber threat when a threat risk parameter from the cyber threat module is equal to or greater than a threshold that can guide the action, without the human being taking the action. The cyber threat module configured in cooperation with the autonomous response module to trigger one or more autonomous actions to be taken to thwart a cyber threat, by responding to the cyber threat without waiting for any human intervention, improves computing devices in the email system by limiting the impact of the cyber threat from consuming unauthorized CPU cycles, memory space, and power consumption on the computing devices.

[0056] The autonomous response module may tag certain users to have a lower threshold for autonomous response depending on the cyber threat situation. For example, a chief financial officer may cause significant damage to a company by conducting financial transactions to embezzle funds. If the cyber threat module identifies a cyber threat to the finance function, the autonomous response module may lower the threshold for autonomous response upon identifying tagged users associated with the cyber threat. The autonomous response module may simultaneously employ several different clustering methods, including matrix-based clustering, density-based clustering, and hierarchical clustering techniques, to identify which users should be tagged with which threat type.

[0057] The cyber threat protection platform 100 may be hosted on the device, on one or more servers, or within its own cyber threat appliance platform.

[0058] FIG. 2 illustrates a block diagram of one embodiment of an example series of non-normal behaviors for a network entity in relation to other networks under analysis.

[0059] The user interface may display a graph 200 of a series of example non-normal behaviors for the SaaS application in relation to other networks under analysis.

[0060] The cyber threat module interfaces with one or more machine learning models that are trained and otherwise configured with mathematical algorithms for cyber threat analysis to infer what may happen due to a series of disparate alerts and / or events that result from a non-normal pattern, and then assign a threat risk associated with the disparate items of the series of alerts and / or events that form the non-normal pattern.

[0061] This is a "behavior pattern analysis" of what is non-normal behavior of a network entity such as a network, system, device, user, or email under analysis by the cyber threat module and machine learning model. The cyber defense system uses non-normal behavior that deviates from normal behavior and then builds a chain of non-normal behaviors and causal relationships between this chain of non-normal behaviors to detect cyber threats. An example behavior pattern analysis of what is non-normal behavior may be as follows: Non-normal patterns may be determined by filtering out activities, events, or alerts that fall within a window of being a normal pattern of life for that network entity under analysis. The remaining activity, event, or alert behavior pattern may then be analyzed to determine if the pattern, after filtering, is indicative of malicious party behavior such as a human, program, email, or other threat. The defense system may go back and pull out some of the filtered out normal activities to help support or refute a possible hypothesis of whether the pattern is indicative of malicious party behavior. Example behavior patterns included within the chain are shown in a graph over an example 7-day time frame. The prevention system detects three instances of a series of anomalous behaviors of non-normal data transfers, and three instances of non-normal characteristics in emails in the monitored system that appear to have some causal relationship to the non-normal data transfers. Similarly, two instances of non-normal credentials attempting the non-normal behavior of attempting to access an information sensitive area, or a malicious IP address and user associated with a non-normal credential attempting the non-normal behavior, are causally related to at least one of those three emails with the non-normal characteristics. When the behavioral pattern analysis of the individual behaviors, or the behavioral pattern analysis of the chain as a group, is deemed to be indicative of a malicious threat, a score is generated of how confident the prevention system is in this assessment that it identifies whether the non-normal pattern was caused by a malicious party.Then, further assigned is a threat level parameter (e.g., a score or probability) indicating what level of threat this malicious party poses to the system. Finally, the cyber threat protection platform is configurable within its user interface of the protection system as to what type of automated response action, if any, the protection system may take when posed by this malicious party for different types of cyber threats that are equal to or greater than the configurable level of threat.

[0062] The cyber threat module may connect the individual alerts and events that form the abnormal pattern into a disparate item for cyber threat analysis of that series of disparate alerts or events. The cyber threat module may reference one or more machine learning models trained on email threats to identify similar features from the individual alerts or events that form the disparate item comprised of the series of alerts or events that form the abnormal pattern.

[0063] One or more machine learning models may also be trained on characteristics and aspects of all manner of cyber threat types to analyze the threat risk associated with a series or group of alerts or events that form an abnormal pattern. Machine learning techniques using advanced mathematics can detect previously unidentified threats and automatically defend the network without relying on predefined rules.

[0064] The model may be implemented by threat detection through probabilistic changes in normal behavior by application of unsupervised Bayesian mathematical models to detect behavioral changes in computers and computer networks. The core threat detection system is called "Bayesian Probability". The Bayesian Probability approach can determine periodicity in multiple time series data and identify changes across single or multiple time series data for the purpose of anomalous behavior detection. A large number of metrics can be obtained from email and network raw sources of data, each of which provides time series data for a given metric.

[0065] Detectors in the Cyber ​​Threat Module, including the Probe Module and any SaaS Module components, may be discrete mathematical models that implement a particular mathematical methodology on a set of different variables with targets. Thus, each model may be specifically targeted to, for example, i) its cybersecurity analysis tools, ii) analyzing various aspects of third-party SaaS interactions, iii) the pattern of life of alerts and / or events brought about by particular devices and / or users in the system, etc.

[0066] Fundamentally, a cyber threat defense platform mathematically characterizes what constitutes "normal" behavior based on the analysis of a large number of different measures / sets of different measures of a device's network behavior. The cyber threat defense platform can build an elaborate "pattern of life" that understands what represents normal for every person, device, email activity, and network activity within the systems protected by the cyber threat defense platform.

[0067] As discussed, each machine learning model may be trained on a particular aspect of the system's normal pattern of life, such as devices, users, network traffic flows, output from one or more cybersecurity analysis tools analyzing the system, email contact associations for each user, email characteristics, and so forth. The one or more machine learning models may use at least an unsupervised learning algorithm to establish what the normal pattern of life is for the system. The machine learning models may be trained on both i) the past normal delivery of alerts and events for that system, and ii) normal delivery information from similar peer systems to establish the normal pattern of life of alert or event behavior for that system. Another set of machine learning models is trained on the characteristics of the SaaS application and the activities and behavior of SaaS application users to establish normal for these.

[0068] The model can detect anomalies using at least two different approaches: comparing each system's behavior to its own history and to the history of its peers, or comparing emails to both the email's characteristics and the activities and behavior of its email users. This multi-source comparison allows the model to avoid learning existing bad behavior as "normal behavior" since compromised entities such as devices, users, components, emails, etc., exhibit different behaviors than their immediate peers.

[0069] In addition, one or more machine learning models can use a comparison of i) normal life patterns for the system corresponding to past normal delivery of alerts and events for the system mapped in the same multi-dimensional space, and ii) the current series of individual alert and event behaviors under analysis. This comparison can result in the detection of one or more non-normal patterns of behavior within the plotted individual alerts or events, which allows for the detection of previously unidentified cyber threats compared to simply finding cyber threats using predefined descriptive objects or signatures. Thus, increasingly sophisticated malicious cyber threats that are picky about when they take action to generate low level alerts and events will still be detected even if they have not yet been identified by other means of cyber analysis. These sophisticated malicious cyber threats can include malware, spyware, keyloggers, malicious links in emails, malicious attachments in emails, and others, as well as rogue internal information technology staff who know well how not to trigger any high level alerts or events.

[0070] Plotting and comparing is a way to filter out what is normal for the system and then focus the analysis on what is abnormal or non-normal for the system. Then, for each hypothesis of what may happen due to a set of non-normal events or alerts, the collection module may gather additional metrics from the data store that originally contained the pool of metrics considered "normal behavior" to support or refute each possible hypothesis of what may happen due to this set of non-normal behavior under analysis.

[0071] It should be noted that each individual alert or event in a chain of alerts or events forming a non-normal pattern may exhibit subtle anomalous behavior. Thus, each alert or event may have a low threat risk associated with it. However, when analyzed by one or more machine learning models as a series or group of disparate alert or event behaviors forming a series of non-normal patterns, the disparate series of alerts or events may now be determined to have a much higher threat risk than any of the individuals and / or events in the chain.

[0072] In addition, modern cyber attacks can be of such severity and speed that a human response cannot occur fast enough. Thanks to these self-learning advances, machines can discover these emerging threats and develop appropriate real-time responses to counter the most serious cyber threats.

[0073] Threat detection systems have the ability to self-learn and detect normality to distinguish true anomalies, allowing organizations of all sizes to understand the behavior of users and machines on their networks at both individual and group levels. Monitoring behavior, rather than using predefined descriptive objects and / or signatures, means that more attacks can be spotted up front and even the most subtle indicators of fraud can be detected. Unlike traditional legacy defenses, a particular attack type or new malware does not need to be seen in the first place before it can be detected. A behavioral defense approach behaviorally and mathematically models both machine, email, and human activity during and after a security breach to predict and understand today's increasingly sophisticated cyber attack vectors. Thus, it is possible to computationally establish what is normal and then detect what is abnormal. Additionally, machine learning uses probabilistic mathematics to constantly revisit assumptions about behavior. The unsupervised machine learning methods of the cyber threat defense platform do not require training the data with predefined labels. Instead, unsupervised machine learning methods may identify key patterns and trends within the data without the need for human input.

[0074] The user interface and output module may also project the individual alerts and / or events that form the chain of behavior onto a user interface that has at least three dimensions: i) a horizontal axis of a window of time, ii) a vertical axis of a scale indicating the threat risk assigned to each alert and / or event in the chain, and iii) a third dimension of different colors for similar characteristics shared between the individual alerts and events that form distinct items of the chain. The different colors may be red, blue, yellow, or other. In the case of grayscale, the user interface may use different shades of gray, black, and white with potentially different hash patterns. These similarities of events or alerts in the chain may be, for example, alerts or events coming from the same device, the same user credentials, the same group, the same source identifier, the same destination Internet Protocol address, the same type of data transfer, the same type of non-normal activity, the same type of alert, the same unusual connections made, the same type of event, or other, so that a human can visually see what makes up a particular chain spatially and content-wise, rather than simply browsing a text log of data. Note that when a human mind visually sees the projected patterns and corresponding data, the human can ultimately determine whether a cyber threat is posed. Again, at least a three-dimensional projection helps the human synthesize this information more easily. The visualization on the user interface allows the cyber threat prevention platform to see data that supports or refutes why these aggregated alerts or events may be potentially malicious. Also, instead of generating a simple binary output "malicious" or "benign," the cyber threat prevention platform's mathematical algorithms result in outputs that indicate different degrees of potential security compromise.

[0075] Defense Systems FIG. 3 illustrates an example cyber threat defense platform protecting an example network. The example network of FIG. 3 shows an example of a network of computer systems 50 using a threat detection system. The system depicted by FIG. 3 shows a simplified example provided to facilitate the description of the present invention. The system 50 comprises a first computer system 10 in a building that uses the threat detection system to detect, and thereby attempt to prevent, threats to computing devices within its range. The first computer system 10 comprises three computers 1, 2, 3, a local server 4, and a multifunction device (MFD) 5 that provides printing, scanning, and faxing capabilities to each of the computers 1, 2, 3. All of the devices in the first computer system 10 are communicatively coupled via a local area network (LAN) 6. As a result, all of the computers 1, 2, 3 can access the local server 4 via the LAN 6 and use the functions of the MFD 5 via the LAN 6.

[0076] The LAN 6 of the first computer system 10 is connected to the Internet 20 which provides the computers 1, 2, 3 with access to a number of other computing devices including a server 30 and a second computer system 40. The second computer system 40 also includes two computers 41, 42 connected by a second LAN 43.

[0077] In this exemplary embodiment of the invention, the computer 1 on the first computer system 10 has a threat detection system and therefore executes a threat detection method for detecting threats to the first computer system. The computer 1 therefore comprises a processor configured to execute the steps of the process described herein, a memory required to store information regarding the execution of this process, as well as a network interface for collecting the required information. This method shall now be described in detail with reference to FIG. 3.

[0078] Computer 1 builds and maintains a dynamic, ever-changing model of the "normal behavior" of each user and machine in system 10. The approach is based on Bayesian mathematics and monitors all interactions, events, and communications within system 10 - which computers talk to which computers, files created, networks accessed.

[0079] For example, Computer 2 is based in the company's San Francisco office and is operated by a marketing employee who regularly accesses the marketing network. Computer 2 is active from approximately 8:30 a.m. to 6:00 p.m. and typically communicates with a machine in the company's UK office in a second computer system 40 from 9:30 a.m. to noon. This same employee virtually never accesses an employee timesheet, rarely connects to the company's Atlanta network, and has no business in Southeast Asia. The threat detection system takes all available information about this employee and establishes a "pattern of life" for that person, which is dynamically updated as more information is gathered. The "normal" model is used as a moving benchmark, enabling the system to spot behavior on the system that appears to deviate from this normal pattern of life and flag this behavior as an anomaly for further investigation.

[0080] Threat detection systems are built to address the fact that today's attackers are becoming increasingly cautious. Attackers may "hide" in the system using normal software protocols to ensure they avoid raising suspicions in the end user, such as by slowing down the user's machine. Thus, any attack process will stop or "back down" if the mouse or keyboard is used. However, still more sophisticated attacks try the opposite, hiding in memory as normal processes and stealing CPU cycles only when the machine is active in an attempt to defeat relatively simple security processes. These sophisticated attackers look for activity that is not directly associated with user input. Advanced persistent threat (APT) attacks typically have very long mission windows, weeks, months, or years, and such processor cycles may not be stolen very frequently so as not to impact machine performance. However, no matter how hidden and sophisticated an attack is, it leaves a measurable difference, even if extremely small, in typical machine behavior before, during, and after a security breach. This difference in behavior can be observed and acted upon in the form of Bayesian mathematical analysis used by a threat detection system installed on computer 1.

[0081] FIG. 4 illustrates in a block diagram the integration of a threat detection system with other network protections. A network typically has a firewall 402 as a first line of defense. The firewall 402 analyzes packet headers on incoming network data packets to enforce network policies. The firewall 402 may be integrated with an intrusion prevention system (IPS) to analyze packet headers and payloads for the entire event. Internally, an identity management module 404 controls the access of users of the network.

[0082] The network security module 406 can implement practices and policies for the network as determined by a network administrator. The encryption module 408 can encrypt communications within the network as well as encrypt and decrypt communications between network entities and external entities. The antivirus or antimalware module 410 can search packets for known viruses and malware. The patch management module 412 can ensure that security applications within the network have the latest patches applied. The centralized logging module 414 can track communications both internal to the network and interactive with the network. The threat detection system can act as real-time threat intelligence 416 for the network. The real-time threat intelligence can interact with other defense components to protect the network.

[0083] The cyber defense self-learning platform uses machine learning techniques. Using advanced mathematics, machine learning techniques can detect previously unidentified threats and automatically defend the network without rules. Note that today's attacks can be of such severity and speed that a human response cannot occur fast enough. Thanks to these self-learning advances, it is now possible for machines to discover emerging threats and deploy appropriate real-time responses to counter the most serious cyber threats.

[0084] A cyber threat defense platform builds an elaborate "pattern of life" that understands what represents normal for all person, device, and network activity within the systems protected by the cyber threat defense platform.

[0085] Threat detection systems can self-learn and detect normality to distinguish true anomalies, allowing organizations of all sizes to understand the behavior of users and machines on their networks at both individual and group levels. Monitoring behavior, rather than using predefined descriptive objects and / or signatures, means that more attacks can be identified in advance and even the most subtle indicators of fraud can be detected. Unlike traditional legacy defenses, a particular attack type or new malware does not need to be seen in the first place before it can be detected. A behavioral defense approach behaviorally and mathematically models both machine and human activity during and after a security breach to predict and understand today's increasingly sophisticated cyber attack vectors. The approach is therefore capable of computationally establishing what is normal and subsequently detecting what is abnormal.

[0086] This intelligent system can make value judgments and perform higher value, more thoughtful tasks. Machine learning requires complex algorithms to be devised and a general framework for interpreting the results that are generated. However, when applied correctly, these approaches can facilitate machines to make logical, probability-based decisions and undertake thoughtful tasks.

[0087] Advanced machine learning is at the forefront of the fight against automated and human-caused cyber threats, overcoming the limitations of rules and signature-based approaches. For example, machine learning learns what is normal in a network without relying on knowledge of previous attacks. Machine learning thrives at the scale of the complexity and diversity of modern business, where every device and person is slightly different. Machine learning directs attackers’ innovations to ensure that any non-normal activity is visible. Machine learning uses probabilistic mathematics to constantly revisit assumptions about behavior. Machine learning is always up-to-date and does not rely on human input. Utilizing machine learning in cybersecurity technology is difficult, but when implemented correctly, it is extremely powerful. Machine learning means that previously unidentified threats can be detected even when their manifestations fail to trigger any rule set or signature. Instead, machine learning allows the system to analyze large data sets and learn the “patterns of life” of what it sees.

[0088] FIG. 5 illustrates the application of a cyber threat protection platform that uses advanced machine learning to detect anomalous behavior. A normal pattern of behavior 510 may describe a set of user or device behaviors that are within a threshold level of occurrence, such as a 98% probability of occurrence based on previous behavior. Anomalous activity 520 may describe a set of user or device behaviors that are above a threshold level of occurrence. The cyber threat protection platform can initiate an autonomous response 530 to disrupt the anomalous activity without affecting normal behavior.

[0089] Machine learning can approximate some human capabilities to machines. Machine learning can approximate thinking by using past information and insights to form judgments. Machine learning can operate in real-time as the system processes information instantly. Machine learning can self-improve by constantly challenging and adapting the model's machine learning understanding based on new information.

[0090] Thus, novel unsupervised machine learning enables computers to recognize emerging threats without prior warning or oversight.

[0091] Unsupervised Machine Learning Unsupervised learning solves problems without predefined labels, allowing the system to handle the unexpected and embrace uncertainty. The system does not always know the characteristics of the target of the search, but it can independently classify data and detect compelling patterns.

[0092] The unsupervised machine learning methods of the cyber threat defense platform do not require training the data with predefined labels. Instead, unsupervised machine learning methods can identify key patterns and trends in the data without the need for human input. Unsupervised learning offers the advantage of allowing the computer to go beyond what the programmer already knows and discover previously unknown relationships.

[0093] Cyber ​​threat prevention platforms use unique implementations of unsupervised machine learning algorithms to analyze network data at scale, intelligently dealing with the unexpected and embracing uncertainty. Instead of relying on past threat knowledge to know what to look for, cyber threat prevention platforms can independently classify data and detect compelling patterns that define what can be considered normal behavior. Any novel behavior that deviates from this notion of "normality" can indicate a threat or security breach. The impact of cyber threat prevention platforms' unsupervised machine learning on cyber security is transformative. Threats that would otherwise go undetected can be identified, highlighted, prioritized according to context, and isolated using these algorithms. The application of machine learning has the potential to provide full network visibility and a much significantly improved level of detection, ensuring that the network has an internal defense mechanism. Machine learning has the ability to learn when to execute an automated response to the most serious cyber threats, crushing ongoing attacks before they become a crisis for the organization.

[0094] This novel mathematics not only identifies important relationships in the data, but also quantifies the uncertainty associated with such inferences. Knowing and understanding this uncertainty allows many results to be organized into a coherent framework based on Bayesian probability analysis. The mathematics behind machine learning is highly complex and difficult to understand properly. Robust and reliable algorithms are developed with scalability that enables their successful application in real-world environments.

[0095] overview In one embodiment, the Cyber ​​Threat Prevention Platform's probabilistic approach to cyber security is based on a Bayesian framework, which allows the Cyber ​​Threat Prevention Platform to synthesize a large number of weak indicators of potentially anomalous network behavior to produce a single, unambiguous measure of how likely a network device is to be compromised. This probabilistic mathematical approach provides the ability to make sense of important information in the noise of the network, even when you do not know what the target of your search is.

[0096] Threat ranking Importantly, the Cyber ​​Threat Prevention Platform's approach accounts for the inevitable ambiguity present in the data and distinguishes between slightly different levels of evidence that different data may contain. Instead of generating a simple binary output "malicious" or "benign," the Cyber ​​Threat Prevention Platform's mathematical algorithms result in outputs that indicate different degrees of potential security compromise. This output allows users of the system to rank different alerts in a rigorous fashion to prioritize those that most urgently require action, while simultaneously eliminating the many false positive issues associated with rule-based approaches.

[0097] At a fundamental level, a cyber threat prevention platform mathematically characterizes what constitutes "normal" behavior based on the analysis of a large number of different measures of network behavior by a device. Such network behavior may include server access, data access, timing of events, credential usage, Domain Name Server (DNS) requests, and other similar parameters. Each measure of network behavior is then monitored in real time to detect anomalous behavior.

[0098] Clustering To be able to properly model what should be considered normal for a device, device behavior must be analyzed within the context of other similar devices on the network. To accomplish this, cyber threat prevention platforms leverage the strengths of unsupervised learning to algorithmically identify naturally occurring groupings of devices, a task that is impossible to do manually on even a moderately sized network.

[0099] To achieve as holistic a view of the relationships in the network as possible, the cyber threat defense platform simultaneously employs several different clustering methods, including matrix-based clustering, density-based clustering, and hierarchical clustering techniques. The resulting clusters are then used to inform exemplary behavior modeling of individual devices. Clustering analyzes behavior within the context of other similar devices on the network. Clustering algorithms identify naturally occurring groupings of devices that would be impossible to do manually. Additionally, the cyber threat defense platform runs multiple different clustering methods simultaneously to inform the model.

[0100] Network Topology Any cyber threat detection system must also recognize that a network is much more than the sum of its individual parts, and most of its meaning is contained in the relationships between its different entities. Any cyber threat prevention platform must further recognize that complex threats can often induce subtle changes in this network structure. To capture such threats, cyber threat prevention platforms employ several different mathematical methods so that they can model multiple aspects of the network topology.

[0101] One approach is based on iterative matrix methods that reveal significant connectivity structures within the network. In parallel with these, the Cyber ​​Threat Defense Platform has developed an innovative application of models from the field of statistical physics that allows modeling the "energy landscape" of the network to reveal the underlying structure of anomalies hidden within it.

[0102] Network Structure A further significant challenge in modeling the behavior of network devices, as well as the network itself, is the high-dimensional structure of the problem with a large number of potential predictor variables. Observing packet traffic and host activity in local area networks (LANs), wide area networks (WANs), and clouds is difficult because both inputs and outputs may contain many interrelated features, such as protocols, source and destination machines, log changes, rule triggers, and more. Learning a sparse and coherently structured predictor function is critical to avoid overfitting.

[0103] In this context, the Cyber ​​Threat Prevention Platform employs state-of-the-art large-scale computational approaches to learn sparse structures in models of network behavior and connectivity based on applying L1-regularization techniques such as the Least Absolute Shrinkage and Selection Operator (LASSO) method, which assigns efficiently solvable convex optimization problems and allows the discovery of true associations between different network components and events that can result in parsimonious models.

[0104] Recursive Bayesian Estimation To combine these multiple analyses of different measures of network behavior to generate a single, comprehensive picture of the state of each device, the cyber threat defense platform leverages the strengths of Recursive Bayesian Inference (RBE) through its implementation of a Bayes filter.

[0105] Using RBE, the mathematical models of a cyber threat defense platform can constantly adapt in a computationally efficient manner as new information becomes available to the system: they continually recalculate threat levels in light of new evidence and identify changing attack behaviors where traditional signature-based methods fail.

[0106] The Cyber ​​Threat Prevention Platform's innovative approach to cyber security pioneered the use of Bayesian methods to track changing device behavior and computer network structures. At the core of the Cyber ​​Threat Prevention Platform's mathematical modeling is the determination of exemplary behavior, enabled by a sophisticated software platform that allows the mathematical models to be applied in real time to novel network data. The result is a system that can identify subtle variations in machine events within computer network behavior history that may indicate a cyber threat or security breach.

[0107] Cyber ​​threat prevention platforms use mathematical analysis and machine learning to detect potential threats, allowing the system to stay ahead of emerging risks. The cyber threat prevention platform approach means that detection no longer relies on archives of previous attacks. Instead, attacks can be discerned against a contextual understanding of what represents normal within the network. No predefinition is required, which allows for the most likely insight and defense against today's threats. In addition to detection capabilities, cyber threat prevention platforms can automatically create digital antibodies as an immediate response to the most threatening cyber breaches. The cyber threat prevention platform approach both detects and defends against cyber threats. Pure unsupervised machine learning removes the reliance on signature-based approaches to cyber security that are not working. Cyber ​​threat prevention platform technology can become an essential tool for security teams looking to understand the scale of their networks, observe levels of activity, and detect potential weak areas. These no longer need to be manually hunted down, but rather are flagged by automated systems and ranked as to their significance.

[0108] Machine learning techniques are a fundamental ally in defending systems against today's hackers and insider threats, and in formulating responses to unknown methods of cyber attacks. It is a groundbreaking step change in cybersecurity. Defense must start inside.

[0109] Example Method The threat detection system shall now be described in further detail with reference to the process flow performed by the threat detection system for the automated detection of cyber threats through probabilistic changes in normal behavior by application of unsupervised Bayesian mathematical models to detect behavioral changes in computers and computer networks.

[0110] The core threat detection system is called "Bayesian Probabilistic," which is a Bayesian system that automatically determines periodicities in multiple time series data and identifies changes across single or multiple time series data for the purpose of anomalous behavior detection.

[0111] 6 illustrates a flow chart of one embodiment of a method for modeling human, machine, or other activity. The cyber threat protection platform first ingests data from multiple sources (block 602). Raw data sources include, but are not limited to, raw network IP traffic obtained from Internet Protocol (IP) or other network test access points (TAPs) or switched port analyzer (SPAN) ports; machine-generated log files; building access ("swipe card") systems; IP or non-IP data flowing through industrial control system (ICS) distributed networks; individual machine, peripheral, or component power usage; telecommunications signal strength; or machine-level performance data obtained from on-host sources, such as central processing unit (CPU) usage, memory usage, disk usage, disk free space, network usage, and others.

[0112] The cyber threat defense platform obtains secondary metrics from the raw data (block 604). From these raw data sources, multiple metrics may be obtained, each yielding time series data for a given metric. The data is dumped into individual time slices. For example, the number of observations may be counted per second, per 10 seconds, or per 60 seconds. These buckets may be combined at a later stage as needed to provide longer range values ​​for any multiple of the selected internal size. For example, if the underlying time slice selected is 60 seconds long, each metric time series stores a single value for that metric every 60 seconds, and then any new time series data for a fixed multiple of 60 seconds (120 seconds, 180 seconds, 600 seconds, etc.) may be calculated without loss of accuracy. Metrics are selected directly and fed into Bayesian probabilities by low-order models that reflect some inherent underlying portion of the data and may be derived from the raw data with specific domain knowledge. The metrics obtained depend on the threats the system is looking for. To provide a secure system, a cyber threat defense platform typically captures multiple metrics regarding a broad range of potential threats: Communications from components within the network that contact known suspicious domains.

[0113] The actual particular metric used, as long as the metric is selected, is largely irrelevant to a Bayesian probabilistic system. Metrics derived from network traffic might include data such as the number of bytes of data entering or leaving a networked device per time interval, file accesses, commonality or rarity of communication processes, invalid Secure-Sockets Layer (SSL) certificates, failed authentication attempts, or email access patterns.

[0114] When Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or other transport layer IP protocols are used across IP networks, and when alternative internet layer protocols such as Internet Control Message Protocol (ICMP) or Internet Group Message Protocol (IGMP) are used, knowledge of the structure of the protocols in use and basic packet header analysis can be utilized to generate additional metrics. Such additional metrics can include the number of multicasts per time interval originating from the networked device and intended to reach publicly addressable IP ranges, the number of internal link-local IP broadcast requests originating from the networked device, packet payload data size, or the number of individual TCP connections made by the device, or the data forwarded by the device, either as a combined aggregate across all destinations or to any definable network range, such as a single target machine or a specific network range.

[0115] In the case of IP traffic, where application layer protocols may be determined and analyzed, further types of time series metrics may be defined, such as, for example: the number of DNS requests a networked device generates per time interval, again either for any definable network range or in aggregate; the number of Simple Mail Transfer Protocol (SMTP), Post Office Protocol (POP), or Internet Message Access Protocol (IMAP) logins or login failures a machine generates per time interval; the number of Lightweight Directory Access Protocol (LDAP) logins or login failures generated; data transferred via file sharing protocols such as Server Message Block (SMB), SMB2, File Transfer Protocol (FTP), or others; or logins to Microsoft Windows Active Directory, Secure Shell (SSH) or local logins to Linux or Unix-like systems, or other authentication systems such as Kerberos.

[0116] The raw data required to obtain these metrics can be collected from virtual switch implementations, cloud-based systems, or the communication devices themselves, via passive fiber or copper connections to the network's internal switch equipment. Ideally, the system receives copies of all communication packets to provide full coverage of the organization.

[0117] For other sources, several domain-specific time series data are obtained, each selected to reflect a distinct and distinguishable phase of the underlying source of the data, which in some way reflects the use or behavior of the system over time.

[0118] Many of these time series data are extremely sparse, with the majority of data points equal to 0. Examples are an employee using a swipe card to access a building or part of a building, or a user logging into their workstation authenticated by a Microsoft Windows Active Directory server, which typically occurs a small number of times per day. Other time series data sets are much denser, such as the size of data moving in and out of an always-on web server, web server CPU utilization, or power usage of a copier.

[0119] Regardless of the type of data, such time series data sets, whether originally generated as a result of explicit human behavior or as a result of automated computer or other systems, tend to exhibit periodicity, with various patterns within the data that repeat at approximately regular intervals. Moreover, such data may have many distinct but independent regular time periods that are evident within the time series.

[0120] The detector performs an analysis of the second order metrics (block 606). A detector is a discrete mathematical model that performs a specific mathematical method on a different set of variables that comprise the target network. For example, a Hidden Markov Model (HMM) might look specifically at the size and transmission time of packets between nodes. The detectors are provided in a hierarchy that is a pyramid of loosely arranged models. Each detector model effectively acts as a filter and passes its output to another model higher up the pyramid. At the top of the pyramid is a Bayesian probability that is the final threat decision model. Each of the lower order detectors monitors different global attributes or "features" of the underlying network and / or computer. These attributes can be values ​​over time for packet rates and morphology, endpoint file system values, and all internal computational features such as TCP / IP protocol timing and events. Each detector is specialized to record and make decisions about different environmental factors based on the detector possessing an internal mathematical model such as an HMM.

[0121] While a threat detection system may be configured to look for any possible threat, in practice the system may keep an eye on one or more specific threats depending on the network in which the threat detection system is being used. For example, the threat detection system provides a way in which known characteristics of the network, such as desired compliance and human resource policies, are encapsulated in explicitly defined heuristics or detectors that can be triggered when they cooperate with a set of anomaly probabilities or varying thresholds resulting from the probability determination output. The heuristics are constructed using a complex set of weighted logical expressions that represent regular expressions with atomic objects derived at run time from the output of data measuring / tokenizing detectors and local contextual information. These sets of logical expressions are then stored in an online library and parsed in real time against the output from the measuring / tokenizing detectors. An example policy may take the form of "Alert if any employee (contextual information) subject to a HR disciplinary situation is accessing sensitive information (heuristic definition) in a manner (Bayesian probability output) that is anomalous when compared to previous behavior." In other words, different arrays of pyramids of detectors are provided to detect specific types of threats.

[0122] The analysis performed by the detector on the secondary metric then outputs data in a form suitable for use with the model of normal behavior. As can be seen, the data is in a form suitable for comparison to the model of normal behavior and for updating the model of normal behavior.

[0123] The threat detection system calculates a threat risk parameter (block 608) that indicates the likelihood that a threat exists using automated adaptive periodicity detection that is mapped to a pattern-of-life analysis of the observed behavior. It infers that a threat exists from a collected set of attributes that have themselves shown deviations from exemplary collective or individual behavior over time. The automated adaptive periodicity detection uses a calculated period of time where a Bayesian probability is most appropriate within the observed network or machine. Furthermore, the pattern-of-life analysis identifies how humans or machines behave over time, for example, when they typically start and finish tasks. These models are inherently harder to defeat than known systems because they are continuously adapted automatically. The threat risk parameter is the probability that a threat exists in a particular configuration. Alternatively, the threat risk parameter is a value that represents the presence of a threat, which is compared to one or more thresholds that indicate the likelihood of a threat.

[0124] In practice, the step of calculating a threat involves comparing current data collected related to a user to a model of normal behavior of the user and the system being analyzed. The current data collected relates to a period in time, which may relate to a certain influx of new data, or a specific period of time, from a few seconds to a few days. In some configurations, the system is configured to predict the expected behavior of the system. The predicted behavior is then compared to the actual behavior to determine if a threat exists.

[0125] The system uses machine learning or artificial intelligence to understand what is normal inside a company's network and when something is not. The system then invokes an automated response to crush the cyberattack until a human team can roll back. This may include interrupting connections, preventing malicious emails from being sent, preventing file access, preventing communications outside the organization, etc. The approach starts as surgically and controlled as possible to disrupt the attack without affecting the normal behavior of, for example, a laptop. If the attack escalates, the cyber threat prevention platform may eventually quarantine the device to prevent wider harm to the organization.

[0126] To improve the accuracy of the system, checks are performed to compare the user's current behavior to related users, e.g., users within a single office. For example, if there is an unexpectedly low level of activity from a user, this may not be attributable to non-normal activity from the user, but rather factors affecting the entire office. Various other factors may be considered to assess whether the abnormal behavior actually indicates a threat.

[0127] Finally, the cyber threat protection platform determines whether further action needs to be taken regarding the threat based on the threat risk parameters (block 610). A human operator may make this determination after being provided with a probability that the threat exists. Alternatively, an algorithm may make the determination, for example, by comparing the determined probability to a threshold value.

[0128] In one configuration, given a unique global input of Bayesian probabilities, a form of threat visualization is provided that allows users to see the threat landscape across all internal traffic, without needing to know how their internal networks are structured or dense, and in such a way that a "universal" view is presented within a single compartment regardless of the size of the network. The topology of the network under scrutiny is automatically projected as a graph based on device communication relationships via an interactive 3D user interface. The projection can be linearly scaled to any node scale without prior seeding or skeleton definition.

[0129] Thus, the threat detection system described above implements a plausible form of recursive Bayesian inference to maintain a distribution over the probability state variables. This distribution is constructed from a complex set of low-level host, network, and traffic observations or "features." These features are iteratively recorded and processed in real-time on the platform. A plausible representation of the relationship information between entities in a dynamic system in general, such as an enterprise network, a biological cell or a social community, or certainly the entire Internet, is a probabilistic network that is topologically rewiring and semantically evolving over time. In many highly structured input / output problems, such as observations of packet traffic and host activity in a distributed digital enterprise where both inputs and outputs can include tens of thousands to millions of interrelated features (such as data transports, host-web-client dialogues, log changes, and rule triggers), learning a predictive function of sparse and consistent structure is challenged by the lack of normal distributions. To overcome this, the threat detection system is equipped with a data structure that determines a rotating continuum, rather than a stepwise method, to which repetitive time cycles such as work days, shift patterns, and other routines are dynamically assigned. In this way, we provide a non-frequentist architecture for inferring and testing causal relationships between explanatory variables, observations, and feature sets. This allows for efficiently solvable convex optimization problems, resulting in parsimonious models. In such configurations, the threat detection process may be triggered by the input of new data. Alternatively, the threat detection process may be triggered by the absence of predictive data. In some configurations, the process may be triggered by the presence of an event that may guide a particular action.

[0130] The method and system are configured to be performed by one or more processing components having any portion of the software stored in executable form on a computer-readable medium. The computer-readable medium may be non-transitory and does not include radio or other carrier waves. The computer-readable medium may be a physical computer-readable medium, such as, for example, a semiconductor or solid-state memory, a magnetic tape, a removable computer floppy disk, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and an optical disk, such as a CD-ROM, a CD-R / W, or a DVD.

[0131] The various methods described above may be implemented by a computer program product. The computer program product may include computer code configured to instruct a computer to perform one or more functions of the various methods described above. The computer program and / or code for implementing such methods may be provided to an apparatus, such as a computer, on a computer-readable medium, or on a computer program product. In the case of a computer program product, the transitory computer-readable medium may include radio or other carrier waves.

[0132] An apparatus, such as a computer, may be configured with such code to perform one or more processes in accordance with the various methods discussed herein.

[0133] 7 illustrates a flow chart of one embodiment of a method for identifying anomalous events from network event data. The cyber threat protection platform can employ a probe module configured to collect probe data from one or more probes deployed on one or more client devices (block 702). The network entities represent at least one of users and network devices. The probe data can describe network managed activity SaaS activity by the network entities.

[0134] The cyber threat protection platform may employ an email module configured to collect email data from an email service (block 704). The cyber threat protection platform may employ a coordinator module to contextualize the email data from the email module with probe data from the probe module to create a combined data set for analysis (block 706). The cyber threat protection platform may employ a cyber threat module configured to analyze the combined data set using at least one machine learning model to identify behavior on the network that deviates from normal, benign behavior (block 708). The at least one machine learning model is trained on normal, benign behavior of network entities. The at least one machine learning model identifies the deviant behavior using a normal behavior benchmark as a benchmark for at least one parameter that corresponds to a normal pattern of activity on the network.

[0135] The cyber threat protection platform includes a comparison module that compares the combined data set, including the third party event data, to at least one machine learning model to identify behavior on the network that deviates from normal, benign behavior for the network entity (block 710). The comparison module can identify whether the network entity is in violation of a normal behavior benchmark (block 712). The cyber threat module can identify whether the violation and a set of associated behavior parameters that deviate from the normal, benign behavior for the network entity correspond to a cyber threat (block 714).

[0136] The cyber threat protection platform may employ a user interface module configured to present a graphical representation of the cyber threat within a graphic user interface (block 716). The cyber threat protection platform may employ an autonomous response module configured to select an autonomous response to take in response to the cyber threat (block 718). The autonomous response may be, for example, reducing the permissions of the network entity or disabling a user account of the network entity. The autonomous response module may send an alert of the cyber threat to an internal system administrator or a third party operator along with a proposed response to the cyber threat (block 720). The autonomous response module may execute the autonomous response in response to the cyber threat (block 722).

[0137] FIG. 8 illustrates third party event data. The network event data represents various administrative events. The administrative event can be a login event 802 describing a user logging in to a user account of an online application or service. The administrative event can be a failed login event 804 describing a user's failure to log in to a user account of an online application or service. The administrative event can be a resource creation event 806 describing the creation of a virtual instance of an online application. The administrative event can be a resource view event 808 describing the viewing of a virtual instance of an online application. The administrative event can be a resource modification event 810 describing the modification of a virtual instance of an online application. The administrative event can be a resource deletion event 812 describing the deletion of a virtual instance of an online application. The administrative event can be a file upload event 814 describing the uploading of a file to an online application. The administrative event can be a file download event 816 describing the downloading of a file from an online application. The administrative event can be an administrative action event 818 describing an action at an administrative level for an online application.

[0138] The cyber threat protection platform may obtain management events using a variety of methods. The network module may pull management events from the client device on an event-by-event basis. FIG. 9 illustrates a flow chart of an embodiment of a method for pulling data from a client device. The network module is configured to direct one or more connectors to send a HyperText Transfer Protocol Secure (FIT TPS) event request to the client network (block 902). The HTTPS event request requests management events from an audit log of the client network. The one or more connectors generate the HTTPS event request (block 904). The one or more connectors send the HTTPS event request to the client network to request the management events (block 906). The network module is configured to receive management events from the one or more connectors in response to the event request (block 908). The network module is configured to obtain metadata for the management events (block 910).

[0139] The autonomous response module can autonomously determine a response using the threat risk parameters generated by the cyber threat module. FIG. 10 illustrates a flow chart of an embodiment of a method for identifying an autonomous response. The cyber threat protection platform can have a cyber threat module configured to generate a threat risk parameter that enumerates a set of values ​​that describe an aspect of a cyber threat (block 1002). The cyber threat protection platform can have an autonomous response module configured to generate a benchmark matrix having a set of benchmark scores (block 1004). The autonomous response module can identify tagged users associated with the cyber threat (block 1006). The autonomous response module can lower a threshold for the autonomous response upon identifying tagged users associated with the cyber threat (block 1008). The autonomous response module can compare the threat risk parameters to the benchmark matrix to determine an autonomous response (block 1010). The autonomous response module can determine an autonomous response based on the comparison (block 1012).

[0140] The cyber threat defense platform can generate threat risk parameters to describe the relative danger of anomalous events. FIG. 11 illustrates a block diagram of threat risk parameters. The threat risk parameters can have a threat type 1102 describing the type of threat identified, such as financial, administrative, information technology, production, or other. The threat risk parameters can have a confidence score 1104 indicating the likelihood of a violation, describing the probability that the template entity is in a violation state. The threat risk parameters can have a severity score 1106 indicating the percentage that the template entity in a violation state deviates from normal behavior as represented by at least one model. The threat risk parameters can have a consequence score indicating the severity of damage resulting from the violation state.

[0141] FIG. 12 illustrates a flow chart of an embodiment of a method for generating a threat risk parameter. The cyber threat module may generate a threat risk parameter that enumerates a set of values ​​that describe aspects of a violation state (block 1202). The cyber threat module may identify a threat type of a cyber threat by using various clustering techniques to group the threat with other identified cyber threats (block 1204). The cyber threat module may generate a confidence score (block 1206). The cyber threat module may generate a severity score (block 1208). The cyber threat module may generate a consequence score (block 1210). The cyber threat module may populate the threat risk parameter with at least one of the confidence score, the severity score, and the consequence score (block 1212).

[0142] FIG. 13 illustrates a block diagram of a benchmark matrix. The autonomous response module in conjunction with the cyber threat module can populate the benchmark matrix with varying benchmarks that can adapt to the changing nature of both the network and the threats to the network. The benchmark matrix can have confidence benchmarks 1302 indicating the likelihood of a violation that describe the probability that a template entity is in a violation state. The benchmark matrix can have severity benchmarks 1304 indicating the percentage that a template entity is in a violation state. The benchmark matrix can have outcome benchmarks 1306 indicating the severity of damage resulting from a violation state that should be exceeded for immediate action to be taken. The autonomous response module can adjust these benchmarks as more data is added and greater user input is received.

[0143] The autonomous response module may assign a weight to each benchmark score to assign a relative importance to each benchmark score to factor into the decision to send an inoculation notification. As with the benchmarks, these weights may evolve over time. For example, a benchmark matrix may have confidence weights 1308 indicating the importance of the confidence benchmark, severity weights 1310 indicating the importance of the severity benchmark, and outcome weights 1312 indicating the importance of the outcome benchmark. Using these assigned weights, different deviations from the benchmarks may have better consequences for the final decision and inoculation notification to send.

[0144] 14 illustrates a flow chart of one embodiment of a method for comparing analyzed input data to benchmarks to trigger an inoculation notification. The autonomous response module may generate a benchmark matrix having a set of benchmark scores to determine an autonomous response (block 1402). The autonomous response module may populate the benchmark matrix with the benchmark scores based on data collected during the violation identification process (block 1404). The autonomous response module may assign a weight to each benchmark score to assign a relative importance to each benchmark score (block 1406).

[0145] Interest Classifier The cyber threat detection platform is configured to perform packet inspection by analyzing a subset of each possible connection. One approach to monitoring connections is to process and inspect all connection traffic to the client device. This approach may not be computationally sensible because not all connections are "interesting" or can be parsed in their entirety. An alternative approach is to perform connection-specific deep packet inspection and processing. In this approach, a traffic manager module, such as in a host-based agent, virtualized sensor, centralized physical device, or centralized cloud device, can process connections differently based on how interesting they are, how much information the cyber threat detection platform can parse from the protocol, and whether the security team wants to see the connection. This approach branches or filters out connections of no interest to save computation, automatically decodes or generates packet captures (PCAPs) for connections of interest, and parses only the metadata for connections that do not offer significant value. The traffic manager module can "shuffle" connections. In other words, the network card can stop processing the actual content of the connection and only provide metadata such as packet counts or bytes, greatly reducing computation time.

[0146] A connection-specific approach can work with a network card that not only processes traffic but is aware of the flow through the network card, such as volume, connection type, or protocol. A connection-specific approach can shunt a connection based solely on metadata being available, such as one involving some encryption protocol, or based on the connection being a large connection that is deemed uninteresting. A connection-specific deep packet inspection and processing approach can also "de-shunt" a connection if it becomes interesting. For example, a cyber threat protection platform can instruct a network card to start processing data again because the data connection is deemed interesting. A cyber threat protection platform can identify an interesting connection based on the connection being anomalous within the context of the device's past behavior or device peer group behavior, multiple connections with similar characteristics across several devices adding to the overall anomaly, or subsequent behavior changing the "interesting" state of an initially uninteresting connection. The autonomous behavior module can then terminate the connection or check that the connection was successfully blocked by capturing data about the connection in a spoofed reset (RST) packet, or feed that information to a third-party firewall. An interesting connection may also be one that is sufficiently anomalous that a security system, decryption method, or team may want to decrypt the data at the packet level in a packet capture. Computationally expensive and slow decryption is reserved for connections with the highest investigative value. Deep packet inspection engines can then collect packet level data so security teams can do so.

[0147] FIG. 15 illustrates a block diagram of a physical traffic manager module. A centralized physical device may have a traffic manager module that uses a network card. The network card may register a connection from a network and send a series of one or more data packets. The network card may analyze the one or more data packets to identify a potential cyber threat. After analysis by the network card, the network card may pass the data packets to a processor for decoding and processing. The processor may send any data to the network by passing the data to an offload module. The offload module may packetize the data and send new data packets to the network.

[0148] The network card may have a registration module to register a connection between one or more devices in the client network and transmit a sequence of one or more data packets. The network card may have a classifier module configured to perform a comparison of characteristics of the connection to a set of interest criteria to determine a cyber threat protection platform interest in the connection. The classifier module may adjust the set of interest criteria based on a set of host parameters for the client network. The set of host parameters may be at least one of storage capacity, processing power, and network bandwidth. The classifier module may be configured to apply an interest classifier that describes an interest in the connection based on the comparison. The network card may have a deep packet inspection (DPI) module to test one or more data packets of the connection for cyber threats if the interest classifier indicates interest. The network card may have a diverter configured to divert one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates no interest.

[0149] FIG. 16 illustrates a block diagram of a virtual traffic manager module. A host-based agent, a hypervisor, or a centralized cloud appliance may have a traffic manager module that uses a virtual netmap module. The netmap module may register connections from a network and send a series of one or more data packets. The netmap module may analyze the one or more data packets to identify potential cyber threats. After analysis by the netmap module, the netmap module may pass the data packets to a processor for decoding and processing. The processor may send any data to the network by passing the data to an offload module. The offload module may packetize the data and send new data packets to the network.

[0150] The virtual netmap module may include a registration module for registering a connection between one or more devices in the client network and transmitting a series of one or more data packets. The virtual netmap module may include a classifier module configured to perform a comparison of characteristics of the connection to a set of interest criteria to determine a cyber threat protection platform interest in the connection. The classifier module may adjust the set of interest criteria based on a set of host parameters for the client network. The set of host parameters may be at least one of storage capacity, processing capacity, and network bandwidth. The classifier may be configured to apply an interest classifier that describes an interest in the connection based on the comparison. The virtual netmap module may include a deep packet inspection (DPI) module to test one or more data packets of the connection for cyber threats if the interest classifier indicates interest. The virtual netmap module may include a diverter configured to divert one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates no interest.

[0151] 17 illustrates a flow chart of one embodiment of a method for establishing interest criteria. A traffic manager module may receive a set of interest criteria from an analyzer module of a cyber threat protection platform (block 1702). A classifier module of the traffic manager module may determine a set of host parameters for the client device (block 1704). The classifier module may adjust the set of interest criteria based on the set of host parameters (block 1706). The classifier module may store the set of interest criteria for future use (block 1708).

[0152] FIG. 18 illustrates a flow chart of an embodiment of a method for processing a data connection with a deep packet inspection engine. A registration module of the traffic manager module may register a connection between one or more devices in a client network to transfer a series of one or more data packets (block 1802). A classifier module of the traffic manager module may perform a comparison of characteristics of the connection to a set of interest criteria to determine a cyber threat protection platform interest in the connection (block 1804). The classifier module may determine that the connection is of interest based on at least one of the following: a short-term connection, being decoded within a parameter set for the client device, or outside of a normal connection pattern (block 1806). The classifier module may apply an interest classifier to the connection that describes the interest as of interest based on the comparison (block 1808). The diverter of the traffic manager module may pass one or more data packets of the connection to a deep packet inspection engine for further testing for cyber threats if the interest classifier indicates an interest (block 1810). The deep packet inspection engine may collect a set of packet metadata for the one or more data packets of the pass-through connection (block 1812). The offload module of the traffic manager module may send the set of packet metadata to an analyzer module of the centralized cyber threat protection platform when processing is performed outside of the centralized cyber threat protection platform (block 1814).

[0153] FIG. 19 illustrates a flow chart of an embodiment of a method for shunting a data connection beyond a deep packet inspection engine. A registration module of the traffic manager module may register a connection between one or more devices in a client network to forward a series of one or more data packets (block 1902). A classifier module of the traffic manager module may perform a comparison of characteristics of the connection to a set of interest criteria to determine a cyber threat protection platform interest in the connection (block 1904). The classifier module may determine that the connection is of no interest based on at least one of the connection being a long-term connection, being unable to be decoded within a parameter set for the client device, and being within a normal connection pattern (block 1906). The classifier module may apply an interest classifier to the connection that describes an interest as of no interest based on the comparison (block 1908). A diverter of the traffic manager module may divert one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates no interest (block 1910). A processing module of the client device may collect a set of packet metadata for one or more data packets of the diverted connection (block 1912). An offload module of the traffic manager module may send the set of packet metadata to an analyzer module of the centralized cyber threat protection platform when processing is performed outside of the centralized cyber threat protection platform (block 1914). The classifier module may monitor at least one of a connection length and a payload size for the diverted connection (block 1916).

[0154] FIG. 20 illustrates a flow chart of one embodiment of a method for processing a data connection with a data packet drop. A traffic manager module can receive a pass-through connection in a deep packet inspection engine (block 2002). The deep packet inspection engine can identify a packet drop in the pass-through connection (block 2004). A classifier module can apply an updated interest classifier that describes a different degree of interest based on the packet drop (block 2006). A diverter can divert the pass-through connection with the packet drop away from the deep packet inspection engine (block 2008).

[0155] FIG. 21 illustrates a flow chart of an embodiment of a method for addressing a data connection during an anomalous event. A cyber threat module of the cyber threat protection platform may detect an anomalous event at a client device (block 2102). A diverter of the traffic manager module may reconnect the diverted connection to a deep packet inspection engine upon detection of an anomalous event at the client device (block 2104). An autonomous behavior module may obtain data regarding the connection in a spoofed reset (RST) packet and terminate the connection or check that the connection is successfully blocked (block 2104). The autonomous behavior module may disconnect the connection upon detection by the analyzer module of an anomalous event at the client device (block 2108). A classifier module of the traffic manager module may adjust a set of interest criteria based on the anomalous event (block 2110).

[0156] Host-Based Decryption The cyber threat detection platform is configured to data mine the communication protocol using decryption to protect the network from cyber threats in this network using encrypted communication protocols. The demand for encryption of Domain Name System (DNS) traffic and other protocols is increasing as the security risks of clear text protocols become prominent. Host-based traffic decryption approaches decryption in a deep packet inspection (DPI) engine in three ways. First, the host-based agent can receive a private key from a third-party proxy or agency. Second, the host-based agent can upload a public / private key pair via a secure shell console or other interface to a centralized device associated with the host-based agent. Third, the host-based agent can retrieve the key from a client device. The third-party proxy or agency works with the device-host-based cyber threat detection and response platform, which uses a universal translator to command the third-party systems and retrieve data from them. Host-based key retrieval consists of two possible approaches: process memory retrieval or personal firewall proxy.

[0157] In the process memory acquisition approach, a module of the host-based agent observes when a new connection is opened on port 443 and notifies another module of the process that a connection has been opened. This process memory acquisition approach is not limited to this port, but rather secure transfers are made using port 443, which is the standard port for HTTPS traffic. Another module locates the memory for the process that opened the connection and scans the memory for a pattern that may be an encryption key. The other module then passes this key through a secure system to the cyber threat detection platform, which routes the key and process information to the correct device and the correct module, such as a DPI engine, where the private key can be matched to the observed traffic and decrypted. Personal firewall proxy mode is when the host-based agent acts as a personal firewall and traffic is controlled on a per-device basis. The host-based agent acts as a "man in the middle" to the traffic and either sees the traffic in a decrypted form or obtains the keys before they leave the device. The personal firewall proxy mode can use an endpoint agent that is performing process analysis and acting as a man-in-the-middle proxy.

[0158] FIG. 22 illustrates a flow chart of one embodiment of a method for using host-based decryption for a data connection on a client device. An analyzer module of the cyber threat defense platform may determine that the client device allows host-based traffic decryption (block 2202). The analyzer module may consider endpoint rarity, timing rarity, domain rarity, or environment in making this determination. The analyzer module may flag the client device for host-based traffic decryption (block 2204). A deep packet inspection engine may perform the decryption at the host-based agent (block 2206). The deep packet inspection engine may receive a private key from a third-party agent. Alternatively, the deep packet inspection engine on the host-based agent may forward the encrypted traffic to a centralized cybersecurity defense platform provisioned with the public / private key pair, allowing a deep packet inspection engine located at the cybersecurity defense platform to perform the decryption and processing instead. Otherwise, the deep packet inspection engine could obtain the private key from the client network.

[0159] External storage for packet data of interest The cybersecurity defense platform can work with storage devices, such as databases on the Internet or private databases in the cloud, to store packet data of interest for longer and more thorough analysis, and as a result, the amount of storage is minimized as a concern. The cybersecurity defense platform can store a large amount of packets in the storage compared to just the cybersecurity platform's own storage. Furthermore, since the storage is not limited in storage size, the storage can store packets for a longer duration. Additionally, the storage can perform more computational artificial intelligence on the data because of the external storage. The cybersecurity defense platform can write packet data processed by the deep packet inspection (DPI) engine to a time-limited storage for retrieval by an operator who wishes to conduct a detailed investigation. Due to the vast amount of data traversing the monitored network and being subjected to deep packet inspection, only a limited amount of data tends to be stored. Thus, packet data is retained based on interest. Data may be configured for tiered expiration based on interest, with interesting data being stored in long-term storage and connections with lower levels of "interest" expiring sooner. Interest may be derived from protocol irregularities, irregularities in the connection's factors (source, destination, timing, etc.), or any number of additional metrics. A user may identify packets with x-type metrics to be sent to external storage through a user interface. A classifier module may augment user input with a classifier for default identification of interesting data packets.

[0160] Performance of artificial intelligence-based inspection of individual byte-level data is too computationally expensive and too slow when packet data is kept in a centralized cybersecurity platform or on a virtualized probe, requiring performance by processes residing on the platform or probe. When stored externally in secure extended storage, external infrastructure can be utilized to perform artificial intelligence analysis on the packet data without sharing computational power with the probe or cybersecurity platform. The external infrastructure can be locally located virtual machines or machine learning micro-services, such as Amazon AWS machine learning. Analysis can include inspection of byte-level strings for anomalous use of Transformer or other deep learning models.

[0161] Safe expansion of storage capacity and some data types maintained within the cybersecurity appliance can in most cases directly collaborate and communicate with the cybersecurity defense platform. The cybersecurity defense platform has a user interface on the display to interface with the end user. The cybersecurity defense platform securely connects to and communicates with separate external storage, whether physical or virtualized. For example, external storage, cloud-based Simple Storage System (S3) buckets in the same Virtual Private Cloud (VPC) or in a Virtual Private Cloud managed by the organization supplying the cybersecurity defense platform. Virtualized probes performing deep packet inspection write connections of interest directly to this outside of the probe storage to be accessed by the connected cybersecurity defense platform via application programming interface calls, tunnels with multiple factor validation, connections through the connected probe, or additional communication methods. This virtualization can allow the probes to auto-scale as traffic volumes increase or decrease without data corruption.

[0162] An additional benefit of this approach is larger and longer term storage. Additional storage clusters or blobs can be created to address demand. An option for standard cybersecurity defense platform deployments can utilize external packet storage in a managed VPC as an additional service. Packet data can also be queried or searchable via application programming interfaces from compatible services such as intelligence tools in the customer network. Finally, such smaller subsets of data enable potential machine learning approaches that are not feasible for larger data sets due to computational expense.

[0163] FIG. 23 illustrates a flow chart of one embodiment of a method for off-site storage of a packet capture from a data connection to a client device. The deep packet inspection engine may collect a packet capture of one or more data packets for the connection (block 2302). The deep packet inspection engine may set an expiration time for the packet capture within the Cloud Simple Storage System indicating when the packet capture may be overwritten (block 2304). The deep packet inspection engine may send the packet capture to the Cloud Simple Storage System for storage (block 2306).

[0164] Website The website may be configured as a browser-based or direct-app tool for configuring, analyzing, and communicating with the cyber threat defense platform.

[0165] network Several electronic systems and devices can communicate with each other in a network environment. The network can include at least one firewall, at least one network switch, a plurality of computing devices operable by users of the network, a cyber threat coordinator component, and a host-based agent. FIG. 24 illustrates a schematic diagram of a networked environment. The network environment has a communications network. The network can include one or more networks selected from an optical network, a cellular network, the Internet, a local area network ("LAN"), a wide area network ("WAN"), a satellite network, a third party "cloud" environment, a fiber network, a cable network, and combinations thereof. In some embodiments, the communications network is the Internet. There can be many server computing systems and many client computing systems connected with each other via the communications network.

[0166] The communication network may connect one or more server computing systems selected from at least a first server computing system and a second server computing system with each other and with at least one or more client computing systems. Each of the server computing systems may optionally include an organizational data structure such as a database. Each of the one or more server computing systems may have one or more virtual server computing systems, and multiple virtual server computing systems may be implemented by design. Each of the one or more server computing systems may have one or more firewalls and similar defenses to protect data integrity.

[0167] At least one or more client computing systems, e.g., mobile computing devices (e.g., smartphones having an Android-based operating system), can communicate with the server. The client computing systems may include, for example, software applications or hardware-based systems that may be able to exchange communications with the first electric personal transportation vehicle and / or the second electric personal transportation vehicle. Each of the one or more client computing systems may have one or more firewalls and similar defenses to protect data integrity.

[0168] A cloud provider platform may include one or more of the server computing systems. A cloud provider may install and run application software within the cloud (e.g., a network such as the Internet), and a cloud user may access the application software from one or more of the client computing systems. In general, a cloud user with a cloud-based site within the cloud may not have sole control over the cloud infrastructure or platform on which the application software runs. Thus, the server computing system and its organizational data structures may be a shared resource, with each cloud user being given dedicated use of a certain amount of the shared resource. Each cloud user's cloud-based site may be given dedicated space and bandwidth of a virtual amount within the cloud. Cloud applications may differ from other applications in scalability, which may be achieved by mimicking tasks on multiple virtual machines at runtime to meet changing work demands. A load balancer distributes work to a set of virtual machines. This process is transparent to cloud users who only see a single access point.

[0169] The cloud-based remote access may be coded to engage in request and response cycles by applications on a client computing system, such as a web browser application in the client computing system, utilizing a protocol such as the Hypertext Transfer Protocol ("HTTP"). The cloud-based remote access may be accessed anytime and / or anywhere by a smart phone, desktop computer, tablet, or any other client computing system. The cloud-based remote access may be coded to engage in: 1) request and response cycles from any web browser-based application; 3) request and response cycles from a dedicated online server; 4) request and response cycles directly between a native application in a client device and a cloud-based remote access to another client computing system; and 5) combinations thereof.

[0170] In an embodiment, the server computing system may include a server engine, a web page management component, a content management component, and a database management component. The server engine may perform basic processing and operating system level tasks. The web page management component may handle the creation and display or routing of web pages or screens associated with receiving and serving digital content and digital advertisements. Users (e.g., cloud users) may access one or more of the server computing systems by their associated uniform resource locators ("URLs"). The content management component may handle most of the functionality within the embodiments described herein. The database management component may include storage and retrieval tasks related to databases, queries to databases, and storage of data.

[0171] In some embodiments, the server computing system may be configured to display information in windows, web pages, and the like. For example, applications, including any program modules, applications, services, processes, and other similar software executable when executed on the server computing system, may cause the server computing system to display windows and user interface screens within a portion of the display screen space. With respect to web pages, a user, for example, via a browser on a client computing system, may interact with the web page and then provide inputs to queries / fields and / or services presented in the user interface screen. The web pages may be served by a web server, for example, the server computing system, on a HyperText Markup Language ("HTML") or Wireless Access Protocol ("WAP") enabled client computing system (e.g., client computing system 802B) or any equivalent thereof. The client computing system may host a browser and / or specific applications to interact with the server computing system. Each application has code written to perform functions in which software components are coded to perform presentation fields, etc. to retrieve desired information details. For example, algorithms, routines, and engines in a server computing system can take information from the submitted fields and place it in a suitable storage medium (e.g., a database), such as a database. A comparison wizard can be written to reference the database and use such data. An application can be hosted, for example, on the server computing system and served, for example, to a specific application or browser on the client computing system. The application then serves a window or page that allows entry of the details.

[0172] Computing Systems The computing system may be, in whole or in part, part of one or more of a server or client computing devices, according to some embodiments. The components of the computing system may include, but are not limited to, a processing unit having one or more processing cores, a system memory, and a system bus coupling various system components, including the system memory, to the processing unit. The system bus may be any of several types of bus structures selected from a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.

[0173] A computing system typically includes a variety of computing machine readable media. Computing machine readable media can be any available media that can be accessed by a computing system and includes both volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, computing machine readable media uses include storage of information such as computer readable instructions, data structures, other executable software, or other data. Computer-storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic disk storage devices, or any other tangible media that can be used to store desired information and that can be accessed by the computing device 900. Transient media such as wireless channels are not included in machine readable media. Communication media typically embodies computer readable instructions, data structures, other executable software, or other transport mechanisms and includes any information delivery media.

[0174] System memory includes computer storage media in the form of volatile and / or nonvolatile memory such as read-only memory (ROM) and random access memory (RAM). A basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within a computing system, such as during start-up, is typically stored in ROM. RAM typically contains data and / or software that is immediately accessible to and / or presently being operated on by the processing unit. By way of example, and not limitation, RAM may include portions of the operating system, application programs, other executable software, and program data.

[0175] The drives and their associated computer storage media discussed above provide storage of computer readable instructions, data structures, other executable software and other data for the computing system.

[0176] A user may input commands and information into a computing system through input devices such as a keyboard, a touch screen, or software or hardware input buttons, a microphone, a pointing device, and / or a scrolling input component such as a mouse, trackball, or touchpad. The microphone may be coupled with voice recognition software. These and other input devices are often connected to the processing unit through a user input interface that is coupled to the system bus, but may also be connected by other interface and bus structures such as a parallel bus, a game port, or a universal system bus (USB). A display monitor or other type of display screen device is also connected to the system bus through an interface such as a display interface. In addition to a monitor, a computing device may also include other peripheral output devices such as speakers, vibrators, lights, and other output devices that may be connected through an output peripheral interface.

[0177] The computing system may operate in a networked environment using logical connections to one or more remote computers / client devices, such as remote computing systems. The logical connections may include a personal area network ("PAN") (e.g., Bluetooth), a local area network ("LAN") (e.g., Wi-Fi), and a wide area network ("WAN") (e.g., cellular networks), but may also include other networks. Such networking environments are common in offices, enterprise-wide computer networks, intranets, and the Internet. Browser applications or direct apps that interact with the cloud platform may reside in the computing device and may be stored in memory.

[0178] It should be noted that the design may be executed on a single computing system and / or on a distributed system where different parts of the design are executed on different parts of the distributed computing system.

[0179] Applications described herein include, but are not limited to, programs that are part of software applications, mobile apps, and operating system applications. Some portions of this description are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, considered to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, transferred, combined, and otherwise manipulated. It has proven convenient at times, primarily for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. These algorithms may be written in a number of different software programming languages, such as Python, C, C+, or other similar languages. Also, the algorithms may be implemented with lines of code in software, configured logic gates in software, or a combination of both. In embodiments, logic consists of electrical circuitry that follows the rules of Boolean logic, software containing patterns of instructions, or any combination of both.

[0180] It should be noted, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.Unless otherwise clearly indicated as will be apparent from the above description, descriptions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" will be understood throughout the description to refer to the acts and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities in the computer system's registers and memory into other data similarly represented as physical quantities in the computer system's memory or registers, or other such information storage, transmission or display device.

[0181] Many functions performed by electronic hardware components can be replicated through software emulation. Thus, software programs written to accomplish those same functions can emulate the functionality of the hardware components in the input-output circuitry.

[0182] Although the foregoing designs and embodiments thereof have been provided in considerable detail, it is not the intention of the applicants to limit the designs and embodiments provided herein. Additional adaptations and / or assembly are possible and are encompassed in their broadest aspects. Thus, the foregoing designs and embodiments may be prepared without departing from the scope created by the following claims, which, when properly interpreted, are limited only by the claims.

Claims

1. 1. A computing device-implemented method for a cyber threat defense system for distinguishing data flows, comprising: registering connections between one or more devices in a client network and forwarding a sequence of one or more data packets in a traffic manager module of the cyber threat protection system; performing a comparison of characteristics of the connection to a set of interest criteria to determine a degree of interest of the cyber threat protection system in the connection, the set of interest criteria being adjusted based on a set of host parameters for the client network; applying an interest classifier that describes the degree of interest in the connection based on the comparison; if the interest classifier indicates interest, passing the one or more data packets of the connection to a deep packet inspection engine for further testing for cyber threats; and if the interest classifier indicates disinterest, diverting the one or more data packets of the connection away from the deep packet inspection engine.

2. 2. The method for a cyber threat protection system of claim 1, further comprising determining that the connection is uninteresting based on at least one of the following: the connection is a long-term connection, cannot be decoded within a parameter set for the client device, and is within a normal connection pattern.

3. 10. The method for a cyber threat protection system of claim 1, further comprising monitoring at least one of a connection length and a payload size for the shunted connections.

4. 10. The method for a cyber threat protection system of claim 1, further comprising the step of collecting a set of packet metadata for the shunted connection.

5. identifying dropped packets within a pass-through connection being processed by the deep packet inspection engine; diverting the pass-through connection with the dropped packet away from the deep packet inspection engine; The method for a cyber threat protection system of claim 1 , further comprising:

6. The method for a cyber threat protection system described in claim 1, further comprising a step of severing the connection upon detection by the analyzer module of an anomalous event at a client device.

7. 7. The method for a cyber threat protection system of claim 6, further comprising the step of obtaining data regarding a connection to a spoofed reset packet.

8. The method for a cyber threat protection system as described in claim 1, further comprising the step of reconnecting the diverted connection to the deep packet inspection engine upon detection by the analyzer module of an anomalous event at a client device.

9. The method for a cyber threat protection system of claim 8 , further comprising adjusting the set of interest criteria based on the anomalous event.

10. 13. A non-transitory computer-readable medium comprising computer-readable code operable, when executed by one or more processing units within the cyber threat protection system, to instruct a computing device to perform the method of claim 1.

11. A traffic manager module for a cyber threat protection system, comprising: a registration module configured to register a connection between one or more devices in a client network and transmit a sequence of one or more data packets; A classifier module, performing a comparison of connection characteristics to a set of interest criteria based on a set of host parameters for the client network, the set including at least one of storage capacity, processing power, and network bandwidth; and a classifier module configured to determine a degree of interest of a cyber threat protection system in the connection and to apply an interest classifier describing the degree of interest in the connection based on the comparison; a deep packet inspection engine configured to examine the one or more data packets of the connection for cyber threats if the interest classifier indicates interest; a diverter configured to divert the one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates disinterest. A traffic manager module comprising:

12. 12. The traffic manager module of claim 11, wherein the deep packet inspection engine is configured to collect a packet capture of the one or more data packets for the connection.

13. 13. The traffic manager module of claim 12, further comprising an offload module configured to transmit the packet capture to a cloud storage system.

14. 14. The traffic manager module of claim 13, wherein the offload module is configured to set an expiration time for the packet capture in the cloud storage system indicating when the packet capture may be overwritten.

15. 12. The traffic manager module of claim 11, wherein the traffic manager module is located on at least one of a host-based agent, a virtualized sensor installed on a hypervisor, a centralized physical appliance, and a centralized cloud appliance.

16. A network, At least one firewall; at least one network switch; a plurality of computing devices operable by users of the network; a cyber threat coordinator component, a probe module configured to collect input data from one or more probes deployed on one or more network devices describing network managed activities performed by the network devices; a cyber threat module configured to identify whether the input data corresponds to a cyber threat to the network; an analyzer module configured to flag host-based agents for host-based traffic decryption; and a traffic manager module, a registration module configured to register a connection between one or more devices on the network and transfer a sequence of one or more data packets; a classifier module configured to perform a comparison of the characteristics of the connection with a set of interest criteria adjusted based on a set of host parameters of a client network to determine a degree of interest of the cyber threat protection system in the connection, and to apply an interest classifier describing the degree of interest in the connection based on the comparison; a deep packet inspection engine configured to examine the one or more data packets of the connection for cyber threats if the interest classifier indicates interest, and to perform host-based traffic decryption of the one or more data packets for the connection; a diverter configured to divert the one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates disinterest; A network comprising:

17. 17. The network of claim 16, wherein the analyzer module of the cyber threat coordinator component is configured to determine to allow host-based traffic decryption by the host-based agent based on at least one of endpoint rarity, timing rarity, domain rarity, and environment.

18. 17. The network of claim 16, wherein the deep packet inspection engine of the host-based agent is configured to perform decryption by at least one of receiving a private key from a third party agent, uploading a public / private key pair to a centralized device associated with the host-based agent, and obtaining a private key from a client network.

19. A traffic manager module for a cyber threat defense system, comprising: a registration module configured to register a connection between one or more devices in the client network and to transmit a sequence of one or more data packets; A classifier module, performing a comparison of characteristics of the connection to a set of interest criteria based on a set of host parameters for the client network; and a classifier module configured to determine an interest of the cyber threat protection system in the connection and to apply an interest classifier describing the interest in the connection based on the comparison; a deep packet inspection engine configured to examine the one or more data packets of the connection for cyber threats if the interest classifier indicates interest; a diverter configured to divert the one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates disinterest. A traffic manager module comprising:

20. A traffic manager module for a cyber threat defense system, comprising: a registration module configured to register a connection between one or more devices in the client network and to transmit a sequence of one or more data packets; a classifier module configured to perform a comparison of the characteristics of the connection with a set of interest criteria to determine a degree of interest of the cyber threat protection system in the connection, and to apply an interest classifier describing the degree of interest in the connection based on the comparison; a deep packet inspection engine configured to collect a packet capture and, if the interest classifier indicates an interest, test the one or more data packets of the connection for cyber threats; a diverter configured to divert the one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates disinterest; an offload module configured to transmit the packet capture to a cloud storage system; A traffic manager module comprising:

21. A traffic manager module for a cyber threat defense system, comprising: a registration module configured to register a connection between one or more devices in the client network and to transmit a sequence of one or more data packets; a classifier module configured to perform a comparison of the characteristics of the connection with a set of interest criteria to determine a degree of interest of the cyber threat protection system in the connection, and to apply an interest classifier describing the degree of interest in the connection based on the comparison; a deep packet inspection engine configured to collect a packet capture and, if the interest classifier indicates an interest, test the one or more data packets of the connection for cyber threats; a diverter configured to divert the one or more data packets of the connection away from the deep packet inspection engine if the interest classifier indicates disinterest; an offload module configured to send the packet capture to a cloud storage system and set an expiration time for the packet capture indicating when the packet capture may be overwritten; A traffic manager module comprising:

Images