Cyber ​​threat detection based on threat context, threat evolution, and / or impact status

Abstract

The embodiments described herein may relate to the detection of cyber threats based on threat context and / or changes in the threat. Cyber ​​threat intelligence (CTI) data may be received from a CTI provider. Endpoint data indicating evidence that an endpoint is a cyber threat may be determined based on the CTI data. Endpoint data may be analyzed and / or compared with stored data associated with the endpoint. Analysis and / or comparison may be performed to determine whether the evidence that the endpoint is a cyber threat has changed. Based on the change, penalties against the endpoint may be determined and sent. Penalties may change how the device filters network traffic associated with the endpoint. Alternative penalties to the default penalties may be determined based on their impact on blocking potentially legitimate network traffic to and / or from the endpoint. Machine learning models may assist in processing and analyzing CTI data, performing threat monitoring, and / or determining the feed, including penalties.

Description

Related applications

[0001] This application claims the interests of U.S. Provisional Patent Application No. 63 / 472,519, filed on 12 June 2023, titled “Detection of Cyber ​​Threats Based on Threat Context and / or Evolving Threat,” which is incorporated herein by reference in its entirety. [Background technology]

[0002] Threats to the security of computer networks, as well as other cyber threats, can take various forms (e.g., attempts to cause unauthorized data transfer, hacking attempts, viruses, bots, and other types of malware). Similar to the efforts of malicious actors to exploit vulnerabilities in computer network security, the scope of such threats continues to expand. Considering the expansion and continuous efforts of malicious actors in this way, when attempting to detect network security threats and other cyber threats, the problems to be addressed are evolving. In some systems, cyber threat intelligence (CTI) data is received from various CTI providers, feeds are assembled based on the CTI providers, and dispositions are determined regarding how to handle network traffic based on the overall CTI data received from the CTI providers. Such dispositions can include blocking network traffic, monitoring network traffic, etc. These processes of receiving CTI data, assembling feeds, and determining dispositions can be executed for the purpose of blocking malicious network traffic as much as possible without affecting non-malicious network traffic. However, since cyber threats are dynamic and can change over time, it can sometimes be difficult to achieve both objectives. There may be little information about newly emerging threats. Therefore, if waiting for the CTI provider to send additional CTI data regarding a new threat, there is a possibility that malicious network traffic will be permitted before being blocked by the cyber threat detection system. Also, if the cyber threat detection system quickly blocks a new threat without waiting for the CTI provider to send additional CTI data, as a result, non-malicious network traffic may be blocked. Permitting malicious network traffic and blocking non-malicious network traffic are both undesirable outcomes. SUMMARY OF THE INVENTION

[0003] The following is a simplified overview of the various embodiments described herein. This overview is not intended to be a comprehensive overview, nor is it intended to identify any key elements or define the scope of the claims. The following overview is merely a simplified introduction to some concepts, serving as a prelude to the more detailed description that follows.

[0004] The embodiments described herein can address one or more problems in cyber threat detection and / or can generally improve systems that perform cyber threat detection. For example, some embodiments described herein enable the detection of cyber threats based on changes in the threat. More specifically, CTI data is received from a provider and that CTI data may contain evidence that an endpoint is a cyber threat, or it may not contain evidence but still indicate that an endpoint is a cyber threat. This evidence can be compared with other previously received evidence that the endpoint is a cyber threat. Based on that comparison, a determination is made as to whether the evidence has changed, and if so, a disposition is determined and sent for that endpoint.

[0005] As another example, several embodiments described herein enable the detection of cyber threats based on threat context. More specifically, when a new cyber threat is identified from one provider, the system may decide on a disposition indicating that network traffic associated with the new cyber threat should be blocked, after waiting for one or more other providers to also identify the new cyber threat.

[0006] As another example, some embodiments described herein allow for disposition decisions based on individual endpoints. In this manner, network traffic is blocked or permitted based on its association with a specific endpoint that one or more providers have identified as a cyber threat.

[0007] As yet another example, several aspects described herein may use machine learning models to assist in processing CTI data, analyzing CTI data, performing additional analysis, conducting threat monitoring, determining feeds including penalties, and / or determining alternative penalties for endpoints based on impact status indicating the potential impact of blocking legitimate network traffic to and from that endpoint. As one example, a provider may send CTI data in many different formats and provide evidence that an endpoint is a cyber threat in many different ways. A machine learning model may be trained to assist in processing that many different formats of CTI data. Once trained, the machine learning model can be used as part of a process to extract evidence that an endpoint is a cyber threat and provide that evidence in a common format and / or common notation. As another example, a machine learning model may be trained to assist in determining whether blocking potentially legitimate network traffic to and / or from an endpoint would impact the operation of an entity (e.g., business operations). Once trained, the machine learning model can be used as part of a process to determine alternative penalties for endpoints that are not known to be malicious and are not known to be non-malicious.

[0008] These features, along with many other features, are described in more detail below. Corresponding devices, systems, and computer-readable media are also within the scope of this disclosure. [Brief explanation of the drawing]

[0009] This disclosure is illustrative, but is not limited to the attached diagrams described below, in which similar reference numerals indicate similar elements.

[0010] Figure 1 shows a block diagram of an exemplary computer environment that can be configured to detect cyber threats based on threat context and / or changes in threat.

[0011] Figure 2 is a block diagram showing examples of data collection in various forms described herein.

[0012] Figures 3A-3D are block diagrams showing examples related to threat analysis in various forms described herein.

[0013] Figure 4 is a block diagram showing examples related to threat monitoring in various forms described herein.

[0014] Figures 5A-5B are block diagrams showing examples of disposal feeds in various embodiments described herein.

[0015] Figures 6A-6B illustrate examples of methods for receiving, storing, and / or processing CTI data and / or excluded data according to various embodiments described herein.

[0016] Figure 7 shows an example of a method for capturing and analyzing endpoint data in various forms described herein.

[0017] Figure 8 shows examples of threat monitoring methods in various forms described herein.

[0018] Figure 9 shows examples of methods for determining the disposal feed according to various embodiments described herein.

[0019] Figures 10A-10E show example flows for determining examples of disposal according to the various embodiments described herein.

[0020] Figures 11A-11B show examples of time-based exclusion flows according to various embodiments described herein.

[0021] Figure 12 is a block diagram of an exemplary computer environment that may be configured to address cyber threats based on impact status, according to various embodiments described herein.

[0022] Figure 13 shows examples of methods for determining a disposal feed based on impact status, according to various embodiments described herein.

[0023] Figure 14 shows an example of a flow for determining a disposal feed based on impact status, according to various embodiments described herein.

[0024] Figure 15 shows an example of another method for determining a disposal feed based on impact status, according to various embodiments described herein.

[0025] Figure 16 shows an example of a computer device that can be used when carrying out one or more embodiments described herein. Detailed description of the invention

[0026] In the following descriptions of various embodiments, reference is made to the accompanying drawings, which constitute part of this specification and illustrate various embodiments in which the aspects of this disclosure may be carried out. It should be understood that other embodiments may be used and structural and functional modifications may be made without departing from the scope of this disclosure. Other embodiments of the aspects of this disclosure are possible and can be carried out in various ways. It should also be understood that the phrases and terms used herein are for illustrative purposes only and should not be considered limiting. Rather, the words and terms used herein are given the broadest possible interpretation and meaning. The use of “including,” “having,” and variations thereof means that what is listed thereafter and its equivalents are included as well as what is added and its equivalents.

[0027] As an introduction, the aspects described in this specification may relate to methods and techniques for detecting cyber threats based on threat context and / or changes in threats. A cyber threat may include the efforts, or suspected efforts, of one or more threat actors attempting some unauthorized action that affects the computer devices and / or networks of a targeted entity. The threat actors may be criminals, criminal organizations, states, or other individuals or groups. Unauthorized actions may include damage to equipment and / or disabling (e.g., hacking a network control system to disable and / or damage industrial equipment, such as ransomware), access to and / or theft (e.g., leakage) of data, causing the computer device and / or network to perform operations that benefit the threat actor (e.g., bots that generate spam, bots that mine cryptocurrency, bots that perform denial-of-service attacks, etc.), and / or other types of actions not permitted by the targeted entity. Cyber threats may include, but are not limited to, the above general types, and may include viruses and other malware, phishing attacks, attempts to hack web servers, and the like.

[0028] Based on the methods and techniques described herein, one or more problems in cyber threat detection can be addressed and / or systems that perform cyber threat detection can be improved. For example, a cyber threat may be detected based on the threat context and / or changes in the threat. The threat context includes, for example, evidence or other indications that an endpoint is a cyber threat. Such evidence or other indications may be contained in CTI data transmitted from the provider. Furthermore, it can be used to determine whether that evidence is changing over time. Thus, cyber threat detection may be based on the threat context determined from CTI data received over time from the provider and / or whether the threat context indicates changes in the cyber threat. This can improve cyber threat detection by enabling a more accurate response to emerging cyber threats (e.g., blocking increases in malicious network traffic, blocking malicious network traffic more quickly, and / or allowing increases in non-malicious network traffic while new cyber threats emerge).

[0029] In general, cyber threats can evolve and change over time. As more information becomes available about a particular cyber threat, CTI data on that threat can become more accurate and / or reliable. Conversely, older CTI data may become (or be) less reliable. For example, older CTI data may indicate that a cyber threat is not a major problem and / or concern, while more recent CTI data may show that the cyber threat is a bigger problem than initially thought. As another example, older CTI may have provided specific information (e.g., one or more configuration indicators) for detecting a cyber threat, but some of that information may later be determined to be inapplicable and / or irrelevant, while more recent CTI data may contain more focused and / or accurate information. As yet another example, threat actors may change the methods they use to attack networks (e.g., to camouflage later attempts or to avoid repeating actions that are likely to be identified by network security systems), and more recent CTI may show new and / or changed IOCs and / or other information related to modified attack methods. As another example, older CTI data may indicate a cyber threat, but that data might have been received from a single provider. Based on this, it may not be reliable enough to block network traffic related to the cyber threat. More recent CTI data, even if it indicates the same cyber threat, may have been received from multiple providers. As more CTI data is received from more providers, it may become reliable enough to block network traffic related to the cyber threat.

[0030] As another example, the disposition can be determined for each endpoint. In doing so, based on the association with a specific endpoint shown by one or more providers as a cyber threat, it is possible to block, permit, etc. malicious network traffic. This enables a more accurate response to emerging cyber threats (e.g., blocking an increase in malicious network traffic, blocking malicious network traffic more quickly, and / or permitting an increase in non-malicious network traffic while a new cyber threat emerges), and the detection of cyber threats can be improved.

[0031] An endpoint can be one or more network-connected devices such as a laptop, mobile device, server, Internet of Things (IoT) device, etc. An endpoint is identified by a domain name, universal resource identifier (URI), Internet protocol (IP) address, classless inter-domain routing (CIDR) address, multi-dimensional (MD) indicator (e.g., an indicator including a range of addresses and ports), a range of addresses (e.g., a range of IP addresses), or other information identifying an endpoint within CTI data. In this way, cyber threats can be detected and / or dispositions can be determined for specific domain names, URLs, IP addresses, CIDR addresses, MD indicators, ranges of addresses, etc.

[0032] Providers can indicate that an endpoint is a cyber threat by including at least one Indicator of Compromise (IOC) on the endpoint. An IOC can be evidence that the endpoint is a cyber threat (e.g., evidence that the endpoint's network security has been compromised). Because the form and method in which an IOC provides evidence that an endpoint is a cyber threat varies from provider to provider, an IOC can take various forms and be expressed in various ways from provider to provider. Some examples of IOCs include: a 5-tuple value (host / resource identifier specifying the L3 host IP address, L4 port, and / or associated L3 protocol type) or part thereof; one or more other L3 header field values ​​or part thereof; one or more other L4 header field values ​​or part thereof; one or more L2 header field values ​​or part thereof; protocol version; hostname or part thereof; domain name (e.g., fully qualified domain name (FQDN), part of a domain name (e.g., top-level domain and / or subdomain)); URI or part thereof; Universal Resource Locator (URL) or part thereof; certificate or part thereof; certificate identifier or part thereof; certificate authority identifier or part thereof; size of the packet or part of the packet; part of the packet's data payload. This includes associated indicators, HTTP request method types (e.g., GET, PUT, PUSH, CONNECT, etc.), etc. An IOC consists of metadata associated with one or more packets. For example, an IOC may contain one or more types of information about one or more parameters or characteristics observable based on the behavior (or results of behavior) of a packet and / or a collection of packets. For example, an IOC may consist of the time associated with the packet's communication, whether the packet has a specified temporal or other relationship with one or more other packets, direction (e.g., incoming or outgoing), etc. In some variations, an IOC may take the form of a Network Threat Indicator (NTI), which may provide additional network information than other, more general forms of IOCs.

[0033] The embodiments described herein may also relate to methods and techniques for addressing challenges related to CTI-based and / or IOC cyber threat detection. For some endpoints, such as newly identified or unseen endpoints, the CTI data may not be sufficient to determine a specific disposition for those endpoints. For example, CTI data may not yet exist for an endpoint, or the existing CTI data may be unreliable and not meet the confidence threshold required to determine a disposition. For such endpoints, a default disposition may be applied depending on the risk tolerance of the entity managing the network. For example, a default block disposition may be applied to network traffic to and from such endpoints for an entity with a relatively low risk tolerance, while a default allow disposition may be applied to network traffic to and from such endpoints for an entity with a relatively high risk tolerance. However, a default block disposition may block legitimate network traffic and negatively impact the entity's network activities. On the other hand, a default allow disposition may permit illegitimate (e.g., malicious) network traffic to the entity's network and expose the network to cyber threats. To improve the use of default penalties, alternative penalties different from the default may be determined, taking into account the impact of blocking potentially legitimate network traffic, and may be applied to endpoints with little (or no) CTI data or endpoints with unreliable CTI data. This may improve the risk profile of the entity. For example, blocking network traffic that does not significantly impact an entity's network activity may improve the entity's risk profile if such traffic were to become a threat to the network.

[0034] Additional examples and details of the above introduction, along with other examples and details, are described below in relation to Figure 1-16. Figure 1 is a block diagram of an example computer environment 100 in which cyber threats may be detected. Figures 2, 3A-3D, 4, and 5A-5B show block diagrams that provide further examples of how the exemplary computer environment 100 may be implemented to detect cyber threats, for example, by configuring machine learning models and / or rules / policies in a particular exemplary configuration. Figures 6A-6B and 7-9 show examples of methods that may be implemented by the exemplary computer environment 100 in relation to cyber threat detection. Figures 10A-10F and 10A-10B show example flows that illustrate how the exemplary computer environment 100 operates in relation to cyber threat detection. Figure 12 shows a block diagram of an example computer environment 1200 that may be configured to address cyber threats based on impact status. Figure 13 shows examples of methods that may be implemented by the exemplary computer environment 1200 in relation to addressing cyber threats based on impact status. Figure 14 shows an example flow illustrating how the exemplary computer environment 1200 operates in relation to addressing cyber threats based on impact status. Figure 15 shows an example of another method that the exemplary computer environment 1200 may perform in relation to addressing cyber threats based on impact status. Figure 16 shows an example of computer equipment that may be used when implementing the various components of the exemplary computer environment 100.

[0035] Figure 1 shows a block diagram of an exemplary computer environment 100, which may be configured to detect cyber threats based on threat context and / or changes in threats. The computer environment 100 may, in particular, be an enterprise computer environment that provides cyber threat detection services to enterprise customers. Briefly outlined, the computer environment 100 is shown as including a data repository 110 and various agents 105, 130, 140, 150 that detect cyber threats based on threat context and / or changes in threats. The data repository 110 may be implemented on one or more storage devices. The various agents 105, 130, 140, 150 may be implemented on one or more computer devices. The data repository 110 and the various agents 105, 130, 140, 150 may be located locally or remotely from one another. Furthermore, some or all of the data repository 110 and the various agents 105, 130, 140, and 150 may be implemented using cloud computing services (for example, the data repository 110 may be implemented using cloud computing services, and the various agents 105, 130, 140, and 150 may be implemented on one or more computer devices that communicate with the cloud computing services over one or more networks). The computer environment 100 may include additional components not shown in Figure 1, such as additional data repositories, additional computer devices, and / or additional networks.

[0036] In relation to cyber threat detection, the exemplary computer environment 100 is shown as including data repositories 110 that include specific types of data repositories (e.g., provider feed repository 112, threat analysis data repository 120) and specific data repositories that store specific types of data (e.g., raw data & metadata archive 114, endpoint data archive 115, event data repository 121, telemetry repository 122, detection data repository 123, exclusion data repository 124, address data repository 126, disposal feed criteria repository 127, and CTI mapping repository 128). The shown data repository 110 is provided as an example of a repository that may be used when detecting cyber threats. Several variations may include additional or fewer data repositories that differ from those shown in the example computer environment 100 in Figure 1.

[0037] Furthermore, in relation to cyber threat detection, the exemplary computer environment 100 is shown as having various agents 105, 130, 140, and 150 that send and receive data to and from the data repository 110, receiving specific input data and providing specific output data, and performing various processes. In fact, one or more data collection agents 105 are shown to receive one or more types of data from various providers 101-1 to 101-X, perform one or more processes for receiving, storing, and / or processing the received data, and output endpoint data 108. One or more threat analysis agents 130 are shown to receive endpoint data 108, perform one or more processes for ingesting and analyzing the endpoint data and other data, and output threat differential data 135. One or more threat monitoring agents 140 are shown to perform one or more processes for threat monitoring and output threat differential data 145. One or more disposal feed agents 150 are depicted as receiving threat differential data (e.g., threat differential data 135 from one or more threat analysis agents 130 and threat differential data 145 from one or more threat monitoring agents 140), executing one or more processes to determine a disposal feed, and outputting a feed notification 152 and a disposal feed 154 (e.g., disposal feed 1 to disposal feed Z), any of which may be received by a computer device 170 via the network 160. The agents 105, 130, 140, 150, processes, input data, and output data shown are provided as examples that may be used when detecting cyber threats. Several variations may include different, added or fewer agents, different, added or fewer processes, different, added or fewer types of input data, and different, added or fewer types of output data than those shown in the example computer environment of Figure 1.

[0038] Furthermore, in relation to cyber threat detection, the exemplary computer environment 100 is shown with various agents 105, 130, 140, and 150, each containing machine learning models and / or rules / policies. For example, as shown in Figure 1, one or more data collection agents 105 may contain one or more machine learning models 105-1 and rules / policies 105-2. One or more threat analysis agents 130 may contain machine learning model 130-1 and rules / policies 130-2. One or more threat monitoring agents 140 may contain machine learning model 140-1 and rules / policies 140-2. One or more disposal feed agents 150 may contain machine learning model 150-1 and rules / policies 150-2. The machine learning models and rules / policies shown are provided as examples of how they may be used by various agents 105, 130, 140, and 150 in relation to cyber threat detection. Some variations may contain additional or fewer machine learning models and / or rules / policies that differ from those shown in Figure 1. Details of the machine learning model and rules / policies will be described later, particularly in relation to Figures 2, 3A-3D, 4, and 5A-5B.

[0039] A more detailed description of an exemplary computer environment 100 and how it detects cyber threats can begin with Provider 101-1 to Provider 101-X and one or more data collection agents 105. Providers can be any data source, including external entities (e.g., providers outside the entity providing the cyber detection service), internal entities (e.g., providers inside the entity providing the cyber detection service), non-governmental organizations, governmental organizations, open-source organizations, subscription-based entities, etc. Providers 101-1 to Provider 101-X can publish or transmit various types of data via Provider Feeds 101-1 to Provider Feeds 101-X. The exemplary computer environment 100 shows examples of two types of data that may be transmitted via Provider Feeds: CTI data (e.g., shown as CTI data 103) and Exclusion data (e.g., shown as Ex data 106). Each Provider Feed can deliver data in real time, based on a publication schedule, and / or based on requests from an Application Programming Interface (API).

[0040] One or more providers 101-1 to 101-X can send their data in any appropriate format via their provider feed. Often, providers use their own data formats. Thus, data received from different providers may have different structures, compositions, representations, and / or semantics. For example, as shown in Figure 1, provider 101-1 provides CTI data 103 via provider feed 1. CTI data 103 may be formatted according to a proprietary CTI data scheme specific to provider 101-1. Therefore, the structure, composition, representation, and / or semantics of CTI data 103 may differ from CTI data provided via different feeds and / or by different providers. Some, but not all, examples of CTI data formats include JavaScript Object Notation (JSON), comma-separated values ​​(CSV), Extensible Markup Language (XML), Structured Threat Information Expression (STIX), and text files. Furthermore, even if CTI data from different feeds and / or different providers is formatted in a similar way (for example, even if CTI data from two different providers is both formatted using JSON), the way the CTI data is represented may differ (for example, two different CTI providers may use different labeling schemes). In addition, CTI data can be structured in a wide variety of ways, from simpler to more complex structures. For example, some CTI data may contain a single element of CTI (for example, an IOC for a single endpoint). Others may contain multiple elements organized as a list (for example, each row containing one element). There may also be CTI data that contains more complexly structured elements. CTI data can also vary widely in terms of data volume.For example, some CTI data could be larger than 1 kilobyte, while other CTI data could be larger than 1 terabyte.

[0041] CTI data can be received in various ways. For example, some providers allow the download of an entire file of CTI data in a single transaction. In this method, some CTI data received by one or more data collectors 105 in a single transaction (e.g., one download and / or one API call) may include the entire file. On the other hand, some providers require multiple downloads or multiple API calls to download the entire file of CTI data. In this method, some CTI data received by one or more data collectors 105 in a single transaction may include a subset of the entire file (e.g., one page of the entire file).

[0042] CTI data 103 provides a simple example of CTI data. As shown in Figure 1, CTI data 103 includes, among other data, the identifier of provider 101-1 ("Provider_ID") and the identifier of the endpoint ("www.xyz123.c"). Based on the endpoint identifier, CTI data 103 may include IOCs for that endpoint. In this way, CTI data 103 may indicate that the endpoint is a potential cyber threat. As shown by the simple example provided by CTI data 103, CTI data can contain much more information than just the identifier of the CTI provider and the identifier of the endpoint. In fact, CTI data can contain information on any number of endpoints, metadata related to the endpoints and / or providers, along with other data (e.g., geospatial data) that is not intended to indicate IOCs or cyber threats. More generally, CTI data contains evidence and other information about cyber threats brought about by one or more endpoints. CTI data may indicate the nature of the threats brought about by one or more endpoints. For example, CTI data may indicate that a threat posed by a particular endpoint includes viruses or other malware, bots, phishing attacks, exfiltrations, hacking attempts, and / or other types of activity that are already known or suspected. CTI data may include information about threat actors associated with or suspected of being associated with the endpoint. CTI data may explicitly identify one or more IOCs, and / or the IOCs may be derived from the information contained in the CTI data. CTI data may include threat levels, which are values ​​that indicate the likelihood that the endpoint is a cyber threat.

[0043] CTI data may, and often may, provide an incomplete assessment of cyber threats posed by endpoints. For example, CTI data may include one type of IOC for one endpoint but lack other types of IOCs and / or threat context associated with the endpoint. Over time, providers 101-1 to 101-X may provide new CTI data that enables a more complete assessment of cyber threats posed by endpoints. The new CTI data may include additional types of IOCs for the endpoint and / or changes to IOCs provided in previous CTI data for the endpoint. Changes in IOCs may result in IOCs not included in earlier CTI data being included in later CTI data, and / or IOCs included in earlier CTI data not being included in later CTI data. As will become apparent from the examples described throughout this disclosure, changes provided by CTI data, including changes to IOCs, can provide a basis for analyzing cyber threats posed by endpoints and determining sanctions against them, indicating how devices should filter network traffic associated with the endpoint.

[0044] Exclusion data identifies one or more endpoints as excluded, or can indicate them as excluded even if they are not specifically identified. An exclusion may indicate one or more conditions that prevent the publication of an endpoint or any disposition related to that endpoint from entering the disposition feed. In this way, as long as one or more conditions of the exclusion are met, a device receiving the disposition feed (e.g., computer device 170) may not receive dispositions indicating the blocking of network traffic related to the endpoint. The one or more conditions depend on the type of exclusion. Types of exclusions include global exclusions and time-based exclusions. Because various characteristics of an endpoint, such as its IP address, can change over time, endpoint exclusion data can also change over time, and therefore providers need to update the exclusion data.

[0045] One or more providers 101-1 to 101-X can send their exclusion data in any appropriate format via their own feeds. Often, exclusion providers use their own formats for their own exclusion data. Thus, exclusion data received from different exclusion providers may have different structures, configurations, representations, and / or semantics. For example, as shown in Figure 1, exclusion provider 101-X provides exclusion data 106 via provider feed X. Exclusion data 106 may be formatted according to a unique exclusion data scheme specific to exclusion provider 101-X. Consequently, the structure, configuration, representation, and / or semantics of exclusion data 106 may differ from exclusion data provided through different feeds and / or by different providers. Some, but not all, examples of exclusion data formats include JavaScriptObjectNotation (JSON), comma-separated values ​​(CSV), ExtensibleMarkupLanguage (XML), StructuredThreatInformationExpression (STIX), and text files. Furthermore, even if exclusion data from different feeds and / or different providers is formatted in a similar manner (for example, even if exclusion data from two different providers is formatted using text files), the way the exclusion data is represented may differ (for example, two different exclusion providers may use different labeling schemes). In addition, exclusion data can be structured in a wide variety of ways, including simpler and more complex structures. For example, there may be exclusion data that includes a list of endpoints for exclusion. On the other hand, there may also be exclusion data that includes more complexly structured elements, such as elements that specify what types of exclusions should apply to an endpoint (e.g., global exclusions, time-based exclusions, etc.) and include any other data related to that type.

[0046] In addition to the CTI data and exclusion data shown by the exemplary computer environment 100, additional or different types of data may be transmitted via provider feeds 101-1 to 101-X. These may include, for example, unstructured data (e.g., unstructured intelligence data), raw network traffic, data providing reports of network activity from corporate customers providing cyber threat detection (e.g., reports showing that a user clicked on malware and was consequently redirected to a malicious entity in a specific geographical area), and various types of non-intelligence data. Furthermore, providers may send multiple types of data through feeds or in separate feeds. For example, one or more providers may send feeds containing both CTI data and exclusion data.

[0047] Various agents 105, 130, 140, and 150 may be configured to perform one or more actions based on additional or different types of data. For example, threat analysis agent 130 may be configured to analyze additional or different types of data and, based on the analysis, determine whether the additional or different types of data indicate that malicious traffic is occurring or has occurred. Examples of types of malicious traffic to be determined include whether one or more ports that should not be accessed have been accessed, whether the source of network traffic is associated with a prohibited geographical location or indicates malicious activity, or whether there is a display of malware. Analysis of unstructured data may include filtering unstructured data or raw network traffic, or extracting specific types of network traffic from reports received from customers. For example, a customer may provide a report of network activity that is used to determine the potential impact of blocking legitimate network traffic between their respective network and one or more endpoints. These network activity reports may show, for example, the volume and / or frequency of network traffic between each customer's network and one or more endpoints, as well as specific addresses (e.g., IP addresses), machines and / or machine types, computer resources (e.g., applications, protocols, services, data stores), users, user groups, departments, offices, geographical locations, etc., associated with each customer's network traffic. Such network activity reports can be used to determine endpoint disposal (e.g., alternatives to default disposal) based on the expected impact on legitimate network traffic between the endpoints and the customer's network.

[0048] By determining instructions for malicious traffic based on additional or different types of data, various agents 105, 130, 140, and 150 may respond to those instructions by creating and updating those policies and rules, which are distributed to enforcement agents that enforce policies and rules to filter packets to and / or from endpoints associated with malicious traffic, for example by allowing, blocking, or monitoring the packets. Notably, since additional or different types of data include, in particular, non-intelligence data and / or reports of network activity received from customers, various agents 105, 130, 140, and 150 may respond to indications of malicious traffic that are infrequent, unpredictable, and / or lack CTI data or IOCs. An example of infrequent, unpredictable, and / or lacking CTI data or IOCs is an attack against a single customer that is short in duration (e.g., lasting only a few seconds) and / or slow in periodicity (e.g., recurring every few weeks). Another example of malicious traffic that is infrequent, unpredictable, and / or lacks CTI data or IOCs is traffic to or from endpoints that the network has never communicated with before, or endpoints previously unknown to the network.

[0049] There are many different analyses that can be performed in relation to determining and responding to a small-scale, targeted attack. For example, raw network traffic going to and / or going to a single customer may be collected and analyzed to identify a set of endpoints that are sending data to and / or receiving data from that customer. From this set of endpoints, trusted endpoints, known legitimate endpoints, or whitelisted endpoints may be excluded. The remaining endpoints may be analyzed to determine whether there is a legitimate reason for traffic to be sent to and from the endpoint. For example, there may be a legitimate reason if the customer is known to do business with endpoints at its geographical location, and / or if the customer sends traffic to a high-risk endpoint in the course of normal business operations. Other examples of potentially legitimate reasons include if the customer historically exhibits a pattern of traffic sent to / received from an endpoint, or if other customers doing similar business are also sending / receiving traffic from and / or with similar frequency to the endpoint, or if the endpoint exceeds a risk threshold, or if the total volume of traffic sent to / received from the endpoint falls below a volume threshold. Based on various analyses, a decision may be made regarding whether to allow, block, or monitor an endpoint (e.g., an alternative to the default action) (e.g., blocking if the total volume of traffic is below a volume threshold, or blocking if there is a legitimate reason for traffic to be sent to or from the endpoint). These analyses and decisions regarding actions may be performed by one or more machine learning models and / or rules / policies specific to a particular customer. For example, one or more machine learning models configured to show the impact of blocking network traffic to and / or from a given endpoint may be trained based on historical network activity data received from the customer.The trained machine learning model may, as input, provide network traffic data related to network traffic to and / or from an endpoint, and as output, provide an indicator of the potential impact of blocking legitimate network traffic to and / or from that endpoint.

[0050] Analysis that may be performed to uncover small-scale targeted attacks may generally involve analyzing CTI, network traffic data, and / or non-intelligence data to identify connections between seemingly unrelated IOCs (or relationships between seemingly unrelated IOCs) received from different CTI providers. Given the large amounts of raw data received from multiple CTI providers, correlating such data to identify new, emerging, or potential malicious activity may be challenging. For example, CTI data from a single CTI provider may not be sufficient on its own to suggest malicious activity targeting one or more endpoints, but such malicious activity can be revealed by correlating or combining CTI data from multiple providers. As described below, CTI and non-CTI data may be pooled to generate datasets (including subsets of data) that are analyzed to produce and output additional CTI data that can be considered when determining the threat context of one or more endpoints. Therefore, a data collection agent (e.g., data collection agent 105) may include one or more data collection agents that ingest and analyze CTI data from one or more CTI providers, as well as non-CTI data (e.g., data received from or otherwise acquired from data repository 110), and output endpoint data (e.g., endpoint data 108) that includes information about potential malicious activity suggested by the connections / relationships discovered between IOCs based on the analysis of the pooled data. Such endpoint data may be provided to a threat analysis agent (e.g., threat analysis agent 130). Thus, potential malicious activity suggested by the connections / relationships discovered in the endpoint data can be one of many factors that the threat analysis agent considers when determining the threat context of the endpoint.The data collection agent may exist as a separate, independent system that takes CTI and non-CTI data provided by another system (e.g., computer environment 100) as input and provides endpoint data as output, which includes information about potential malicious activity suggested by the discovered connections / relationships.

[0051] Generally, discovering hidden connections / relationships between IOCs based on pooled data may involve generating datasets containing overlapping contextual information and analyzing those datasets to identify commonalities between different contexts associated with each IOC. This may also involve determining the relative importance of such overlaps. For example, some attributes may be relatively more important than others when determining whether the commonality of attributes suggests a connection or relationship between IOCs. As an example, a certain category of contextual information may, in the absence of other similarities, be at least useful in discovering connections / relationships between IOCs. As will be further discussed below, overlapping (or crossover) between contexts may be analyzed to determine the likelihood (e.g., probability) of connections / relationships between IOCs in order to assess the degree of potential threats associated with the originally received CTI data.

[0052] Contexts may overlap if they share at least some of the same attributes and / or characteristics. Furthermore, IOCs may be described as having direct connections / relationships with other IOCs. IOCs may also have indirect connections or relationships with other IOCs. For example, an indirect connection / relationship between one IOC (e.g., IOC A) and another IOC (e.g., IOC C) may be based on the fact that both of those IOCs have direct connections / relationships with a common IOC (e.g., IOC B). For ease of reference, connections / relationships may be indicated by double arrow notation in this specification (e.g., IOC A ←→ IOC B ←→ IOC C). An IOC (e.g., IOC D) may also have an indirect relationship with another IOC (e.g., IOC G) through a chain of direct or indirect relationships with multiple IOCs (e.g., IOC E and IOC F) (e.g., IOC D ←→ IOC E ←→ IOC F ←→ IOC G). For ease of reference, connections / relationships between IOCs can be characterized based on their degree of separation. The degree of separation between IOCs may also be based on the amount of attributes / characteristics in the links between IOCs. For example, two IOCs may have a direct connection / relationship that shares one common attribute (e.g., IOC A [category a] ←→ [category a] IOC B), and this may be described as having a degree of separation of 1. IOCs with a direct relationship may share multiple common attributes, and the reliability of the relationship between IOCs may be based on the amount of attributes they share (e.g., reliability based on a relatively large number of common attributes is relatively high, and reliability based on a relatively small number of common attributes is relatively low).As another example, two IOCs with an indirect connection / relationship can be described as having x degrees of separation, depending on the amount of common attributes that connect them through a chain of common attributes shared among multiple IOCs (for example, if there is a 2-degree separation between IOC A and IOC C, such as IOC A[category a] ←→[category a] IOC B[signature b] ←→[signature b] IOC C, or if there is a 3-degree separation between IOC A and IOC D, such as IOC A[category a] ←→[category a] IOC B[signature b] ←→[signature b] IOC C[timeframe c] ←→[timeframe c] IOC D). Direct connections / relationships are sometimes called primary connections / relationships, indirect connections / relationships with a 2-degree separation are sometimes called secondary connections / relationships, indirect connections / relationships with a 3-degree separation are sometimes called tertiary connections / relationships, and so on (for example, x degrees of connection / relationship).

[0053] The reliability of connections / relationships between IOCs can be determined. Reliability indicates (e.g., quantifiable) the degree to which a connection / relationship is not accidental or coincidental. This reliability can be used to determine whether to obtain and analyze contextual information related to IOCs that are one degree further away from the current IOC being evaluated. In addition, or alternatively, additional contextual information about additional IOCs can be obtained regardless of the reliability related to connections / relationships between IOCs. For example, additional contextual information can be obtained for IOCs that are x degrees (e.g., 3 degrees) away from the IOC currently being evaluated. The degree used when considering additional contextual information may be a configurable parameter, which may be set specifically for a single entity (e.g., per entity) or globally for multiple (or all) entities. Obtaining additional contextual information that is x degrees away from the IOC currently being evaluated involves selecting one or more attributes / characteristics (e.g., category, signature, descriptor, metric, timestamp, etc.) and obtaining any contextual information that includes the selected one or more attributes / characteristics. The selected characteristics / attributes can be identified manually (for example, by network security personnel) and / or automatically (for example, by data collection agents, threat agents, etc.).

[0054] Based on the credibility of the threat context, additional contextual information may be obtained. This additional contextual information (e.g., additional characteristics / attributes) may be obtained for both low-credibility threat contexts (e.g., scenarios where the credibility of available CTI data can be described as "low," or scenarios where CTI data is unavailable) and high-credibility threat contexts (e.g., scenarios where the credibility of available CTI data can be described as "high"). Various agents described herein (e.g., threat analysis agents) can obtain additional contextual information (e.g., as part of performing threat analysis). The obtained additional contextual information may indicate that updating the credibility of the threat context and / or CTI data is justified (e.g., upgrading credibility from "low" to "high," or downgrading it from "high" to "low"), or it may affirm the determined credibility. For example, for unreliable threat contexts, the additional contextual information obtained may indicate patterns associated with the threat context, the threshold prevalence of the threat context, and / or changes in the threat context over time (e.g., an increase in instances of the threat context), all of which may suggest an upgrade (e.g., an increase) in the reliability of the threat context and / or the reliability of the CTI data (e.g., an upgrade from unreliable to reliable). As another example, for unreliable CTI data, the additional contextual information obtained may contradict the unreliable CTI data and / or indicate that the threat context is not widespread (e.g., does not meet the threshold prevalence), thereby affirming the determined reliability of the threat context and / or the CTI data. On the other hand, for reliable threat contexts and / or CTI data, additional contextual information can be obtained to affirm the determined high reliability, for example, based on the impact of filtering (e.g., blocking) network traffic associated with the threat context.For example, even if high reliability is determined, if the potential impact of blocking network traffic is significant (e.g., meeting the impact thresholds described herein), additional contextual information may be obtained to affirm that high reliability. In other words, in scenarios where a false affirmation could have a significant impact on the entity's network operations, additional contextual information may be obtained even for reliable threat context and / or reliable CTI data. Additional contextual information may be obtained for an additional degree of isolation, for example, based on the decrease in return for changes in determined reliability. For example, additional contextual information may be obtained for a further degree of isolation until the change in determined reliability no longer meets the change threshold (e.g., meets or exceeds the change threshold). As another example, additional contextual information may be obtained for a further degree of isolation until a sufficient balance is achieved between the risk posed by the potential threat and the reliability of that threat (e.g., until the difference between the risk score and the threat score is minimized). For example, in a given scenario, positive contextual information (e.g., agreement on the threat context) may increase at relatively close degrees of separation, while conflicting contextual information (e.g., mismatch on the threat context) may increase at more distant degrees of separation. In other words, mismatch on the threat context may decrease as additional contextual information is acquired, and this may continue until the mismatch on the threat context begins to increase as additional contextual information is acquired. In this regard, the relationship between mismatch (or agreement) on the threat context and the degree of separation may be parabolic, where the degree of mismatch (e.g., the quantified mismatch value) decreases as the degree of separation increases, and this continues until there is a minimum point (lower limit) where the degree of mismatch begins to increase as the degree of separation increases (or alternatively, where the degree of agreement (e.g., the quantified agreement value) increases as the degree of separation further increases, and this continues until there is a maximum point (peak) where the degree of agreement begins to decrease as the degree of separation increases).Therefore, determining the extent of additional contextual information obtained may involve minimizing inconsistencies between the obtained contextual information (e.g., minimizing quantified inconsistencies) or maximizing inconsistencies between the obtained contextual information (e.g., maximizing quantified inconsistencies). Determining the extent of additional or alternatively obtained contextual information may be based on comparisons between combined confidence levels determined for contextual information indicating a high probability of a potential threat (e.g., "bad" contextual information), contextual information indicating a low probability of a potential threat (e.g., "good" contextual information), or contextual information indicating neither high nor low probability (e.g., "neutral" contextual information). For example, combined confidence levels may be determined for all confidence levels determined for "bad" contextual information, and combined confidence levels may be determined for all confidence levels determined for "good" contextual information. Additional contextual information may be obtained until these combined confidence levels are compared with each other and a sufficient difference is observed between the combined confidence levels (e.g., until the difference between the combined confidence levels exceeds a difference threshold). For example, additional contextual information may be acquired until the confidence determined for "good" contextual information significantly outweighs the confidence determined for "bad" contextual information (or vice versa). The reasons why contextual information is determined to be "good" or "bad" may be less important than the fact that a sufficient amount of "good" or "bad" contextual information has been acquired in this way.

[0055] Analyzing additional contextual information to discover connections / relationships between seemingly unrelated / unrelated IOCs can be helpful in assessing threats associated with the originally received CTI data (e.g., the originally received IOCs). By leveraging discovered connections / relationships between IOCs, potential attacks associated with these contexts that might otherwise have gone unnoticed can be discovered. Context can be assessed as described herein to discover hidden attacks, previously unnoticed attacks, and / or potential future attacks whenever a data collection agent (e.g., one or more of the data collection agents 105) ingests new CTI providing IOCs with corresponding contexts. Contextual information received with IOCs can be assessed as described herein to determine whether the threat context has changed. Even if the threat context has not changed, the re-presentation of IOCs in the received CTI data may suggest that the threat is still active. As a result, the status of that threat context may be updated, and contextual information may be analyzed as described herein to identify connections / relationships with other IOCs in order to potentially update the threat context of the IOC based on such analysis results, and to determine the disposition of the endpoint.

[0056] To illustrate these principles for discovering connections / relationships between contexts, the following scenarios are provided. These scenarios are illustrative and not limiting.

[0057] In one exemplary scenario, contextual information associated with two different endpoints (e.g., different domains) may only overlap because their respective certificates are signed by the same Certificate Authority (CA), otherwise indicating that the endpoints have different characteristics and / or attributes. For example, the contextual information may indicate that different endpoints determine to have different IP addresses (or different rotation groups of IP addresses that change randomly for each request), that different endpoints are hosted by different host providers, that different endpoints run different applications and / or services (e.g., indicate different application / service fingerprints), that different endpoints provide different services (e.g., HTTP, HTTPS, Representation State Transfer - REST, etc.), that different endpoints employ different security mechanisms (e.g., Transport Layer Security (TLS) protocol, Secure Sockets Layer (SSL) protocol, etc.), or otherwise that different endpoints provide some data and / or functionality. As a result, the reliability of the connections / relationships between the contextual information of endpoints may be relatively low (e.g., suggesting accidental or coincidental connections / relationships). In this example scenario, we might receive additional contextual information (e.g., new CTI data) indicating that a common CA was previously in a vulnerable state (e.g., a malicious actor impersonates the CA using stolen / vulnerable CA keys to obtain keys and signed certificates issued to various endpoints). This additional contextual information can be pooled with existing contextual information to reveal connections / relationships between IOCs that had some kind of relationship with the vulnerable CA during the period it was in a vulnerable state (e.g., by increasing the credibility to suggest that the connections / relationships were accidental or not). In this way, the threat context of each IOC is updated, and the appropriate action is determined.

[0058] In another scenario, CTI data might be received indicating that a malicious actor may use two different versions or two different types of malware to engage in malicious activity. A separate threat context may be identified for each malware version / type, and these different threat contexts may not be correlated with one another. For example, CTI data for one version / type of malware may be received at a certain time from one CTI provider, while CTI data for different versions / types of malware may be received at different times from different CTI providers. Thus, threat contexts and their corresponding contextual information may remain isolated from each other until additional contextual information (e.g., CTI data and / or non-CTI data) that connects the contextual information of those threat contexts is received.

[0059] In a further example, CTI data might be received containing an IOC for a specific endpoint, and contextual information might indicate that the IP address associated with that endpoint was also associated with multiple different domains (e.g., 100 different domains) over a previous period (e.g., the past two weeks). Therefore, the data collection agent can obtain contextual information related to those different domains (e.g., those separated by up to x degrees from the contextual information of the IOC in the received CTI data) and analyze the contextual information of additional IOCs that have primary, secondary, tertiary, etc. relationships with the current IOC under evaluation. Based on these connections / relationships, the threat context of that IOC is determined (e.g., updated).

[0060] In another scenario, CTI data may be received for two different endpoints, identified by different IP addresses, domain names, and / or URLs, with no apparent overlap in threat context. Subsequently, additional CTI data may be received with a one-to-one direct relationship or a greater-than-one indirect relationship, such as when the domain of one endpoint and the IP address of another endpoint are controlled by a single threat actor group employing diverse attack methods. By analyzing the combination of CTI data, it may be possible to change the disposition of relevant IOCs and detect threat activities that were missed due to insufficient or incomplete threat context.

[0061] In another scenario, CTI data may be received for endpoints with conflicting threat contexts, such as a benign context indicating an undesirable but legitimate service and a malicious context indicating malware command and control (C2) activity. Analyzing such conflicting contexts can lead to inappropriate IOC decisions due to insufficient confidence in the threat severity. Subsequently, additional CTI data may be received that supports the belief that one context, such as C2 activity, is more reliable than another context, such as a benign service. Receiving a series of additional CTI data may further increase or decrease the reliability of the relevant contexts. By analyzing combinations of CTI data, the decision on the relevant IOC can be modified based on the importance, risk level, or potential impact of one context on another.

[0062] In another scenario, CTI data may be received for endpoints with a threat context that includes risk scores and / or confidence scores for a specific threat. The criteria for selecting a penalty may depend on the risk score and confidence score, and if they fall below a threshold, unintended penalties may be imposed. Subsequently, CTI data may be received from one or more CTI sources for endpoints with similar low-risk threat contexts. A series of additional CTI data may be received in real time. By continuously analyzing the additional threat contexts from different providers, even if each of them is insufficient to exceed the threshold individually, the combined risk scores and / or confidence scores may collectively exceed the threshold, thus achieving the intended penalty.

[0063] In another scenario, CTI data, such as a high-reliability and high-risk threat context, and non-CTI data, such as an impact context, may be received for a single endpoint. The threat context may strongly suggest a specific disposition, such as blocking all traffic to the relevant endpoint. However, the impact context may strongly suggest that the endpoint is a frequently used service critical to business needs. By analyzing the combination of CTI and non-CTI data, it may be possible to modify the disposition of relevant IOCs, such as limiting the available dispositions to a subset of dispositions, such as monitoring all traffic to the relevant endpoint, thereby reducing the risk of disrupting business functions while making threat activity detectable.

[0064] In another scenario, CTI data, such as context with temporal context, may be received for a single endpoint. Temporal context can indicate changes in the applicability, reliability, risk, and / or validity of a threat context based on the time it is used or applied. Additional CTI data may be received from other sources that provide different temporal contexts for the same threat context at the same endpoint. Additional non-CTI data may be received for the same endpoint as impact context and associated temporal context. Analyzing a combination of CTI and non-CTI data allows for modifying the handling of relevant IOCs over time, based on an assessment of the business impact of specific measures at specific time periods. For example, monitoring and allowing daytime traffic with high threat risk but high business impact risk, while blocking nighttime traffic with high threat risk but low business impact risk.

[0065] Additional examples and scenarios will be understood by referring to the disclosures herein.

[0066] While the above explanations regarding additional or different types of data provide the basis for a wide range of ways in which malicious traffic (e.g., small-scale targeted attacks) can be identified and responded to, the remaining explanations regarding Figures 1-15 will, for simplicity, be described in terms of the CTI data and excluded data shown in Figure 1.

[0067] As shown in Figure 1, one or more data collection agents 105 may receive data of one or more data types from one or more providers 101-1 to 101-X via provider feeds 1 to X. In some variations, there may be multiple data collection agents 105 operating in the computer environment 100. For example, in such a variation, each provider feed and / or each of providers 101-1 to 101-X may have its own data collection agent (for example, there may be X data collection agents for providers 101-1 to 101-X). In another example, in such a variation, a subset of provider feeds and / or a subset of providers 101-1 to 101-X may have its own data collection agents (for example, provider feeds and / or providers may be grouped into subsets by a classification scheme based on trustworthiness of the providers and / or by a classification scheme based on similarity measurements between different proprietary formats and proprietary notations of the providers). One or more of the multiple data collection agents 105 can operate in parallel with others, listening to their specific provider feeds and receiving their specific CTI data, etc. Furthermore, one or more of the multiple data collection agents 105 may have their own one or more machine learning models 105-1 and / or rules / policies 105-2. In this way, one or more machine learning models 105-1 and / or rules / policies 105-2 may be configured to process data from a specific provider.

[0068] After receiving data from a specific provider (for example, CTI data 103 from provider 101-1 via provider feed 1), one or more data collection agents 105 can perform one or more processes to receive, store, and / or process the received data. For simplicity, these one or more processes will be described in terms of receiving CTI data 103 and exclusion data 106.

[0069] Based on the receipt of data from a provider (for example, CTI data 103 from provider 101-1 via provider feed 1, or excluded data 106 from provider 101-X via provider feed X), one or more data collection agents 105 may store the received data in a raw data & metadata archive 114 as part of one or more processes for receiving, storing, and / or processing the received data. By storing the received data in this way, the raw data & metadata archive 114 will have an unprocessed copy of the received data as it was received by one or more data collection agents 105 (for example, an unprocessed copy of the CTI data 103 and an unprocessed copy of the excluded data 106). Furthermore, by storing the received data in this way, the unprocessed copies can be reprocessed as needed or made available for future use (for example, as training data for one or more machine learning models such as machine learning models 105-1).

[0070] One or more data collection agents 105 can classify the received data according to its data type as part of one or more processes for receiving, storing, and / or processing the received data. For example, one or more data collection agents 105 can classify CTI data 103 as a CTI data type and excluded data 106 as an excluded data type. This classification can be done based on specific data contained in the received data. For example, CTI data 103 is shown as containing the identifier of provider 101-1 ("Provider_ID") and the identifier of an endpoint ("www.xyz123.c"). One or more data collection agents 105 can classify CTI data 103 as a CTI data type based on the fact that this information is contained in a specific syntax. As another example, excluded data 106 is shown as containing a list of exclusions that identify at least two endpoints as exclusions (for example, the endpoint "www.abc987.c" is one exclusion and the endpoint "10.20.81.0 / 24" is another exclusion). One or more data collection agents 105 can classify excluded data 106 as excluded data type based on the inclusion of an exclusion list. In some arrangements, this classification may be performed by one or more machine learning models and / or based on the rules / policies of one or more data collection agents 105. By classifying the received data, one or more data collection agents 105 can perform processing specific to the type of received data (for example, the processing of CTI data 103 may differ from the processing of excluded data 106). Further details on how one or more data collection agents 105 may perform processing specific to the type of received data will be discussed later in relation to the processing of CTI data 103 and excluded data 106.

[0071] After classifying the received data (e.g., CTI data 103) as a CTI data type, one or more data collection agents 105 may, as part of one or more processes for receiving, storing, and / or processing the CTI data 103, determine metadata associated with the CTI data 103 and / or associated with provider 101-1, and store that metadata in the raw data & metadata archive 114. The metadata associated with the CTI data 103 and / or associated with provider 101-1 may, in this specification, be interchangeably referred to as CTI-based metadata. The CTI-based metadata or any part thereof may be contained in the CTI data 103, contained in a CTIP feed that provides CTI-based metadata separately from the CTI data 103 (e.g., the CTI-based metadata may be received via a CTIP feed different from provider feed 1), determined from any data received from provider 101-1, and / or determined based on how the data collection agent received the CTI data 103. CTI-based metadata may include, for example, an indication of whether the reception of CTI data 103 was successful or unsuccessful, an indication of the time taken to receive CTI data 103, an indication of the amount of data contained in CTI data 103, an indication of the type of transaction performed to receive CTI data 103 (e.g., via file download, via API call, and / or via a software development kit (SDK)), an identifier that uniquely identifies the transaction performed to receive CTI data 103, an indication of how often provider 101-1 sends that CTI data, and / or how often provider 101-1 is requested to send that CTI data. These examples of CTI-based metadata are only a few examples of the types of metadata that may be determined.

[0072] Table I provides a more detailed example of CTI-based metadata. In particular, Table I provides example descriptions of attributes that may be included in CTI-based metadata, and for each description, examples of attribute-value pairs (e.g., attribute:value) that may be included in CTI-based metadata. The examples in Table I are only a small sample of attributes and values ​​that may be included in CTI-based metadata. [Table 1]

[0073] One or more data collection agents 105 may determine endpoint data 108 based on the CTI data 103 as part of one or more processes for receiving, storing, and / or processing the CTI data 103. Endpoint data 108 may contain endpoint data in a common format, and / or endpoint data may be represented by endpoint data 108 in a common notation. Endpoint data may be included in endpoint data 108 because CTI data 103 contains endpoint IOCs. The common format may be JSON, CSV, XML, STIX, a text file, or other appropriate format. The common notation may include attribute value pairs in an attribute naming scheme configured for one or more data collection agents 105.

[0074] The determination of endpoint data 108 may be performed by determining endpoint data from CTI data 103, mapping the data to a common format and / or common notation, and / or deriving additional data in a common format and / or common notation based on CTI data 103. This determination may be performed by using one or more machine learning models 105-1 and / or by applying rules / policies 105-2 of one or more data collection agents 105. In variations using one or more machine learning models 105-1, one or more machine learning models may be trained using a corpus of CTI data previously received from provider 101-1 (e.g., one stored in the raw data and metadata archive 114). In variations using rules / policies 105-2, the rules / policies 105-2 may be created by a human user who has knowledge of how provider 101-1 provides its CTI data (e.g., the rules / policies 105-2 may have rules for extracting data for endpoints from CTI data 103 and rules for mapping the data to a common format and / or common notation).

[0075] The determination of endpoint data 108 may be performed based on mapping information contained in the CTI mapping repository 128. The mapping information may provide rules for mapping CTI data from providers 101-1 to 101-X to a common format and / or common notation used by endpoint data 108. The CTI mapping repository 128 may be created by one or more human operators with knowledge of any of the providers 101-1 to 101-X that provide data of the CTI data type.

[0076] In some variations, endpoint data 108 may contain data for a single endpoint. In this way, endpoint data 108 may represent a single object of endpoint data for a single endpoint indicated by CTI data 103. While CTI data 103 may contain data such as IOCs for one or more endpoints, one or more data collectors 105 may determine one or more objects of endpoint data for one or more endpoints indicated by CTI data 103. Once determined, endpoint data 108 may be stored in the endpoint data archive 115 for later use (for example, as training data for one or more machine learning models 130-1 of one or more threat analysis agents 130). Furthermore, one or more threat analysis agents 130 may be notified that endpoint data 108 is available for analysis. Notifying one or more threat analysis agents 130 that endpoint data 108 is available for analysis may include sending the endpoint data 108 to at least one of the one or more threat analysis agents 130, inserting the endpoint data 108 into the feed between one or more data collection agents 105 and one or more threat analysis agents 130, and / or storing the endpoint data 108 in a location accessible to one or more threat analysis agents 130.

[0077] Figure 1 shows a generalized example of endpoint data 108, which is based on an example of CTI data 103, also shown in Figure 1. In fact, as shown in Figure 1, CTI data 103 is shown as containing data for the endpoint "www.xyz123.c". The generalized example of endpoint data 108 may be the result of one or more data collection agents 105 determining the endpoint data from CTI data 103 and mapping the data to a common format and / or common notation. In fact, with CTI data 103 shown as containing data for the endpoint "www.xyz123.c", the generalized example of endpoint data 108 contains data for the endpoint "www.xyz123.c" after mapping the data for the endpoint "www.xyz123.c" to a common format and / or common notation (e.g., "EndP:www.xyz123.c"). Endpoint data 108 may contain additional information related to the endpoint "www.xyz123.c". In fact, as shown in Figure 1, endpoint data 108 includes another attribute-value pair (e.g., "Prov:Provider_ID") with an identifier for provider 101-1. The generalized example of endpoint 108 is just one example of a common format and / or common notation that may be used by one or more data collection agents.

[0078] Table II provides a more detailed example of endpoint data 108. In particular, Table II shows example descriptions of attributes that may be included in endpoint data 108, and for each description, examples of attribute-value pairs (e.g., attribute:value) that may be included in endpoint data 108. The examples in Table II are only a few examples of attributes and values ​​that may be included in endpoint data, such as endpoint data 108 shown in Figure 1. Furthermore, the values ​​and / or attributes included in endpoint data 108 may depend on what the provider includes in the endpoint. For example, if the provider does not provide data corresponding to an attribute, some values ​​may remain empty or NULL, and / or if the provider does not provide data corresponding to an attribute, the attribute may not be included by endpoint data 108. [Table 2]

[0079]

[0080] After classifying received data (e.g., excluded data 106) as excluded data type, one or more data collection agents 105 may, as part of one or more processes for receiving, storing, and / or processing the excluded data 106, determine metadata associated with the excluded data 106 and / or associated with provider 101-X, and store that metadata in the raw data & metadata archive 114. The metadata associated with the excluded data 106 and / or associated with provider 101-X may, for interchangeable purposes herein, be referred to as excluded-based metadata. The excluded-based metadata or a portion thereof may be contained in the excluded data 106, or separately from the excluded data 106, in a provider feed providing the excluded-based metadata (e.g., the excluded-based metadata may be received via a different provider feed than provider feed X), or may be determined from any data received from provider 101-X, and / or based on how the data collection agent received the excluded data 106. The excluded-based metadata may include, for example, data similar to that described above with respect to CTI-based metadata, but the data is related to the excluded data and / or the provider that sent the excluded data. For example, exclusion-based metadata may include attribute-value pairs similar to those in Table I, except that the attribute-value pairs relate to excluded data and / or provider 101-X. Other examples of exclusion-based metadata may include an indication of whether the reception of excluded data 106 was successful or unsuccessful, an indication of the time taken to receive excluded data 106, an indication of the amount of data contained by excluded data 106, an indication of the type of transaction performed to receive excluded data 106 (e.g., via file download, via API call, and / or via a software development kit (SDK)), an identifier that uniquely identifies the transaction performed to receive excluded data 106, an indication of how often provider 101-X sends excluded data, and / or how often provider 101-X is requested to send excluded data.These examples of exclusion-based metadata are just a few examples of the types of metadata that can be determined.

[0081] One or more data collection agents 105 may determine endpoint data based on the exclusion data 106 as part of one or more processes for receiving, storing, and / or processing the exclusion data 106. Once determined, the endpoint data may indicate one or more exclusions of an endpoint. The endpoint data determined based on the exclusion data 106 may be similar to the generalized example of the endpoint data 108 described above. In this way, such endpoint data may include data for endpoints indicated as excluded in the exclusion data 106 in a common format and / or common notation. The common format and common notation may be the same as those used for endpoint data 108. For example, the endpoint data determined based on the exclusion data 106 may include attribute-value pairs similar to those described above in relation to Table II, except that the data is associated with endpoints indicated as excluded (for example, an attribute-value pair may indicate that the endpoint is excluded, such as "isExclusion": true).

[0082] Determining endpoint data based on excluded data 106 can be done by determining endpoint data from excluded data 106, mapping the data to a common format and / or common notation, and / or deriving additional data in a common format and / or common notation based on excluded data 106. This determination can be done by using one or more machine learning models 105-1 and / or by applying rules / policies 105-2 of one or more data collection agents 105. In variations using one or more machine learning models 105-1, one or more machine learning models may be trained using a corpus of excluded data 106 previously received from provider 101-X (for example, one stored in the raw data & metadata archive 114). In variations using rule / policy 105-2, rule / policy 105-2 may be created by a human user who has knowledge of how provider 101-X provides its exclusion data (for example, rule / policy 105-2 may have rules for extracting endpoint data from exclusion data 106, and rules for mapping the data to a common format and / or common notation).

[0083] In some variations, the endpoint data determined based on exclusion data 106 may include data for a single endpoint. In this manner, such an endpoint may represent a single object of endpoint data for a single endpoint indicated as excluded by exclusion data 106. Exclusion data 106 may include exclusion instructions for one or more endpoints, and one or more data collectors 105 may determine one or more objects of endpoint data for one or more endpoints indicated by exclusion data 106 as excluded. Once determined, the endpoint data determined based on exclusion data 106 may be stored in the endpoint data archive 115 for later use (e.g., as training data for machine learning models). Furthermore, the endpoint data determined based on exclusion data 106 may be stored in the exclusion data repository 124 (which may be monitored and / or accessed by one or more threat monitoring agents 140 and / or one or more disposal feed agents 150). Furthermore, one or more threat analysis agents 130 may be notified that the endpoint data determined based on exclusion data 106 is available for analysis. Notifying one or more threat analysis agents 130 that endpoint data determined based on exclusion data 106 is available for analysis may include sending the endpoint data to at least one of the one or more threat analysis agents 130, inserting the endpoint data into a feed between one or more data collection agents 105 and one or more threat analysis agents 130, and / or storing the endpoint data in a location accessible to one or more threat analysis agents 130. Furthermore, the exclusion data 106 can be used to determine, modify, construct, and / or dismantle a disposal feed via one or more disposal feed agents 150 (for example, via endpoint data determined based on exclusion data 106 and / or via data indicating exclusion data 106 stored in the threat analysis data repository 120).Details about the disposal feed and one or more disposal feed agents 150 will be described later.

[0084] As briefly mentioned above, different types of exclusions can exist. Also briefly mentioned above, two examples are global exclusions and time-based exclusions. A global exclusion can ensure that an endpoint, or any disposal related to an endpoint, is not included in any disposal feed, as long as the global exclusion remains in place (for example, as long as the exclusion data repository 124 and / or the event data repository stores the global exclusion, and / or until the global exclusion is removed from the exclusion data repository 124 and / or the event data repository 121). Thus, one or more conditions for a global exclusion may be whether the global exclusion is stored in one of the threat analysis data repositories 120.

[0085] Time-based exclusions, as the name suggests, can have one or more time-based conditions. For an endpoint designated as a time-based exclusion, any endpoint data related to that endpoint can trigger a determination of whether one or more time-based conditions are met. An example of a time-based condition is whether a threshold window time (e.g., a few seconds) has elapsed since an IOC was received for the endpoint (e.g., since the first IOC was received for the endpoint). If the time-based condition is not met (e.g., the threshold window time has elapsed since the IOC was received for the endpoint), the time-based exclusion is enforced, and as a result, the time-based exclusion can prevent the endpoint, or any dispositions related to the endpoint, from being included in any disposition feed. If the time-based condition is met (e.g., the threshold window time has not elapsed since the IOC was received for the endpoint), the time-based exclusion is not enforced, and as a result, the endpoint, or any dispositions related to the endpoint, may be included in the disposition feed. Thus, based on time-based exclusions, a disposition feed may include dispositions for endpoints designated as time-based exclusions within the threshold window time, but not outside of the threshold window time. In some variations, any disposition against an endpoint designated as a time-based exclusion may be included in the temporary feed for time-based exclusions during the threshold window period. In some variations, only the temporary feed may include dispositions against an endpoint while the temporary feed is being built. Time-based exclusions may be for specific endpoints (e.g., domains) that are trusted and / or may rarely pose a cyber threat. Thus, the temporary feed may indicate that the dispositions included are against endpoints within a trusted network infrastructure. Thus, time-based exclusions may indicate that the endpoint is within a trusted network infrastructure.Furthermore, using time-based exclusions can enable a rapid response to emerging cyber threats within trusted network infrastructure, and / or allow for further consideration of these threats before taking further action.

[0086] In variations, there may be other types of exclusions. Provider-based exclusions and feed-based exclusions are two additional examples of exclusion types that may be supported by the computer environment 100. Provider-based exclusions allow a condition to be applied to an endpoint to exclude IOCs received from a specific provider. In this way, even if a decision is made and sent to the endpoint, it may be done in a way that excludes IOCs received from excluded providers. Provider-based exclusions may be useful, for example, when a particular provider is not very trustworthy and / or when it is found that there are many false positives of cyber threats brought about by the endpoint. Feed-based exclusions allow a condition to be applied to an endpoint to exclude IOCs received from a specific provider feed. In this way, even if a decision is made and sent to the endpoint, it may be done in a way that excludes IOCs received from excluded provider feeds. Feed-based exclusions may be useful, for example, when a particular provider's feed is not very trustworthy and / or when it is found that there are many false positives of cyber threats brought about by the endpoint.

[0087] In some variations, the computer environment 100 may be configured to cause exclusions to expire. This expiration can be performed so that only certain types of exclusions expire, or so that all types of exclusions expire. Expiring an exclusion may involve removing it from one or more of the threat analysis data repositories 120. For example, to cause an exclusion to expire, it may be removed from the exclusion data repository 124. Another example is that to cause an exclusion to expire, it may be shown as having been removed from the event data repository 121. Details on removal from the event data repository 121 will be discussed later.

[0088] Continuing with a more detailed description of the exemplary computer environment 100, one or more threat analysis agents 130 are shown to receive endpoint data 108, execute one or more processes to ingest and analyze the endpoint data, and output threat differential data 135. This endpoint data 108 may be based on CTI data (e.g., CTI data 103 as shown in the generalized example of endpoint data 108), exclusion data (e.g., exclusion data 106), or other data received by one or more data collection agents 105. One or more threat analysis agents 130 can receive endpoint data 108 after receiving notification that endpoint data 108 is available. Based on the receipt of endpoint data 108, one or more threat analysis agents 130 can execute one or more processes to ingest and analyze the endpoint data 108.

[0089] As part of performing one or more processes to ingest and analyze endpoint data 108, one or more threat analysis agents 130 may determine the threat status of the endpoint indicated by the endpoint data 108 based on the endpoint data 108. In some variations, the threat status can indicate various statuses such as changed, unchanged, duplicate, etc. To illustrate an example of threat status, assume that the endpoint data 108 includes attribute-value pairs that indicate the endpoint's IOCs. In this example, the threat status may indicate changed if the IOCs for the endpoint have changed based on one or more previous IOCs for the endpoint; the threat status may indicate unchanged if the IOCs for the endpoint have not changed based on one or more previous IOCs for the endpoint; and the threat status may indicate duplicate if the IOCs for the endpoint are the same as one or more previous IOCs.

[0090] The threat status may be determined based on a comparison between endpoint data 108 and additional context stored in the event storage repository 121. The additional context may represent various attributes of the threat posed by the endpoint indicated by endpoint data 108. For example, the additional context may include endpoint IOCs previously received from providers 101-1 to 101-X. As will be discussed in more detail later, the format and notation of the additional context stored in the event storage repository 121 may be similar to that of endpoint data 108. Therefore, the determination of the threat status may be performed by comparing matching attributes between endpoint data 108 and the additional context stored in the event storage repository 121 and determining whether the values ​​of the matching attributes differ from each other.

[0091] The event storage repository 121 may be configured as a time series of events across all providers 101-1 to 101-X that have been stored. In this manner, the event storage repository 121 can store a time-based record containing or indicating all objects of endpoint data received by one or more threat analysis agents 130. This time-based record may indicate, for example, when an IOC for an endpoint is repeatedly received from the same provider and / or provider feed (e.g., by indicating that an IOC for an endpoint was received redundantly), when an IOC is removed from the provider feed (e.g., by indicating that an IOC for an endpoint was removed from the provider feed), and / or when an IOC for an endpoint changes (e.g., by indicating the changed IOC for the endpoint). The event storage repository 121 and / or its time-based record may be searchable and indexed according to various attributes (e.g., provider, provider feed, endpoint, etc.).

[0092] In variations where the event storage repository 121 is configured as a time series of stored events, the threat status may be determined based on the time series of stored events. For example, additional context may include one or more recently stored events relating to the endpoint indicated by the endpoint data 108. In this way, the threat status may be determined based on one or more recently stored events, perhaps only one recently stored event. As another example, additional context may include events stored within a threshold window time. In this way, the threat status may be determined based on stored events created and / or stored within that window time. As yet another example, additional context may include stored events indicating an ongoing cyber threat posed by the endpoint (for example, additional context may include events stored from the current time up to a time when, as indicated by the time series of stored events, the endpoint's IOC does not enter the threshold time from another IOC of that endpoint).

[0093] The computer environment 100 may be configured such that event data is not deleted from the event storage repository 121 (for example, never deleted, or not deleted for a period of time in units of weeks, months, or years). In this way, for example, if provider 101-1 to provider 101-X deletes an IOC from its data, event data indicating that the IOC has been deleted may be stored in the event storage repository 121. The deletion or expiration of exclusions may be handled in a similar manner by storing event data indicating the deletion or expiration of exclusions in the event storage repository 121. Thus, the process of storing event data indicating deletion, erasure, expiration, etc., is in contrast to deleting actual data from the event storage repository 121. Deleting actual data from the event storage repository 121 may include, for example, searching for event data containing an IOC based on the deletion of an IOC from a provider feed and deleting the event data containing the IOC from the event storage repository 121.

[0094] In variations that include an exclusion data repository 124, the exclusion data repository 124 may be configured similarly to the event data 121, except that it is dedicated to storing exclusion data. For example, the exclusion data repository 124 may be configured as a time series of stored events across all exclusion providers 104-1 to 104-Y. The threat status may be determined based on the exclusion data repository 124. The computer environment 100 may be configured such that event data is not deleted from the exclusion data repository 124 (for example, never deleted, or not deleted for a period of time in units of weeks, months, or years).

[0095] Based on the threat status, one or more threat analysis agents 130 can perform various threat status-specific actions. For example, in some variations, the threat status may indicate various statuses such as changed, unchanged, duplicate, etc. If the threat status indicates a change, one or more threat analysis agents 130 can perform actions based on the decision that a change has occurred. Such actions include saving event data to the event data repository 121 based on the type of change that occurred. For example, event data that adds an IOC to an endpoint may be saved to the event data repository 121, event data indicating the removal of an IOC to an endpoint may be saved to the event data repository 121, and / or event data indicating a change in an IOC to a previously added endpoint may be saved to the event data repository 121. Changes can occur in various forms. For example, one change is when the IOC in endpoint data 108 is different from the IOC in additional context saved by the event storage repository, even if that IOC is from the same provider and / or provider feed. In other words, if different IOCs are received from the provider and / or provider feed, it may indicate that the threat status has changed.

[0096] If the threat status indicates no change, one or more threat analysis agents 130 can take action based on the decision that no change has occurred. An example of no change is when the IOCs in the endpoint data 108 are no different from IOCs in additional context, with respect to the provider and / or provider feed. In other words, if the same IOCs are received from different providers and / or via different provider feeds, the threat status may indicate no change.

[0097] If the threat status indicates duplication, one or more threat analysis agents 130 can take action based on the determination that duplicate endpoint data exists. One example of duplication is receiving the same IOC from the same provider and / or provider feed. In other words, if the same IOC is received repeatedly from the same provider and / or provider feed at different times, the threat status may indicate duplication. Change, no change, and duplication are just three examples of the types of statuses that the exemplary computer environment 100 may support.

[0098] Threat status-specific actions performed based on the threat status may include storing data in a specific threat analysis data repository 121 and / or notifying one or more disposal feed agents 150 that data, such as threat differential data 135, is available. For example, if the threat status indicates a change, one or more threat analysis agents 130 may determine threat differential data 135 and event data (not shown) indicating the change. The event data may be stored in the event data repository 121. One or more disposal feed agents 150 may be notified that threat differential data 135 is available. Notifying one or more disposal feed agents 150 that threat differential data 135 is available may include sending the threat differential data 135 to at least one of the one or more disposal feed agents 150, inserting the threat differential data 135 into the feed between one or more threat analysis agents 130 and one or more disposal feed agents 150, and / or storing the threat differential data 135 in a location accessible to one or more disposal feed agents 150. As another example, if the threat status indicates no change, one or more threat analysis agents 130 may determine event data (not shown) indicating that there was no change. If the threat status indicates a duplicate, one or more threat analysis agents 130 may determine event data (not shown) indicating that a duplicate of the endpoint data 108 was found.

[0099] The threat differential data 135 may include or indicate context related to additional context stored by the endpoint data 108 and / or the event data repository 121. This context may provide the latest snapshot of the cyber threats brought about by the endpoints indicated by the endpoint data 108. The exact information contained in the threat differential data 135 may be based on what is contained in the endpoint data 108 and / or additional context stored by the event data repository 121. Due to the dynamic nature of cyber threats, the exact information contained in the threat differential data 135 may change over time as more CTI data is received (e.g., as IOCs are added / removed from the provider feed by providers 101-1 to 101-X) and / or as more exclusion data is received (e.g., as exclusion data is added / removed from the provider feed by providers 101-1 to 101-X).

[0100] Figure 1 shows a typical example of what threat differential data 135 may contain. As shown in Figure 1, threat differential data 135 may contain or indicate changes about the endpoint, such as what IOCs have changed between the endpoint data 108 and additional context stored in the event data repository 121. Threat differential data 135 may contain or indicate occurrences of IOCs for the endpoint across all providers 101-1 to 101-X. Occurrences are the number of IOCs received at the endpoint, the number of providers that provided the IOCs, and / or attributes describing the IOCs. Threat differential data 135 may contain or indicate one or more exclusions associated with the endpoint, such as whether the endpoint is or has been indicated as a global exclusion, a time-based exclusion, etc.

[0101] Event data may include or indicate threat status and other information related to how one or more threat analysis agents analyzed endpoint data 108. For example, event data may include the time the event was created or stored in the event storage repository 121, a display of the threat status (e.g., changed, unchanged, duplicate), and an event hash with a hash value calculated from one or more attribute-value pairs of the event data. In some variations, event data may be a copy of threat differential data 135, or otherwise include threat differential data 135. For example, in such a variation, if it indicates that the threat status has changed, event data may include a copy of threat differential data 135.

[0102] Both threat differential data 135 and event data may be in a similar format and notation to endpoint data 108. In this method, both threat differential data and event data may include one of the attribute value pairs described in relation to Table II. It may contain attribute value pairs in a format and / or notation similar to that of DATA108.

[0103] Table III provides more detailed examples of attribute-value pairs that may be included in threat differential data 135 and event data. Similar to the previous table, Table III provides example descriptions of attributes that may be included in threat differential data 135 and event data. Furthermore, for each example description, an example of attribute-value pair (e.g., attribute:value) is also provided. The examples in Table III are only a few examples of attributes and values ​​that may be included in threat differential data 135 and event data. In addition, since threat differential data 135 and event data may include any of the attribute-value pairs described in relation to Table II, what may be included in threat differential data 135 and event data may depend on how one or more data collection agents 105 determine endpoint data 108. Just as endpoint data 108 may depend on the provider, what may be included in threat differential data 135 and event data may depend on providers 101-1 to 101-X. [Table 3]

[0104] Due to the dynamic and data-intensive nature of cyber threat detection, in some variations, there may be multiple threat analysis agents 130 operating in the computer environment 100. Each of these multiple threat analysis agents 130 may operate in parallel with others, listen to any feed(s) from one or more data collection agents 105, receive endpoint data, process its own endpoint data, determine its own threat differential data, store its own event data, and so on. Furthermore, each of these multiple threat analysis agents 130 may have its own one or more machine learning models 130-1 and / or rules / policies 130-2.

[0105] Multiple threat analysis agents 130 can also receive other data not depicted in Figure 1. For example, multiple threat analysis agents 130 can receive endpoint queries (e.g., DNS queries) from devices associated with a company's customer (e.g., computer device 170). For example, computer device 170 may send a DNS query to identify an endpoint, and the threat analysis agent can receive the DNS query and determine the endpoint's threat differential data. In some variations, multiple threat analysis agents 130 can receive endpoint queries based on APIs that the customer and its devices can access and use, and / or via a feed mechanism similar to a provider's feed. More detailed examples involving endpoint queries will be discussed later in relation to Figure 10D.

[0106] As shown in Figure 1, one or more threat monitoring agents 140 may also determine threat differential data. In particular, one or more threat monitoring agents 140 are shown determining threat differential data 145 and / or sending it to one or more disposal feed agents 150. Threat differential data 145 may contain data similar to that described in relation to threat differential data 135. In fact, threat differential data 145 may include or indicate changes about the endpoint, occurrences of endpoint IOCs across all providers 101-1 to 101-X, and / or one or more exclusions related to the endpoint. Furthermore, threat differential data 145 may include the same or similar attribute-value pairs as described in relation to threat differential data 135 (for example, attribute-value pairs as described in relation to Tables II and III). However, instead of determining threat differential data 145 based on endpoint data 108 as shown in Figure 1, one or more threat monitoring agents 140 may determine threat differential data based on one or more processes for threat monitoring.

[0107] As part of one or more processes for threat monitoring, one or more threat monitoring agents 140 may monitor changes in data sources and / or repositories. Based on any changes, one or more threat monitoring agents 140 may decide whether to notify one or more disposal feed agents 150 of the changes. These changes may be notified to one or more disposal feed agents 150 by threat differential data 145. For example, one or more threat monitoring agents 140 may monitor any or all changes in the threat analysis data repository 120, evaluate the changes based on monitoring criteria, and, if the monitoring criteria are met, determine threat differential data 145 that includes information indicating or related to the changes.

[0108] More specifically, one or more threat monitoring agents 140 may monitor changes in the event data repository 121 (for example, event data that modifies an exclusion, or event data indicating that an exclusion has expired, event data that adds an IOC for an endpoint, or request data sent from a disposal feed agent indicating a request for more context for an endpoint). Based on these changes, one or more threat monitoring agents 140 may determine threat differential data 145 (for example, indicating a modified or expired exclusion, or indicating the scope of endpoints associated with an endpoint that has an added IOC, or indicating additional context for an endpoint based on a request from a disposal feed agent).

[0109] In variations that use the exclusion data repository 124, one or more threat monitoring agents 140 may monitor the exclusion data repository 124 for changes (e.g., changes to exclusions or indications that an exclusion has expired). Based on those changes, one or more threat monitoring agents 140 may determine threat differential data 145 (e.g., indicating changed or expired exclusions, or indicating the scope of endpoints associated with added or expired exclusions).

[0110] One or more threat monitoring agents 140 may monitor changes in the telemetry data repository 122. The telemetry data repository 122 may contain statistics, calculations, and other data determined in relation to the computer environment 100 (for example, data determined from any metadata stored in the raw data & metadata archive 114). The statistics, calculations, and other data stored by the telemetry data repository 122 may be determined by any of agents 105, 130, 140, 150, or by other software applications (not shown) running in the computer environment 100. Once determined, the statistics, calculations, and other data may be stored in the telemetry data repository 122. For example, one or more data collection agents 105 may process metadata into one or more statistics about providers, provider feeds, and endpoints before storing the metadata in the raw data & metadata archive 114, or they may process the metadata into a normalized format that transforms it from a proprietary format used by the CTI provider. One or more statistics and / or metadata are processed into a normalized format and stored in the telemetry data repository 122. One or more threat monitoring agents 140 may monitor changes in statistics, calculations, and other data stored by the telemetry data repository 122. Based on these changes, one or more threat monitoring agents 140 may determine threat differential data 145 (for example, indicating endpoints associated with CTI providers where statistics have changed).

[0111] Statistics, calculations, and other data stored in the telemetry data repository 122 may each be stored as an object containing various fields. For example, each object may contain one or more fields indicating the value of the statistics, calculations, or other data that the object covers. This value may have a specific data type (integer, string, floating-point number, etc.), a specific unit (millisecond, calculation, etc.), and a data fact (actual statistics, calculations, or other data, etc.). Each object may contain a field indicating the type of statistics, calculations, or other data that the object covers. Each object may contain a field (for example, in milliseconds) indicating the time when the statistics, calculations, or other data were created or stored in the telemetry data repository 122. Each object may contain a field indicating where the statistics, calculations, or other data were created (for example, a location that identifies a specific data center, cloud service, or geographical region where the data center or cloud service is located). Each object may contain a field indicating additional information about the location (for example, a location that identifies the zone of the data center where the statistics were determined). Each object may include fields that provide additional context for statistics, calculations, or other data (for example, one or more endpoints related to statistics, calculations, or other data, one or more CTI providers related to statistics, calculations, or other data, and / or one or more provider feeds related to statistics, calculations, or other data). Due to the dynamic nature of cyber threats and the data-intensive nature of their detection, there is a broad and virtually unlimited diversity of types of statistics, calculations, or other data that can be created and / or stored in the telemetry data repository 122.To provide some additional generalized examples of the types of statistics, calculations, or other data that can be created and / or stored in the telemetry data repository 122, the telemetry data repository 122 can be queried (for example, by a human operator and / or one or more monitoring agents 140) to collect statistical, calculation, and other data answers to queries such as: the total time taken to collect feeds from the provider; the total time for an endpoint (for example, an IOC of an endpoint received with CTI data) from receipt by one or more data collection agents 105 until the disposal is included in the disposal feed 154 based on the endpoint; the elapsed time until all CTI data from the provider's feed is received and processed by one or more data collection agents 105; the number of disposal feed agents out of one or more disposal feed agents 150 that have built and / or inserted disposals into the disposal feed based on specific threat differential data, the number of disposal feed agents currently operating in the computer environment 140, and the elapsed time to build the disposal feed.

[0112] One or more threat monitoring agents 140 may monitor changes in the detection data repository 123. The detection data repository 123 may include statistics, calculations, and other data determined in relation to the device receiving the disposal feed 154. As some examples, statistics, calculations, and other data may be determined to indicate how much and how often network traffic for which endpoints was blocked or monitored by disposals sent via the disposal feed. Statistics, calculations, and other data may be determined to indicate how many and how often queries were sent to which endpoints from the device receiving the disposal feed 154 (e.g., how many DNS queries were sent by computer device 170). These statistics, calculations, and other data are determined by the device receiving the disposal feed 154 (e.g., computer device 170 configured as RULEGATE by Centripetal, which can determine statistics about how RULEGATE monitors and / or blocks network traffic), or by another device communicating with such device (e.g., one or more threat monitoring agents 140 and / or one or more disposal feed agents 150). Once determined, statistics, calculations, and other data may be stored in the detection data repository 123 (for example, by one or more threat monitoring agents 140 and / or one or more disposal feed agents 150). One or more threat monitoring agents 140 may monitor changes in the statistics, calculations, and other data stored by the detection data repository 123. Based on those changes, one or more threat monitoring agents 140 may determine threat differential data 145 (for example, indicating endpoints related to DNS queries received from computer device 170).

[0113] One or more threat monitoring agents 140 may also monitor additional or alternative data sources and / or repositories, not just those shown in Figure 1. For example, an operator may monitor a data source that transmits signals from a human operator. This allows endpoints to be identified in the threat differential data 145 based on human-observed behavior or as a way to trigger a reconfiguration of the disposal feed 154.

[0114] One or more threat monitoring agents 140 may also access information stored in the threat analysis data repository 120 in connection with one or more processes for threat monitoring. For example, several changes (e.g., changes in exclusions) and / or several monitoring criteria (e.g., seeing whether an endpoint is associated with other endpoints when an exclusion is changed) may allow one or more threat monitoring agents 140 to access the address data repository 126. The address data repository 126 may contain a list of all non-overlapping CIDR ranges. Based on several changes and / or several monitoring criteria, one or more threat monitoring agents 140 may identify the endpoint associated with the changes and / or monitoring criteria, search the list of all non-overlapping CIDR ranges for the range of CIDR addresses containing the endpoint, and determine threat differential data 145 that shows the range of CIDR addresses.

[0115] In a variation that allows one or more disposal agents 150 to communicate with one or more threat monitoring agents 140, the one or more threat monitoring agents 140 may monitor request data sent from the one or more disposal agents 150 (for example, a disposal agent may send data indicating a request for more context about the endpoint). Based on the received request data, the one or more threat monitoring agents 140 access various repositories 110, search for stored information related to the endpoint, analyze and / or filter the stored information so that additional context is determined about the endpoint, and determine threat differential data 145 based on the additional context (for example, indicating additional context about the endpoint requested by the disposal feed agent).

[0116] The above description of one or more threat monitoring agents 140 provides some examples of different changes that can be monitored by one or more threat monitoring agents 140, and / or different monitoring criteria that can be implemented by one or more threat monitoring agents 140. Since there are many different types of monitoring criteria that can be implemented by one or more threat monitoring agents 140, there are many different changes that can be monitored by one or more threat monitoring agents 140. In fact, due to the dynamic nature of cyber threats and the data-intensive nature of their detection, such agents can monitor a wide variety of changes or implement countless combinations of monitoring criteria.

[0117] Given the wide variety of changes and / or monitoring criteria, in some variations, there may be multiple threat monitoring agents 140 operating in the computer environment 100. For example, each of the multiple threat monitoring agents 140 may monitor its own changes and / or enforce its own monitoring criteria. Each of the multiple threat monitoring agents 140 may operate in parallel with others, monitoring its specific changes, determining whether its monitoring criteria are met, determining its threat differential data, and so on. Furthermore, each of the multiple threat monitoring agents 140 may have its own one or more machine learning models 140-1 and / or rules / policies 140-2. In this way, one or more machine learning models 140-1 and / or rules / policies 140-2 may be configured to be used in relation to the threat monitoring agent's monitoring of changes and / or enforcement of monitoring criteria.

[0118] In some variations, one or more threat monitoring agents 140 may determine and store event data (not shown) related to one or more processes for threat monitoring in the event data repository 121. For example, one or more threat monitoring agents 140 may determine and store event data that indicates or includes event data indicating monitoring criteria, threat differential data 145, etc. The event data determined by one or more threat monitoring agents 140 may be in a common format and / or common notation used throughout the computer environment 100. Furthermore, the event data may include attribute-value pairs similar to those described above in relation to Tables II and III, which have additional attribute-value pairs indicating information about monitoring criteria, threat monitoring agents, etc.

[0119] As shown in Figure 1, one or more disposal feed agents 150 are shown to receive threat differential data (e.g., threat differential data 135 from one or more threat analysis agents 130 and threat differential data 145 from one or more threat monitoring agents 140), perform one or more processes for determining a disposal feed, and output feed notifications 152 and disposal feeds 154 (e.g., disposal feed 1 to disposal feed Z), any or all of which may be received by computer equipment 170 via network 160. Disposal feed 154 may include disposals determined on an endpoint basis (as opposed to provider-based and / or provider-based). In this way, disposal feed 154 may include individual disposals for each endpoint indicated by various objects of threat differential data received by one or more disposal feed agents 150. The remaining description of one or more disposal feeds 150 provides further examples of disposals determined on an endpoint basis.

[0120] As part of one or more processes for determining a disposal feed, one or more disposal feed agents 150 may receive threat differential data (e.g., threat differential data 135 or threat differential data 145) and determine whether the threat differential data meets the feed criteria. If the threat differential data meets the feed criteria, one or more disposal feed agents 150 may, if necessary, construct a disposal feed, determine disposals based on the threat differential data, and include or transmit disposals through the disposal feed. If the threat differential data does not meet the feed criteria, one or more disposal feed agents 150 may wait for the next threat differential data to be received and / or dismantle the disposal feed.

[0121] Due to the dynamic nature of cyber threats and the data-intensive nature of their detection, there is broad, almost unlimited diversity in the types of feed criteria that can be used by one or more disposal feed agents 150. Furthermore, users of devices receiving disposal feeds 154 may have their own unique needs and preferences for filtering network traffic, and therefore each user and / or device receiving a disposal feed can define or use as the basis for defining its own feed criteria. Exclusion data can form the basis of feed criteria (for example, feed criteria can be established to enforce or not enforce time-based exclusions), which can further increase the diversity of feed criteria. Machine learning models can be trained and used as the basis for determining feed criteria to build new disposal feeds, which can further increase the diversity of feed criteria. As some generalized examples of the types of feed criteria that may be used by one or more disposal feed agents 150, the feed criteria may be based on endpoints indicated by threat differential data, changes indicated by threat differential data, IOCs indicated by threat differential data, providers indicated by threat differential data, provider feeds indicated by threat differential data, exclusions indicated by threat differential data, arbitrary address data indicated by threat differential data, exclusion data indicated by threat differential data, exclusion data related to endpoints indicated by threat differential data, etc.

[0122] In some variations, there may be multiple disposal feed agents 150 operating in the computer environment 100. For example, each of the multiple disposal feed agents 150 can use its own feed criteria to build its own disposal feed. In this way, each of the multiple disposal feed agents 150 can build and dismantle its own disposal feed. Each of the multiple disposal feed agents 150 can operate in parallel with others, monitoring threat differential data, determining whether the threat differential data meets its feed criteria, and making disposal decisions based on the threat differential data, etc. Furthermore, each of the multiple disposal feed agents 150 may have its own one or more machine learning models 150-1 and / or rules / policies 150-2. In this way, one or more machine learning models 150-1 and / or rules / policies 150-2 may be configured to be used in conjunction with its own disposal feed agent.

[0123] In some further variations, there may be a second set of multiple disposal feed agents 150 operating in the computer environment 100. For example, each of these second sets may have its own one or more machine learning models 150-1 and / or rules / policies 150-2 that are trained and used as a basis for determining new feed criteria for a new disposal feed. In this way, the second set may determine new feed criteria based on threat differential data. These new feed criteria can then be used by a new disposal feed agent to construct a new disposal feed.

[0124] Feed criteria may be stored in the disposal feed criteria repository 127. In some variations, the disposal feed criteria repository 127 may store feed criteria and data that associates those feed criteria with a specific disposal feed agent among one or more disposal feed agents 150. In other variations, the disposal feed criteria repository 127 may store the code of one or more disposal feed agents 150, which may include feed criteria. In other variations, the disposal feed criteria repository 127 is not used, and the code and feed criteria of one or more disposal feed agents 160 may be stored in an alternative location (for example, within one or more computer devices running one or more disposal feed agents 150).

[0125] As described above, if the feed criteria are met, one or more disposal feed agents 150 may, as necessary, construct a disposal feed, determine disposals based on threat differential data, and include or send disposals through the disposal feed. When constructing a disposal feed, one or more disposal feed agents 150 may assign a name to the disposal feed that will be used to uniquely identify the disposal from other disposal feeds. This name can be determined in various ways, such as randomly or based on naming rules. For example, the naming rules may be based on any provider and / or any endpoint indicated by the threat differential data. Furthermore, the naming rules may be based on mapping information stored by the CTI mapping repository 128 (for example, the name may use a common format and / or common notation used throughout the computer environment 100).

[0126] If the feed criteria are met, one or more disposition feed agents 150 may decide on a disposition based on the threat differential data. A disposition may indicate the threat level of an endpoint, and may cause a device receiving the disposition to filter network traffic associated with the endpoint based on the severity of the threat posed by the endpoint. For example, a disposition may indicate removing an endpoint, which would remove the endpoint from the disposition feed (for example, because the endpoint is no longer a threat and / or is subject to exclusion). A disposition may indicate adding an endpoint, which would add the endpoint to the disposition feed (for example, because the endpoint is a new threat and / or is no longer subject to exclusion). A disposition may indicate instructing monitoring an endpoint, which would instruct blocking the endpoint (for example, because the endpoint is a growing threat). The above are just examples of what a disposition can indicate. In some variations, the disposal may be of a lesser type (e.g., monitoring or blocking only) or a more more type (e.g., the disposal indicates a time-based exclusion on the endpoint to show that further disposals will be carried in a temporary disposal feed during the time window for time-based exclusions).

[0127] Due to the dynamic nature of cyber threats and the data-intensive nature of their detection, there is a wide and almost limitless variety in the one or more methods by which the feed agent 150 determines sanctions based on threat differential data. Furthermore, users of devices receiving the sanction feed 154 may have their own unique needs and preferences for filtering network traffic, and therefore each user and / or device receiving the sanction feed can be used as a basis for how sanctions are determined. As some generalized examples of how one or more sanction feed agents 150 can determine sanctions based on threat differential data, sanctions may be determined based on endpoints indicated by threat differential data, changes indicated by threat differential data, IOCs indicated by threat differential data, providers indicated by threat differential data, provider feeds indicated by threat differential data, exclusions indicated by threat differential data, address data indicated by threat differential data, exclusion data indicated by threat differential data, exclusion data related to endpoints indicated by threat differential data, etc. Furthermore, sanctions may also be determined based on statistics, calculations, and other data determined based on the above examples. In practice, a disposition may be determined by first determining a confidence value that indicates the credibility of the threat level (for example, a confidence value associated with an IOC that indicates the credibility of the IOC's threat level), and then determining whether the confidence value exceeds one or more confidence thresholds (for example, a first threshold for monitoring dispositions, and a second higher threshold for blocking dispositions). Alternatively, a disposition may be determined by first determining the count of an IOC or provider as indicated by the threat differential data, and then determining whether the count is greater than one or more count thresholds (for example, a first threshold for monitoring dispositions, and a second higher threshold for blocking dispositions).

[0128] As some examples of how one or more disposition feed agents 150 may determine a disposition based on threat differential data, a disposition may be determined based on the number of providers 101-1 to 101-X that have shown an IOC for the endpoint (for example, if at least three providers have sent data showing an IOC for the endpoint, a disposition to monitor network traffic related to the endpoint may be determined, and if at least seven providers have sent data showing an IOC for the endpoint, a disposition to block network traffic related to the endpoint may be determined). A disposition may also be determined based on which of providers 101-1 to 101-X showed an IOC for the endpoint (for example, if a provider is associated with a reliability value indicating a low level of trust in the provider, a disposition to monitor may be determined, and if a provider is associated with a reliability value indicating a high level of trust in the provider, a disposition to block may be determined). A disposition may also be determined based on whether one or more of providers 101-1 to 101-X have shown the same IOC repeatedly for the endpoint (for example, based on threat differential data showing how many duplicates have been received for the endpoint, a disposition to block if the number of duplicates exceeds a threshold). Disciplinary action may be determined based on the time difference between received IOCs for an endpoint (for example, if the time difference is below a threshold, a discipline to block the endpoint may be determined). Disciplinary action may be determined based on the time that indicates how long the endpoint has been subject to the current discipline (for example, if the endpoint has been subject to monitoring discipline for at least a threshold of time, a discipline to block the endpoint may be determined). Disciplinary action may be determined based on confidence values, weights, attribute-value pairs, or other combinations of data (for example, a discipline may be determined based on a first confidence value associated with a first IOC, a second confidence value associated with one or more attributes of the threat differential data, and a third confidence value associated with at least one provider).

[0129] After a disposition has been determined, one or more disposition feed agents 150 may include the disposition in the disposition feed. Once a disposition is included in the disposition feed, the disposition feed may deliver or transmit the disposition to a device (e.g., computer device 170) that can receive the disposition feed in real time and / or based on an API request. The disposition feed can take various forms. For example, a disposition feed may be a DNS feed (e.g., a CleanDNS feed), an Advanced Cyber ​​Threat (ACT) feed, a Response Policy Zone (RPZ) feed, and / or a composite feed. A DNS feed (e.g., a CleanDNS feed) may allow for a feed built to the needs of a particular user (for example, a DNS feed may be built based on feed criteria specific to device 170 or users of device 170, and / or based on a specific endpoint request from device 170). An ACT feed may provide the disposition to a rule enforcement agent (e.g., computer device 170 which may be configured as a RULEGATE by Centripetal) that receives the feed. An RPZ feed may allow an RPZ file to contain dispositions and be available for requests (for example, responding to endpoint requests with dispositions for requested endpoints based on the RPZ file, and / or download requests for the RPZ file). A composite feed may be configured for a specific threat level and / or a specific set of endpoints or providers. In this way, a composite feed may allow a device to receive a feed containing only the desired level of threat (for example, a composite feed containing only block dispositions), the desired endpoints (for example, a composite feed containing dispositions for a specific CIDR range of endpoints), and / or the desired providers (for example, a composite feed containing dispositions for IOCs sent by provider 101-1).

[0130] Figure 1 shows a typical example of a disposal feed and the feed data 155 it contains. As shown, disposal feed Z contains feed data 155. The format of feed data 155 may depend on the type of feed (e.g., DNS feed, ACT feed, RPZ feed, composite feed). Feed data 155 may also contain one or more disposals. Each disposal may indicate the threat level of an endpoint, and once received, it may cause the receiving device to filter traffic based on the threat level. For example, if a disposal indicates monitoring an endpoint, the receiving device (e.g., device 170) may record network traffic associated with the endpoint. This record may be sent back to the computer environment 100 for storage and / or to determine statistics, calculations, and other data based on the record and store them in the discovery data repository 123. If a disposal indicates blocking an endpoint, the receiving device (e.g., device 170) may block network traffic associated with the endpoint. The display of what network traffic is being blocked may be sent back to the computer environment 100 for storage and / or to determine statistics, calculations, and other data based on the records and to be stored in the detection data repository 123.

[0131] One or more disposal feed agents 150 may contain any number or combination of the above types of feeds. The exact number or combination may change over time. As a way of notifying the device which feeds are currently being built, one or more disposal feed agents 150 may identify the built feeds in a feed notification 152. The feed notification 152 may include the name of the disposal feed currently being built and / or an indication of what feed criteria are being used for the disposal feed being built. In this way, the receiving device (e.g., device 170) can determine the received disposal feed and begin receiving the desired disposal feed.

[0132] In some variations, one or more disposal feed agents 150 may determine event data (not shown) related to one or more processes for determining a disposal feed and store it in the event storage repository 121. For example, one or more disposal feed agents 150 may determine and store event data indicating a constructed disposal feed, event data indicating that an API call was made through the disposal feed, and so on. The event data determined by one or more disposal feed agents 150 may be in a common format and / or common notation used throughout the computer environment 100. Furthermore, the event data may include attribute-value pairs similar to those described above in relation to Tables II and III, along with additional attribute-value pairs indicating information about disposal feeds, API calls, disposal feed agents, etc.

[0133] Having described an example of the computer environment 100 in Figure 1, we will now describe examples of machine learning models and / or rules / policies for various agents 105, 130, 140, and 150 in Figure 1. These examples are shown in Figures 2, 3A-3D, 4, and 5A-5B. Many of these examples include machine learning models, which may be neural networks of various configurations (e.g., feedforward neural networks, multilayer perceptron neural networks, radial basis function neural networks, recurrent neural networks, modular neural networks). These machine learning models may be trained using a controlled training process that uses human-labeled training data. Additional details regarding the training data will be provided where appropriate when describing the examples in Figures 2, 3A-3D, 4, and 5A-5B. Furthermore, in these examples, the common format and / or common notation refers to the common format and / or common notation described throughout the computer environment 100 in Figure 1, including Tables II and III. Moreover, these examples should be understood as variations that can operate in any combination within the computer environment 100 example in Figure 1 or similar computer environments. Also, for simplicity, the rules / policies shown in these examples are described as receiving output, making decisions, and / or performing analysis. This expression is used to simplify the fact that a computer device configured with an agent or rule / policy applies the rule / policy based on received input, making decisions, and / or performing analysis. In other words, this expression is used as a simplified representation of the fact that a rule / policy is used in relation to an agent or computer device that receives input, makes decisions, and / or performs analysis.

[0134] Beginning with Figure 2, Figure 2 shows examples 205, 210, and 215 of data collection agents (for example, one or more data collection agents 105 in Figure 1). Example data collection agent 205 uses a rule / policy 208 to determine endpoint data 209 based on data 206 (for example, CTI data 103 or exclusion data 106 in Figure 1). In this method, the rule / policy 208 may be written to map the data from the format and notation of data 206 to a common format and / or common notation of endpoint data 209.

[0135] Example 205 of the data collection agent uses a machine learning model 212 to determine endpoint data 213 based on data 211 (for example, CTI data 103 or exclusion data 106 in Figure 1). In this method, the machine learning model 212 may be configured to receive data in a specific format and notation (for example, the format of provider 101-1 in Figure 1) as input and to map the data from that format and notation to a common format and notation. The machine learning model 212 is trained using a corpus of data in a specific format and notation (for example, a corpus of data received from provider 101-1 in Figure 1).

[0136] Example 205 of the data collection agent uses a machine learning model 217 and rules / policies 218 to determine endpoint data 213 based on data 216 (e.g., CTI data 103 or exclusion data 106 in Figure 1). In this method, the machine learning model 212 may be configured to receive data in a specific format and notation (e.g., the format of provider 101-X in Figure 1) as input and map the data to a format that rules / policies 218 can convert to a common format and / or notation. The machine learning model 212 can be trained using a corpus of data with that specific format and notation (e.g., a corpus of data received from provider 101-X in Figure 1). Rules / policies 208 may be written to convert the output of the machine learning model 217 to a common format and / or common notation for endpoint data 209. Subsequently, Figure 3A shows examples 305, 315, and 320 of threat analysis agents (e.g., one or more threat analysis agents 130 in Figure 1). Each of the threat analysis agent examples 305, 315, and 320 uses rules / policies to analyze endpoint data (e.g., sent from one or more data collection agents 105) and event data (e.g., stored by the event storage repository 121 in Figure 1). More specifically, threat analysis agent example 305 includes a rule / policy 308 for determining that a change has occurred on an endpoint based on endpoint data 307 and event data 306. Based on that change, the rule / policy 308 may determine event data 309 and threat differential data 311. The rule / policy 308 may be created to determine that a change has occurred on an endpoint, determine event data 309, and determine threat differential data 311.

[0137] Example 315 of a threat analysis agent includes a rule / policy 318 to determine that no change has occurred for an endpoint, based on endpoint data 317 and event data 316. Based on the absence of change, the rule / policy 318 may determine event data 319 in a common format and / or notation. The rule / policy 318 may be created to determine that a change has occurred for an endpoint and to determine event data 309.

[0138] Example 320 of a threat analysis agent includes a rule / policy 328 for determining that an endpoint duplicate has been received, based on endpoint data 322 and event data 321. Based on the duplicate, the rule / policy 328 may determine event data 324 in a common format and / or notation. The rule / policy 328 may be created to determine that an endpoint duplicate has been received and to determine event data 319.

[0139] Next, in Figure 3B, examples 325, 335, and 340 of threat analysis agents (for example, one or more threat analysis agents 130 in Figure 1) are shown. Each of the threat analysis agent examples 325, 335, and 340 uses a machine learning model to analyze endpoint data (for example, sent from one or more data collection agents 105) and event data (for example, stored by the event storage repository 121 in Figure 1). More specifically, threat analysis agent example 325 includes a machine learning model 329 configured to analyze endpoint data 327 and event data 326. In this method, the machine learning model 329 may be configured to receive event data and endpoint data as input, determine that a change has occurred for the endpoint, and based on that determination, output event data 329 and threat differential data 331. Furthermore, the machine learning model 329 may indicate that a change has occurred for the endpoint based on a confidence value indicating whether a change has occurred for the endpoint. The machine learning model 329 may be trained using a corpus of combined event data and endpoint data. The combined corpus of event and endpoint data may include human-labeled combinations of event and endpoint data, where the labels indicate whether the combination is varied, unchanged, or duplicated.

[0140] An example of a threat analysis agent 335 includes a machine learning model 338 for analyzing endpoint data 336 and event data 337. In this method, the machine learning model 338 may be configured to receive event data and endpoint data as input, determine that no changes have occurred for the endpoint, and, based on that determination, output the event data 338 in a common format and / or common notation. Other outputs of the machine learning model 338 may be ignored. Furthermore, the machine learning model 338 may indicate that no changes have occurred for the endpoint based on a confidence value indicating whether changes have occurred for the endpoint. The machine learning model 338 may be trained using a corpus of combined event data and endpoint data. The corpus of combined event data and endpoint data may include human-labeled combinations of event data and endpoint data, where the labels indicate whether the combination has changed, not changed, or is duplicated.

[0141] An example of a threat analysis agent 340 includes a machine learning model 343 for analyzing endpoint data 342 and event data 341. In this method, the machine learning model 343 may be configured to receive event data and endpoint data as input, determine if an endpoint duplicate has been received, and, based on that determination, output event data 344 in a common format and / or common notation. Other outputs of the machine learning model 343 may be ignored. Furthermore, the machine learning model 343 may indicate that an endpoint duplicate has been received based on a confidence value indicating whether or not a duplicate has been received. The machine learning model 343 may be trained using a corpus of combined event data and endpoint data. The corpus of combined event data and endpoint data may include human-labeled combinations of event data and endpoint data, where the labels indicate whether the combination is varied, unchanged, or duplicate.

[0142] Next, in Figure 3C, examples 345, 355, and 365 of threat analysis agents (for example, one or more threat analysis agents 130 in Figure 1) are shown. Each of the threat analysis agent examples 345, 355, and 365 uses a machine learning model to analyze endpoint data (for example, sent from one or more data collection agents 105) and event data (for example, stored by the event storage repository 121 in Figure 1). As shown by each of the threat analysis agent examples 345, 355, and 365, the output of the machine learning model is used as the basis for rules / policies to determine event data and / or threat differential data. More specifically, threat analysis agent example 345 includes a machine learning model 348 configured to analyze endpoint data 347 and event data 346. In this method, the machine learning model 348 may be configured to receive event data and endpoint data as input, determine that a change has occurred for an endpoint, and output at least a confidence value for whether a change has occurred for an endpoint. Rule / policy 349 may receive a confidence value (and other outputs from machine learning model 348). Based on the confidence value, rule / policy 349 may determine that a change has occurred and, based on event data 346 and endpoint data 347 (and / or other outputs from machine learning model 348), determine threat differential data 352 and event data 350. Machine learning model 348 is trained using a corpus of combined event and endpoint data. The corpus of combined event and endpoint data may include human-labeled combinations of event and endpoint data, where the labels indicate whether the combination has changed, not changed, or is duplicated. Rule / policy 349 may be created to determine that a change has occurred for an endpoint, determine threat differential data 352, and determine event data 350.

[0143] Example 355 of a threat analysis agent includes a machine learning model 358 configured to analyze endpoint data 357 and event data 356. In this method, the machine learning model 358 may be configured to receive event data and endpoint data as input, determine that no change has occurred for an endpoint, and output a confidence value of 3 or less for whether no change has occurred for an endpoint. A rule / policy 359 may receive the confidence value (and other outputs from the machine learning model 358). Based on the confidence value, the rule / policy 359 may determine that no change has occurred and, based on the event data 356 and endpoint data 357 (and / or other outputs from the machine learning model 358), determine event data 360 in a common format and / or common notation. The machine learning model 358 may be trained using a corpus of combined event data and endpoint data. The corpus of combined event data and endpoint data may include human-labeled combinations of event data and endpoint data, where the labels indicate whether the combination is changed, unchanged, or duplicated. Rule / policy 359 may be created to determine that no changes have occurred regarding the endpoint and to determine event data 360.

[0144] Example 365 of a threat analysis agent includes a machine learning model 368 configured to analyze endpoint data 367 and event data 366. In this method, the machine learning model 368 may be configured to receive event data and endpoint data as input, determine if an endpoint duplicate has been received, and output at least a confidence value for whether an endpoint duplicate has been received. A rule / policy 369 may receive the confidence value (and other outputs from the machine learning model 368). Based on the confidence value, the rule / policy 369 may determine if an endpoint duplicate has been received and, based on the event data 366 and endpoint data 367 (and / or other outputs from the machine learning model 368), determine the event data 370 in a common format and / or common notation. The machine learning model 368 may be trained using a corpus of combined event data and endpoint data. The corpus of combined event data and endpoint data may include human-labeled combinations of event data and endpoint data, where the labels indicate whether the combination is varied, unchanged, or duplicate. Rule / policy 369 may be created to determine that a duplicate endpoint has been received and to determine event data 370.

[0145] Next, in Figure 3D, examples 375, 385, and 392 of threat analysis agents (for example, one or more threat analysis agents 130 in Figure 1) are shown. Each of the threat analysis agent examples 375, 385, and 392 uses a machine learning model to analyze endpoint data (for example, sent from one or more data collection agents 105) and event data (for example, stored by the event storage repository 121 in Figure 1). As shown by each of the threat analysis agent examples 375, 385, and 392, the output of the machine learning model is used as the basis for rules / policies to determine event data and / or threat differential data. More specifically, threat analysis agent example 375 includes a machine learning model 378 configured to analyze endpoint data 377 and event data 376. In this method, the machine learning model 378 may be configured to receive event data and endpoint data as input, determine that a stored event exists for the endpoint indicated by the endpoint data 377 (for example, event data 376 contains a stored event for the endpoint indicated by the endpoint data 377), and output at least a confidence value for the stored events that exist. The rule / policy 379 may receive the confidence value (and other outputs from the machine learning model 378). Based on the confidence value, the rule / policy 379 may proceed to perform further analysis on the event data 346 and / or endpoint data 347. Based on the further analysis, the rule / policy 379 may determine that a change has occurred for the endpoint and determine threat differential data 382 and event data 380 based on the event data 376 and endpoint data 377 (and / or other outputs from the machine learning model 378). The machine learning model 378 may be trained using a corpus of combined event data and endpoint data.The combined corpus of event data and endpoint data may include human-labeled combinations of event data and endpoint data, where the labels indicate whether the event data contains data from the endpoint indicated by the endpoint data. Rules / policies 379 may be created to determine that a change has occurred for an endpoint, to determine threat differential data 382, ​​and to determine event data 380.

[0146] Example 385 of a threat analysis agent includes a machine learning model 388 configured to analyze endpoint data 387 and event data 386. In this method, the machine learning model 388 may be configured to receive event data and endpoint data as input, determine that there are stored events for the endpoint indicated by the endpoint data 387 (for example, event data 386 includes stored events for the endpoint indicated by the endpoint data 387), and output at least a confidence value for the stored events that exist. A rule / policy 390 may receive the confidence value (and other outputs from the machine learning model 388). Based on the confidence value, the rule / policy 390 may proceed to perform further analysis on the event data 386 and / or endpoint data 387. Based on the further analysis, the rule / policy 390 may determine that no changes have occurred for the endpoint, or that a duplicate endpoint has been received. Accordingly, based on the event data 386 and endpoint data 387 (and / or other outputs from the machine learning model 388), the rule / policy 390 may determine event data 391 in a common format and / or common notation. The machine learning model 388 may be trained using a corpus of combined event data and endpoint data. The corpus of combined event data and endpoint data may include human-labeled combinations of event data and endpoint data, where the labels indicate whether the event data contains data from the endpoint indicated by the endpoint data. The rule / policy 390 may be created to determine event data 390 in order to determine that no change has occurred for an endpoint and / or that a duplicate has been received for an endpoint.

[0147] Example 392 of a threat analysis agent includes a machine learning model 395 configured to analyze endpoint data 394 and event data 393. In this method, the machine learning model 395 may be configured to receive event data and endpoint data as input, determine that a stored event exists for the endpoint indicated by endpoint data 394 (for example, event data 393 contains a stored event for the endpoint indicated by endpoint data 394), and output at least a confidence value for the existing stored event. A rule / policy 396 may receive the confidence value (and other outputs from the machine learning model 395). Based on the confidence value, the rule / policy 396 may proceed to determine that a change has occurred for the endpoint (for example, because no stored event exists). Accordingly, based on the event data 393 and endpoint data 394 (and / or other outputs from the machine learning model 395), the rule / policy 396 may determine event data 398 and threat difference data 399. The machine learning model 395 may be trained using a corpus of combined event data and endpoint data. The combined event data and endpoint data corpus may include human-labeled combinations of event data and endpoint data, where the label indicates whether the event data contains endpoint data indicated by the endpoint data. Rules / policies may be created to determine when a change has occurred for an endpoint, determine threat differential data 399, and determine event data 398.

[0148] Next, in Figure 4, examples 405, 420, and 430 of threat monitoring agents (for example, one or more threat monitoring agents 140 in Figure 1) are shown. Each of the threat monitoring agent examples 405, 420, and 430 monitors one or more data repositories (for example, the threat analysis data repository 121 in Figure 1) using machine learning models and / or rules / policies. As also shown by each of the threat monitoring agent examples 405, 420, and 430, each of the threat monitoring agent examples 405, 420, and 430 determines threat differential data and / or event data using machine learning models and / or rules / policies based on the monitoring criteria that are met.

[0149] Example 405 of a threat monitoring agent includes a machine learning model 407 configured to analyze event data 406 (for example, stored in the event data repository 121 in Figure 1). In this method, the machine learning model 407 may be configured to receive event data as input, determine whether monitoring criteria are met based on the event data, and output at least a confidence value for whether the monitoring criteria are met. A rule / policy 409 may receive the confidence value (and other outputs from the machine learning model 407). Based on the confidence value, the rule / policy 409 may proceed to determine that the monitoring criteria are met (for example, because the confidence value is above a threshold). Accordingly, based on the event data 406 (and / or other outputs from the machine learning model 407), the rule / policy 409 may determine event data 410 and threat differential data 412. The machine learning model 407 may be trained using a corpus of event data. The corpus of event data may include human-labeled event data where labels indicate whether the event data meets the satisfaction criteria. Rule / policy 409 may be created to determine if the sufficiency criteria are met, to determine the threat differential data 412, and to determine the event data 410.

[0150] An example threat monitoring agent 420 includes a rule / policy 422 configured to analyze event data 421 (for example, stored in the event data repository 121 in Figure 1), determine if monitoring criteria are met, determine event data 423, and determine threat differential data 425. The event data 410 and threat differential data 412 may be in a common format and / or common notation. A rule / policy 409 may be created to determine if the satisfaction criteria are met, determine threat differential data 412, and determine event data 410.

[0151] An example threat monitoring agent 435 includes a machine learning model 432 configured to analyze event data 431 (for example, stored in the event data repository 121 in Figure 1). In this method, the machine learning model 432 may be configured to receive event data as input, determine whether monitoring criteria are met based on the event data, and output event data 433 and / or threat differential data 435 together in a common format and / or common notation. The machine learning model 407 may be trained using a corpus of event data. The corpus of event data may include human-labeled event data where labels indicate whether the event data meets the satisfaction criteria.

[0152] Next, in Figure 5A, examples 505, 515, 525, and 530 of disposal feed agents (for example, one or more disposal feed agents 150 in Figure 1) are shown. Each of the disposal feed agents 505, 515, 525, and 530 uses a machine learning model and / or rules / policies to determine whether the feed criteria are met, and if so, includes the feed data in the disposal feed. More specifically, example disposal feed agent 505 includes a machine learning model 508 configured to analyze threat differential data 507 based on the feed criteria 506. In this method, the machine learning model 508 can be configured to receive threat differential data as input, determine whether the feed criteria are met based on the threat differential data, and output at least a confidence value for whether the feed criteria are met. A rule / policy 509 can receive the confidence value (and other outputs from the machine learning model 508). Based on the confidence value, the rule / policy 509 may proceed to determine that the feed criteria are met (for example, because the confidence value is above a threshold). Accordingly, based on the threat differential data 507 (and / or other outputs from the machine learning model 506), the rule / policy 509 may determine feed data 510 containing dispositions for endpoints indicated by the threat differential data 507. The feed data 510 may be included in the disposition feed. The machine learning model 508 may be trained using a corpus of threat differential data. The corpus of threat differential data may include human-labeled threat differential data, where labels indicate whether the feed criteria are met. The rule / policy 509 may be constructed to determine that the feed criteria are met, determine the feed data 510 and the dispositions it contains, and include the feed data 510 in the disposition feed.

[0153] Example 515 of a disposal feed agent includes a machine learning model 518 configured to analyze threat differential data 517 based on a feed criterion 516. In this method, the machine learning model 518 may be configured to receive threat differential data as input, determine that the feed criterion is not met based on the threat differential data, and output at least a confidence value for whether the feed criterion is not met. A rule / policy 519 may receive the confidence value (and other outputs from the machine learning model 518). Based on the confidence value, the rule / policy 519 may proceed to determine that the feed criterion is not met (for example, because the confidence value is below a threshold). Accordingly, based on the threat differential data 517 (and / or other outputs from the machine learning model 516), the rule / policy 519 may decide not to output to the differential feed. The machine learning model 518 may be trained using a corpus of threat differential data. The corpus of threat differential data may include human-labeled threat differential data where labels indicate whether the feed criterion is met. A rule / policy 51 may be created to determine that the feed criterion is not met.

[0154] An example of a disposition feed agent 525 includes a rule / policy 528 configured to analyze threat differential data 527 based on a feed criterion 526 and determine that the feed criterion 526 is met based on the threat differential data 527. Based on the fulfillment of the feed criterion 526, the rule / policy 528 may determine feed data 529 that includes dispositions for endpoints indicated by the threat differential data 527. The feed data 529 may be included in the disposition feed. The rule / policy 528 may be written to determine that the feed criterion 527 is met, determine the feed data 529 and the dispositions it contains, and include the feed data 529 in the disposition feed.

[0155] Example 530 of a disposal feed agent includes a rule / policy 533 configured to analyze threat differential data 532 based on a feed criterion 531 and determine that the feed criterion 531 is not met based on the threat differential data 532. Based on the fact that the feed criterion 531 is not met, the rule / policy 533 may decide not to output to the disposal feed. The rule / policy 533 may be written to determine that the feed criterion 531 is not met.

[0156] Next, in Figure 5B, examples 540 and 550 of disposal feed agents (for example, one or more disposal feed agents 150 in Figure 1) are shown. Each example of disposal feed agents 540 and 550 uses a machine learning model and / or rules / policies to determine whether a new feed criterion should be used to create a new disposal feed. More specifically, example disposal feed agent 540 includes a machine learning model 542 configured to analyze threat differential data 541. In this method, the machine learning model 542 may be configured to receive threat differential data as input, determine a new feed criterion based on the threat differential data, and output at least one confidence value for the new feed criterion. A rule / policy 543 may receive at least one confidence value and the new feed criterion. Based on at least one confidence value, the rule / policy 543 may proceed to determine that the new feed criterion should form the basis of a new disposal feed (for example, because at least one confidence value is greater than or equal to a threshold). Accordingly, a new disposal feed agent 545 is created, and a feed criterion 544 containing the new feed criteria output by the machine learning model 542 is assigned to the new disposal feed agent 545, and the new disposal feed agent can create a new disposal feed based on the feed criterion 544. The machine learning model 542 may be trained using a corpus of threat differential data. The corpus of threat differential data may include human-labeled threat differential data where the labels indicate various criteria that can be used as the basis for the new feed criteria. A rule / policy 543 may be created to determine that the new feed criteria should form the basis for the new disposal feed and to create a new disposal feed agent.

[0157] Example 550 of a disposal feed agent includes a machine learning model 552 configured to analyze threat differential data 551. In this method, the machine learning model 552 may be configured to receive threat differential data as input, determine new feed criteria based on the threat differential data, and output at least one confidence value for the new feed criteria. A rule / policy 553 may receive at least one confidence value and the new feed criteria. Based on at least one confidence value, the rule / policy 553 may proceed to determine that the new feed criteria should not be the basis for a new disposal feed (for example, because at least one confidence value is below a threshold). The machine learning model 552 may be trained using a corpus of threat differential data. The corpus of threat differential data may include human-labeled threat differential data where the labels indicate various criteria that can be used as the basis for a new feed criterion. The rule / policy 553 may be written to determine that the new feed criteria should not be the basis for a new disposal feed.

[0158] Having described an example of the computer environment 100 in Figure 1 along with machine learning models and / or rule / policy examples for various agents 105, 130, 140, and 150 in Figure 1, we will now describe examples of methods that can be implemented by the various agents 105, 130, 140, and 150 in Figure 1. Figures 6A and 6B show examples 600 and 650 of methods that can be implemented by one or more data collection agents 105 in Figure 1 (for example, as part of one or more processes for receiving, storing, and / or processing CTI data and / or exclusion data). Figure 7 shows example 700 of methods that can be implemented by one or more threat analysis agents 130 in Figure 1 (for example, as part of one or more processes for ingesting and analyzing endpoint data). Figure 8 shows example 800 of methods that can be implemented by one or more threat monitoring agents 140 in Figure 1 (for example, as part of one or more processes for threat monitoring). Figure 9 shows example 900 of methods that can be implemented by one or more disposal feed agents 150 in Figure 1 (for example, as part of one or more processes for determining disposal feeds). Examples of methods 600, 650, 700, 800, and 900 are just a few examples of the processes that can be performed by the various agents 105, 130, 140, and 150 in Figure 1. Other variations include omitting steps from examples of methods 600, 650, 700, 800, and 900, adding new steps to examples of methods 600, 650, 700, 800, and 900, and / or changing the order of steps from examples of methods 600, 650, 700, 800, and 900. In these examples of methods 600, 650, 700, 800, and 900, the common format and / or common notation refers to the common format and / or common notation described throughout the computer environment 100 in Figure 1, including Tables II and III. Furthermore, for simplicity, examples of methods 600, 650, 700, 800, and 900 will be described in terms of being performed by one or more computer devices.

[0159] We begin with Example 600 of the method in Figure 6A. In step 605, one or more computer devices may determine whether data of the CTI data type has been received from the provider. This determination may be based on a process of classifying the received data as a CTI data type. This classification may be the same as or similar to the classification described in relation to Figure 1. The data to be classified may be received via the provider's feed (for example, in real time or based on an API call). If data of the CTI data type has been received (for example, via provider feed 1 in Figure 1), the method 600 may proceed to step 615. If no data of the CTI data type has been received, the method 600 may proceed by repeating step 605, waiting for data of the CTI data type to be received. For simplicity, in the remaining steps of Figure 6A, data of the CTI data type will be referred to as "CTI data".

[0160] In step 615, one or more computer devices may store the CTI data for storage in a raw data archive (for example, the raw data & metadata archive 114 in Figure 1).

[0161] In step 620, one or more computer devices may determine endpoint data from the CTI data. Once determined, for example, the endpoint data may represent one or more IOCs of an endpoint. This determination involves extracting a portion from the CTI data and mapping that portion from the format and / or notation of the CTI data (e.g., a first format and / or first notation) to a common format and / or common notation of the endpoint data (e.g., a second format and / or second notation). Once determined, the endpoint data may contain attribute-value pairs similar to those described above in relation to Table II, where the attributes of the endpoint data may have values ​​based on the portion extracted from the CTI data and / or values ​​related to how the CTI data was received. This determination can be performed based on any variation of the example data collection agent described in relation to Figure 2.

[0162] In step 630, one or more computer devices may store the endpoint data in an endpoint data archive (for example, endpoint data archive 115 in Figure 1).

[0163] In step 635, one or more computer devices may determine CTI-based metadata based on endpoint data. This CTI-based metadata may be the same as or similar to the CTI-based metadata described in relation to Figure 1.

[0164] In step 640, one or more computer devices may store CTI-based metadata. The CTI-based metadata may be stored in a raw data archive (for example, raw data & metadata archive 114 in Figure 1).

[0165] In step 645, one or more computer devices may notify that endpoint data is available for analysis. Notifying that endpoint data is available for analysis may include: sending the endpoint data to at least one threat analysis agent (e.g., one or more threat analysis agents 130 in Figure 1); inserting the endpoint data into a feed between a data collection agent (e.g., one or more data collection agents 105 in Figure 1) and a threat analysis agent (e.g., one or more threat analysis agents 130 in Figure 1); and / or storing the endpoint data in a location accessible to the threat analysis agent.

[0166] Next, in example 650 of the method in Figure 6B, in step 655, one or more computer devices may determine whether data of an excluded data type has been received from the provider. This determination may be based on a process of classifying the received data as a CTI data type. This classification may be the same as or similar to the classification described in relation to Figure 1. The data to be classified may be received via the provider's feed (for example, in real time or based on an API call). If data of an excluded data type has been received (for example, via excluded feed Y in Figure 1), method 650 may proceed to step 665. If data of an excluded data type has not been received, method 650 may proceed by repeating step 655, waiting for data of an excluded data type to be received. For simplicity, in the remaining steps of Figure 6B, data of an excluded data type will be referred to as “excluded data”.

[0167] In step 665, one or more computer devices may store the excluded data in a raw data archive (for example, the raw data & metadata archive 114 in Figure 1) for storage.

[0168] In step 670, one or more computer devices may determine one or more exclusions for endpoints from the exclusion data. This determination involves extracting portions from the exclusion data and mapping those portions from the format and / or notation of the exclusion data (e.g., the first format and / or first notation) to a common format and / or common notation (e.g., the second format and / or second notation). The common format and / or common notation may be the same as or similar to the common format and / or common notation used for the endpoint data in step 620 of Figure 6A. Once determined, one or more exclusions may include attribute-value pairs similar to those described above in relation to Table II, as described in relation to Figure 1, and / or endpoint data determined based on the exclusion data. In fact, in some variations, one or more endpoints may be included in the endpoint object described in relation to Figure 1. The attributes of one or more exclusions may have values ​​based on the portions extracted from the exclusion data and / or values ​​related to how the exclusion data was received. This determination may be performed based on one of the variations of the data collection agent, such as those described in relation to Figure 2.

[0169] In step 680, one or more computer devices may store one or more exclusions in one or more data repositories (for example, endpoint data archive 115 and / or exclusion data repository 124 in Figure 1).

[0170] In step 685, one or more computer devices may determine exclusion-based metadata based on one or more exclusions. This exclusion-based metadata may be the same as or similar to the exclusion-based metadata described in relation to Figure 1.

[0171] In step 690, one or more computer devices may store exclusion-based metadata. Exclusion-based metadata may be stored in a raw data archive (for example, raw data & metadata archive 114 in Figure 1).

[0172] In step 695, one or more computer devices may notify that one or more exclusions are available for analysis. Notifying that one or more exclusions are available for analysis may include: sending one or more exclusions to at least one threat analysis agent (e.g., one or more threat analysis agents 130 in Figure 1); inserting one or more exclusions into the feed between a data collection agent (e.g., one or more data collection agents 105 in Figure 1) and a threat analysis agent (e.g., one or more threat analysis agents 130 in Figure 1); and / or storing the endpoint data in a location accessible to the threat analysis agent.

[0173] Next, in Example 700 of Method in Figure 7, we will describe Example 700 from the perspective of processing endpoint data. This endpoint data is the same as or similar to the endpoint data in Figure 6A. The same or similar methods may be used to process one or more exclusions in Figure 6B. Furthermore, as explained in relation to Figure 6B, in Figure 6B, one or more endpoints may be included in an object of endpoint data (for example, as explained in relation to Figure 1). In such variations, Example 700 of Method can be considered as processing either the endpoint data in Figure 6A, and / or the object of endpoint data in Figure 6B that includes one or more exclusions.

[0174] In step 705, one or more computer devices may receive endpoint data. This endpoint data may indicate one or more IOCs of an endpoint.

[0175] After receiving endpoint data, one or more computer devices may determine whether a change has occurred with respect to the endpoint and / or whether a duplicate endpoint has been received. The remaining steps 720–770 of Example Method 700 illustrate how one or more computer devices may make these decisions and what actions may be taken in response.

[0176] In step 720, one or more computer devices may determine whether a stored event exists for the endpoint data. This determination may include identifying the endpoint indicated by the endpoint data and searching the event data repository (for example, event data repository 121 in Figure 1) to determine whether a stored event indicating the endpoint exists. In some variations, this determination may be performed based on the variation described in relation to Figure 3D. If a stored event exists, Example Method 700 may proceed to step 725. If no stored event exists, Example Method 700 may proceed to step 750 (for example, to indicate that a change has occurred for the endpoint).

[0177] In step 725, one or more computer devices may determine the threat status by comparing at least stored events with endpoint data. This determination may be performed in the same or similar manner as the threat status determination described in relation to Figure 1. In some variations, the threat status may show change, no change, or duplication. Comparing at least stored events with endpoints may be performed in the same or similar manner as comparing endpoint data 108 with additional context stored in the event storage repository 121, where at least stored events are or contain additional context, as described in relation to Figure 1.

[0178] In step 730, one or more computer devices may determine whether the threat status has changed. If the threat status has changed, the method proceeds to step 750. If the threat status has not changed, the method proceeds to step 735.

[0179] In step 735, one or more computer devices may determine whether the threat status is duplicated. If the threat status is duplicated, method 700 may proceed to step 765. If the threat status is not duplicated, method may proceed to step 745.

[0180] In step 740, one or more computer devices may determine whether or not there has been a change in the threat status. If the threat status indicates no change, method 700 may proceed to step 770. If the threat status does not indicate no change, method 700 may proceed to step 745.

[0181] In step 745, one or more computer devices may save a display of the threat status. Saving the display of the threat status allows the threat status to be viewed again. The display of the threat status may be saved in an event data repository or other data repository.

[0182] In step 750, one or more computer devices may indicate that a change has occurred on the endpoint by determining event data and threat differential data. This determination may be performed in the same or similar manner as the determination of event data and threat differential data as described in relation to Figure 1. In this method, for example, the event data and threat differential data may be in a common format and / or common notation and may include attribute-value pairs as described in relation to Tables II and III. This determination may also be performed based on one or more variations as described in relation to Figures 3A-3C.

[0183] In step 755, one or more computer devices may store event data. The event data may be stored in an event data repository (for example, event data repository 121 in Figure 1).

[0184] In step 760, one or more computer devices may notify that threat differential data is available. Notifying that threat differential data is available may include: sending the threat differential data to at least one disposal feed agent (e.g., one or more disposal feed agents 150 in Figure 1); inserting the threat differential data into the feed between the threat analysis agent (e.g., one or more threat analysis agents 130 in Figure 1) and the disposal feed agent (e.g., one or more disposal feed agents 150 in Figure 1); and / or storing the threat differential data in a location accessible to the disposal feed agent.

[0185] In step 765, one or more computer devices may determine and store event data indicating that an endpoint duplicate has been received. This event data may be the same as or similar to the event data described above in relation to Figure 1. The event data is related to Tables II and III. As described, it may be in a common format and / or common notation and may include attribute-value pairs. Event data may be stored in an event data repository (for example, event data repository 121 in Figure 1). This decision may also be made based on one or more variations described in relation to Figures 3A-3C.

[0186] In step 770, one or more computer devices may determine and store event data indicating that no change occurred for an endpoint. This event data may be the same as or similar to the event data described above in relation to Figure 1. The event data may be in a common format and / or common notation, as described in relation to Tables II and III, and may include attribute-value pairs. The event data may be stored in an event data repository (for example, event data repository 121 in Figure 1). This determination may also be made based on one or more variations described in relation to Figures 3A-3C.

[0187] After steps 760, 765, and 770, method 700 may terminate. Method 700 may be repeated each time a notification is received that endpoint data is available (for example, based on step 645 in Figure 6A).

[0188] Next, in the example 800 of the method in Figure 8, in step 805, one or more computer devices may be configured to monitor threat changes based on one or more threat analysis data repositories and / or monitoring criteria. Threat changes may be changes to one or more threat analysis data repositories (for example, one or more threat analysis data repositories 120 in Figure 1). The monitoring criteria may be the same as or similar to the monitoring criteria described above in relation to Figure 1 and one or more threat monitoring agents 140.

[0189] In step 810, one or more computer devices may monitor changes in the threat. For example, one or more computer devices may monitor changes in one or more threat analysis data repositories. This may be the same as or similar to the monitoring performed by one or more threat monitoring agents 140 in Figure 1. Alternatively, this decision may be made based on one or more variations (e.g., example 405 of the threat monitoring agent) described in relation to Figure 4.

[0190] In step 815, one or more computer devices may determine whether the monitoring criteria have been met. This determination may be based on changes in the monitored threat. This may be the same or similar manner in which one or more threat monitoring agents 140 in Figure 1 determine whether the monitoring criteria have been met. 1. Determine whether the monitoring criteria have been met. This determination may also be made based on one or more variations (e.g., examples 405, 420, 430 of threat monitoring agents) described in relation to Figure 4. If the monitoring criteria have been met, method 800 may proceed to step 820. If the monitoring criteria have not been met, method 800 may proceed to step 810 to continue monitoring changes in the threat.

[0191] In step 820, one or more computer devices may determine the threat differential data. The threat differential data may be the same as or similar to the threat differential data 145 in Figure 1. In this method, the threat differential data may be in a common format and / or common notation, as described in relation to Tables II and III, and may include attribute-value pairs.

[0192] In step 830, one or more computer devices may notify that threat differential data is available. Notifying that threat differential data is available may include: sending the threat differential data to at least one disposal feed agent (e.g., one or more disposal feed agents 150 in Figure 1); inserting the threat differential data into the feed between a threat monitoring agent (e.g., one or more threat monitoring agents 140 in Figure 1) and a disposal feed agent (e.g., one or more disposal feed agents 150 in Figure 1); and / or storing the threat differential data in a location accessible to the disposal feed agent.

[0193] Next, in Example 900 of the Method in Figure 9, Example 900 of the Method provides a further example of how disposal is determined for each endpoint. In step 905, one or more computer devices may be configured to construct a disposal feed based on feed criteria. The disposal feed may be the same as or similar to disposal feed 154 in Figure 1. The feed criteria may be the same as or similar to the feed criteria described in relation to Figure 1.

[0194] In step 910, one or more computer devices may receive threat differential data. This data may be received, for example, from a threat analysis agent (based on step 760 in Figure 7) or from a threat monitoring agent (based on step 830 in Figure 8).

[0195] In step 915, one or more computer devices may determine whether the feed criteria are met. This determination may be the same as or similar to the determination made by one or more disposal feed agents in Figure 1. Alternatively, this determination may be based on one or more variations described in relation to Figure 5A. If the feed criteria are met, method 900 may proceed to step 920. If the feed criteria are not met, method 900 may proceed to step 910 and wait to receive further threat differential data.

[0196] In step 920, one or more computer devices can construct a disposal feed. The structure may be the same as or similar to the structure of disposal feed 154 in Figure 1. In fact, a disposal feed can be constructed as a DNS feed, ACT feed, RPZ feed, or composite feed. Also, if the feed criteria are based on time-based exclusion, the disposal feed may be a temporary feed for time-based exclusion.

[0197] The construction of a disciplinary feed can be based on customer preferences related to the customers receiving the disciplinary feed. In this way, a disciplinary feed can be built specifically for a customer based on their preferences. For example, customer preferences may indicate certain exclusions that should be enforced or ignored, in which case a disciplinary feed that enforces or ignores those exclusions may be created based on the customer's preferences. Due to the dynamic nature of cyber threats and the differences in customer needs, disciplinary feeds can be built based on a variety of customer preferences.

[0198] In step 925, one or more computer devices may notify that a disposal feed is available. This may include sending the name of the disposal feed or other identifier of the disposal feed via a feed notification (e.g., feed notification 152) to inform the devices that a disposal feed is available.

[0199] In step 930, one or more computer devices may determine a penalty based on the threat differential data. This decision may be the same as or similar to the penalty decisions performed by one or more penalty feeds 150 in Figure 1. The penalty may be the same as or similar to the penalty described in relation to Figure 1. In this method, the penalty may indicate the level of threat to the endpoint as indicated by the threat differential data. Some examples of penalties may include monitoring network traffic associated with the endpoint, or blocking network traffic associated with the endpoint. This penalty decision may be endpoint-based (as opposed to provider feed-based and / or CTI provider-based, for example). Thus, the penalty may be for a single endpoint as indicated by the threat differential data.

[0200] In step 935, one or more computer devices may send a disposition via a disposition feed. Sending a disposition via a disposition feed may include inserting or including a disposition in the disposition feed. The transmission may depend on the type of disposition, such as some feeds sending dispositions in real time, while others send dispositions via APL. Dispositions may be sent in the same or similar manner as the disposition feed 154 in Figure 1 sends. Since dispositions may be determined on an endpoint-by-endpoint basis, the disposition feed 154 may send or include individual dispositions for each endpoint, indicated by various objects in the threat differential data, over time.

[0201] In step 940, one or more computer devices may receive additional threat differential data. This additional threat differential data may be received, for example, from a threat analysis agent (based on step 760 in Figure 7) or from a threat monitoring agent (based on step 830 in Figure 8).

[0202] In step 945, one or more computer devices may determine whether the feed criteria are met. This determination may be the same as or similar to the determination made by one or more disposal feed agents in Figure 1 regarding whether the feed criteria are met, as described in relation to Figure 1. Alternatively, this determination may be made based on one or more variations described in relation to Figure 5A. If the feed criteria are met, method 900 may proceed to step 930 to determine disposal based on additional threat differential data. If the feed criteria are not met, method 900 may proceed to step 950.

[0203] In step 950, one or more computer devices may decide whether to dismantle the disposal feed. Dismantling a disposal feed is based on the feed criteria, or other criteria used as criteria for dismantling a disposal feed (e.g., the type of disposal feed). For example, if the feed criteria are based on time-based exclusion, one or more computer devices may decide to dismantle the disposal feed based on the time-based conditions for time-based exclusion. In a specific example, if the time-based condition indicates a threshold window time for time-based exclusion, one or more computer devices may decide whether the threshold window time has expired. If the threshold window time has expired, one or more computer devices may decide to dismantle the disposal feed. If one or more computer devices decide to dismantle the disposal feed, the method may proceed to step 955. If one or more computer devices decide not to dismantle the disposal feed, method 900 may proceed to step 940 to await additional threat differential data.

[0204] In step 955, one or more computer devices may dismantle a disposal feed. Dismantling a disposal feed may depend on the type of feed (for example, how dismantling is performed may depend on whether the disposal feed is a DNS feed, ACT feed, RPZ feed, or composite feed). Furthermore, if the feed criteria are based on time-based exclusion, the disposal feed may be a temporary feed for time-based exclusion. In such variations, dismantling the disposal may reinstate the time-based exclusion (for example, network traffic related to an endpoint may be blocked).

[0205] In step 960, one or more computer devices may notify that a disposal feed is unavailable or has been dismantled. This may include transmitting, via a feed notification (e.g., feed notification 152), the name of the disposal feed, other identifiers of the disposal feed, and an indication of dismantling to notify the device that the disposal feed is unavailable or has been dismantled.

[0206] Having described the example of the computer environment 100 in Figure 1 and the machine learning models and / or rules / policies of the various agents 105, 130, 140, and 150 in Figure 1, along with examples of how these can be implemented by the various agents 105, 130, 140, and 150 in Figure 1, we will now describe examples of how the various agents determine the various dispositions that, once received, cause the device to filter network traffic. Figures 10A–10F show examples of flows in which a disposition is determined and the device filters network traffic based on that disposition. In these flow examples, common formats and / or common notations refer to the common formats and / or common notations described throughout the computer environment 100 in Figure 1, including Tables II and III. Also, for simplicity, the flow examples should be understood as operating in the computer environment 100 in Figure 1 or a similar computer environment. Therefore, Provider 1094, Agents 1095-1098, and Computer Device 1099 may be the same as or similar to Providers 101-1-101-X, Agents 105, 130, 140, 150, and Computer Device 170 in Figure 1, respectively. The flow examples in Figures 10A-10F are merely examples of how disposal is determined and / or how the devices filter network traffic based on disposal.

[0207] Starting with the example flow in Figure 10A, in item 1004, one or more providers 1094 may send CTI data (e.g., CTI data 103) indicating the first IOC to the endpoint www.xyz123.c. Based on the transmission of the CTI data, the data collection agent 1095 may, in item 1003, classify, receive, and store the CTI data (as part of one or more processes for receiving, storing, and / or processing the CTI data, for example, as described in relation to Figure 1).

[0208] Based on the CTI data, the data collection agent 1095 may determine endpoint data (for example, by performing a method similar to example 600 of the method in Figure 6A). In item 1005, the data collection agent 1095 may send the endpoint data to the threat analysis agent 1096. In item 1007, the threat analysis agent 1096 may ingest and analyze the endpoint data (for example, as part of one or more processes for ingesting and analyzing endpoint data, as described in relation to Figure 1).

[0209] Based on endpoint data and a determination that a change has occurred for the endpoint www.xyz123.c, the threat analysis agent 1096 may determine threat differential data (for example, by performing a method similar to Example 700 of the method in Figure 7). This threat differential data may indicate that the IOC is a first occurrence for the endpoint www.xyz123.c. In item 1009, the threat analysis agent 1096 may send the threat differential data to the disposal feed agent 1098. In item 1011, the disposal feed agent 1098 may determine the disposal feed. As a result, the disposal feed agent 1098 may determine the feed data (for example, by performing a method similar to Example 900 of the method in Figure 9). The feed data may indicate a disposal to monitor the endpoint www.xyz123.c. Monitoring may be directed because an insufficient IOC was received and / or the credibility of the received IOC fell below the threshold for blocking.

[0210] In item 1013, the disposal feed agent 1098 may transmit feed data to the computer device 1099 via the disposal feed. Based on the disposal monitoring the endpoint www.xyz123.c, the computer device 1099 may be configured in item 1035 to monitor network traffic related to the endpoint www.xyz123.c.

[0211] As can be seen from the comparison of the flow examples in Figure 10A and Figure 10B, endpoint disposal can be changed from monitoring to blocking based on additional IOCs for the endpoint.

[0212] Next, in the example flow of Figure 10B, in item 1021, one or more providers 1094 may send CTI data (www.xyz123.c) indicating the Xth IOC to the endpoint. Based on the transmission of the CTI data, the data collection agent 1095 may, in item 1023, classify, receive, and store the CTI data (for example, as part of one or more processes for receiving, storing, and / or processing the CTI data, as described in relation to Figure 1).

[0213] Based on the CTI data, the data collection agent 1095 may determine endpoint data (for example, by performing a method similar to example 600 of the method in Figure 6A). In item 1025, the data collection agent 1095 may send the endpoint data to the threat analysis agent 1096. In item 1027, the threat analysis agent 1096 may ingest and analyze the endpoint data (for example, as part of one or more processes for ingesting and analyzing endpoint data, as described in relation to Figure 1).

[0214] Based on endpoint data and a determination that a change has occurred for the endpoint www.xyz123.c, the threat analysis agent 1096 may determine threat differential data (based on performing a procedure similar to Example 700 of the method in Figure 7). This threat differential data may indicate that the IOC is the Xth occurrence for the endpoint www.xyz123.c. In item 1029, the threat analysis agent 1096 may send the threat differential data to the disposal feed agent 1098. In item 1031, the disposal feed agent 1098 may determine the disposal feed. As a result, the disposal feed agent 1098 may determine feed data (for example, based on performing a procedure similar to Example 900 of the method in Figure 9). The feed data may indicate a disposal that blocks the endpoint www.xyz123.c. A blocking disposal may be indicated because sufficient IOCs have been received and / or the reliability of the received IOCs exceeds the threshold for blocking.

[0215] In item 1033, the disposal feed agent 1098 may transmit feed data to the computer device 1099 via the disposal feed. Based on the disposal that blocks the endpoint www.xyz123.c, the computer device 1099 may be configured in item 1035 to block network traffic associated with the endpoint www.xyz123.c.

[0216] Next, in the example flow of Figure 10C, in item 1041, one or more providers 1094 may send CTI data indicating the Yth IOC to the endpoint www.tgb567.c. Based on the transmission of the CTI data, the data collection agent 1095 may, in item 1043, classify, receive, and store the CTI data (for example, as part of one or more processes for receiving, storing, and / or processing the CTI data, as described in relation to Figure 1).

[0217] Based on the CTI data, the data collection agent 1095 may determine endpoint data (for example, by performing a method similar to example 600 of the method in Figure 6A). In item 1045, the data collection agent 1095 may send the endpoint data to the threat analysis agent 1096. In item 1047, the threat analysis agent 1096 may ingest and analyze the endpoint data (for example, as part of one or more processes for ingesting and analyzing endpoint data, as described in relation to Figure 1).

[0218] Based on endpoint data and a determination that a change has occurred for the endpoint www.xyz123.c, the threat analysis agent 1096 may determine event data (by performing a method similar to example 700 of the method in Figure 7). This event data may indicate that the IOCs for the endpoint www.tgb567.c have changed (for example, a new IOC is added or a previous IOC is modified). In item 1049, the threat analysis agent 1096 may store the event data (for example, in the event data repository 121 in Figure 1). In item 1051, the threat monitoring agent 1097 may notice the change and receive the event data based on its storage and the continued performance of threat monitoring by the threat monitoring agent 1097.

[0219] Based on the event data, the threat monitoring agent 1097 may determine that the monitoring criteria have been met and determine the threat differential data (for example, by performing a method similar to Example 800 of the method in Figure 8). The threat differential data may indicate a range of CIDR addresses associated with the endpoint www.tgb567.c. In item 1053, the threat monitoring agent 1097 may send the threat differential data to the disposal feed agent 1098.

[0220] Based on the threat differential data, the disposition feed agent 1098 may determine the disposition feed in item 1055. As a result, the disposition feed agent 1098 may determine the feed data (for example, by performing a method similar to example 900 of the method in Figure 9). The feed data may indicate a disposition that blocks a range of CIDR addresses associated with the endpoint www.tgb567.c. The reason for indicating a blocking disposition may be that sufficient IOCs were received for the endpoint, and / or that the reliability of the received IOCs exceeded the threshold for blocking.

[0221] In item 1057, the disposal feed agent 1098 may transmit feed data to the computer device 1099 via the disposal feed. Based on a disposal that blocks a range of CIDR addresses associated with the endpoint www.tgb567.c, the computer device 1099 may be configured in item 1059 to block network traffic associated with a range of CIDR addresses associated with the endpoint www.tgb567.c.

[0222] Next, in the example flow in Figure 10D, in item 1061, computer device 1099 may receive a request to the endpoint (for example, based on a browser request to retrieve a web page associated with the endpoint). Based on this request, computer device 1099 may determine an endpoint query (for example, a DNS query to identify the endpoint). In item 1063, computer device 1099 may send an endpoint query. The endpoint query may be sent based on an API call to the threat analysis agent 1096. When this API call and / or endpoint query is received, it may cause the threat analysis agent 1096 to respond to the endpoint query, as if the endpoint query had been received as endpoint data identifying the requested endpoint. Figure 10D shows an example of a response that may occur based on an endpoint query. As shown in Figure 10D, in item 1065, the threat analysis agent 1096 ingests and analyzes the endpoint query and, as a result, may determine threat differential data indicating Y occurrences of IOCs for the endpoint indicated by the endpoint query (for example, based on performing a similar process to step 750 in Figure 7). In item 1067, the threat monitoring agent 1097 may send the threat differential data to the disposal feed agent 1098.

[0223] Based on the threat differential data, the disposition feed agent 1098 may determine the disposition feed in item 1069. As a result, the disposition feed agent 1098 may determine the feed data (for example, by performing a method similar to example 900 of the method in Figure 9). The feed data may indicate a disposition that blocks the endpoint indicated by the endpoint query. A disposition to block may be indicated because sufficient IOCs were received for the endpoint, and / or the reliability of the received IOCs exceeded the threshold for blocking.

[0224] In item 1071, the disposal feed agent 1098 may transmit feed data to the computer device 1099 via the disposal feed. Based on the disposal that blocks the endpoint, the computer device 1099 may be configured in item 1073 to block network traffic associated with the endpoint.

[0225] Next, in the example flow in Figure 10E, item 1074 indicates that one or more providers 1094 may send exclusion data indicating exclusion for one or more endpoints (for example, 12.20.18.0 / 24 as shown in Figure 10E). Based on the transmission of exclusion data, the data collection agent 1095 may classify, receive, and store the exclusion data in item 1075 (for example, as part of one or more processes for receiving, storing, and / or processing the exclusion data, as described in relation to Figure 1).

[0226] Based on the exclusion data, the data collection agent 1095 may determine endpoint data indicating one or more endpoints as excluded (for example, by performing a method similar to Example 600 of the method in Figure 6A). This endpoint data may be sent to one or more threat analysis agents 1096 in item 1076.

[0227] Based on endpoint data indicating one or more endpoints as excluded, the threat analysis agent 1096 may ingest and analyze the endpoint data in item 1077 (for example, by performing a method similar to example 700 of the method in Figure 7). Ingesting and analyzing the endpoint data may involve querying one or more repositories (for example, data repository 110 in Figure 1) for events and / or information that match or are associated with the endpoints indicated by the endpoint data. In this method, if the endpoint data indicates multiple endpoints as excluded, the threat analysis agent 1096 may query each endpoint (for example, each endpoint within the range indicated by 12.20.18.0 / 24). Based on the query results, the threat analysis agent 1096 may determine threat differential data for each of the one or more endpoints indicated by the excluded data. Accordingly, the threat analysis agent 1096 is shown to determine multiple threat differential data, one for each endpoint within the range indicated by 12.20.18.0 / 24, as shown in Figure 10E. In item 1078, the threat analysis agent 1096 may transmit multiple threat differential data to at least the disposal feed agent 1098.

[0228] The disposal feed agent 1098 may determine the disposal feed in item 1079 based on multiple threat differential data transmitted in 1078. As a result, the disposal feed agent 1098 may determine the feed data (for example, by performing a method similar to example 900 of the method in Figure 9). The feed data may indicate disposals based on endpoints indicated by multiple threat differential data. In this method, disposals do not include monitoring or blocking disposals for endpoints that are within the scope of excluded endpoints. In item 1080, the disposal feed agent 1098 may transmit the feed data to the computer device 1099 via the disposal feed. In this method, the disposal feed may enforce the exclusion of the scope of endpoints provided by the exclusion data.

[0229] Next, in the example flow of Figure 10F, the threat analysis agent 1096 is shown determining threat differential data indicating the occurrence of the first IOC for endpoint www.xyz123.c (based on performing a method similar to Example 700 of the Method in Figure 7). This threat differential data may also be determined based on the threat analysis agent 1096 determining the occurrence of a change in endpoint www.xyz123.c (similar to the process described above in relation to items 1001-1009 in Figure 10A, for example). In item 1081, the threat analysis agent 1096 may send the threat differential data to the feed disposal agent 1098. In item 1082, the disposal feed agent 1098 may determine the disposal feed. This determination of the disposal feed allows the disposal feed agent 1098 to determine the feed data (similar to performing a method similar to Example 900 of the Method in Figure 9, for example). The feed data may indicate a disposal monitoring endpoint www.xyz123.c. Monitoring may be initiated because insufficient IOCs were received and / or the credibility of the received IOCs fell below the threshold for blocking. As also shown in Figure 10F, this disposition feed decision may allow the disposition feed agent 1098 to determine request data (for example, as described in relation to one or more disposition agents 150 and one or more threat monitoring agents 140 in Figure 1). This request data may indicate that the disposition feed agent is requesting more threat context for the endpoint www.xyz123.c. The request data may be determined based on a disposition to monitor the endpoint (for example, the request data may be determined based on a disposition to monitor the endpoint that was first sent via the disposition feed in item 1083) and / or the request data may be determined based on the credibility of the received IOCs being above the threshold for requesting additional threat context.

[0230] In item 1085, the disposal feed agent 1098 may send request data so that it may eventually be received by the threat monitoring agent 1097. For example, request data may be sent by the disposal feed agent 1098 to be stored in one of the data repositories monitored by the threat monitoring agent 1097 (for example, one of the data repositories 110 in Figure 1, such as event data 121). The threat monitoring agent 1097 may receive the request data after it has been stored in the data repository, based on the process of threat monitoring. In another example, the threat monitoring agent 1097 and the disposal feed agent 1098 may be configured to communicate with each other. In this way, the disposal feed agent 1098 may send request data directly to the threat monitoring agent 1097.

[0231] Based on the request data, the threat monitoring agent 1097 may, as part of performing processing for threat monitoring in item 1086, determine that the monitoring criteria are met based on the request data and may determine threat differential data based on the request data (for example, by performing a method similar to Example 800 of the method in Figure 8). The threat differential data may indicate additional threat context related to the endpoint (for example, newly received IOCs, additional intelligence data collected or related to the endpoint www.xyz123.c, a timestamp indicating when the last IOC for the endpoint was received, or an indication of a timeout value for raising the penalty from monitoring to blocking or lowering the penalty from monitoring). In item 1087, the threat monitoring agent 1097 may send the threat differential data to the disposal feed agent 1098. In this method, the threat monitoring agent 1097 may cause the disposal feed agent 1098 to redetermine the penalty for the endpoint based on the requested additional threat context.

[0232] In this way, the disposition feed agent 1098 may determine a disposition feed in item 1088 based on threat differential data indicating additional threat context. As a result, the disposition feed agent 1098 may determine feed data (for example, by performing a method similar to example 900 of the method in Figure 9). The feed data may indicate a disposition to block the endpoint www.xyz123.c. The reason for indicating a blocking disposition may be that, compared to the previous decision in item 1082, the additional threat context provided sufficient threat context to block the endpoint, and / or the additional threat context increased confidence and caused it to exceed the threshold for blocking.

[0233] As part of the disposition feed determination in item 1088, the disposition feed agent 1098 may or may not re-request additional threat context for the endpoint. For example, if additional threat context changes the disposition (e.g., from monitoring to blocking), the disposition feed agent 1098 may not re-request additional threat context. As another example, if additional threat context indicates that there has been a change related to the endpoint since the last disposition feed determination in item 1082 (e.g., some new event related to the endpoint has occurred), the disposition feed agent 1098 may re-request additional threat context by determining additional request data for the endpoint. As yet another example, if additional threat context indicates that there have been no changes related to the endpoint since the last disposition feed determination in item 1082 (e.g., no new events related to the endpoint have occurred), the disposition feed agent 1098 may not re-request additional threat context.

[0234] The disposal feed agent 1098 may transmit feed data to the computer device 1099 via the disposal feed, as described in item 1089. The computer device 1099 may be configured to block network traffic associated with the endpoint www.xyz123.c based on a disposal that blocks the endpoint, as described in item 1090.

[0235] Having described examples of how various agents may determine the various dispositions that cause a device to filter network traffic, we will now describe examples of how time-based exclusion is performed. Figures 11A-11B show examples of flows in which time-based exclusion is performed. In these example flows, common formats and / or common notations refer to the common formats and / or common notations described throughout computer environment 100 in Figure 1, including Tables II and III. Also, for simplicity, the example flows should be understood as operating in computer environment 100 in Figure 1 or a similar computer environment. Thus, exclusion provider 1193, CTI provider 1194, and agents 1195-1198 may be the same as or similar to exclusion providers 104-1 to 104-Y, CTI providers 101-1 to 101-X, and agents 105, 130, 140, and 150 in Figure 1, respectively. The example flows in Figures 11A-11B are just examples of how time-based exclusion may be performed.

[0236] Starting with the example flow in Figure 1lA, one or more exclusion providers 1193 may send exclusion data in item 1101 that indicates a time-based exclusion for the endpoint www.typ345.c. A time-based exclusion may also indicate a threshold window period during which dispositions for the endpoint www.typ345.c may be sent via a temporary disposition feed. Based on the transmission of exclusion data, the data collection agent 1195 may receive and store the exclusion data in item 1103 (for example, as part of one or more processes for receiving, storing, and / or processing the exclusion data, as described in relation to Figure 1).

[0237] Based on the exclusion data, the data collection agent 1095 may determine endpoint data indicating the endpoint www.typ345.c as a time-based exclusion (for example, by performing a method similar to Example 600 of the method in Figure 6A). The endpoint data may indicate a threshold window time for time-based exclusion. This endpoint data may ultimately lead to the event data indicating the endpoint as a time-based exclusion being stored in the event data repository. This process is represented by item 1105. Based on the storage of the event data, the threat monitoring agent 1197 may perform threat monitoring and become aware of the changes caused by the storage of the event data in item 1107.

[0238] Based on event data indicating endpoint www.typ345.c as a time-based exclusion, threat monitoring agent 1197 may determine that the monitoring criteria are met (for example, by performing a method similar to example 800 of the method in Figure 8) and determine threat differential data. The threat differential data may indicate endpoint www.typ345.c as a time-based exclusion. In item 1109, threat monitoring agent 1197 may send the threat differential data to each disposal feed agent 1198 currently operating in the computer environment.

[0239] Based on the threat differential data, at least one of the currently operating disposition feed agents 1198 in the computer environment may determine the disposition feed in item 1111. As a result, at least one disposition feed agent may change what dispositions are sent to or included in its disposition feed. In fact, as shown in the example flow in Figure 11A, at least one disposition feed agent may, based on time-based exclusions, ensure that the disposition feed does not include a disposition for endpoint www.typ345.c. In this way, at least one disposition feed agent may implement time-based exclusions based on the receipt of threat differential data.

[0240] Furthermore, another disposal feed agent among those currently operating in the computer environment may, based on time-based exclusions, determine new feed criteria in 1113 based on threat differential data. These new feed criteria may be for temporary feeds that include disposals during the window time of the threshold for time-based exclusions. Based on the new feed criteria, a new disposal feed agent may be configured to operate in the computer environment. In particular, the new disposal feed agent may be configured to determine whether the new feed criteria are met based on the received threat differential data. The determination of the new feed criteria and the configuration of the new disposal feed may be performed based on one or more variations described in relation to Figure 5B.

[0241] The example of time-based exclusion continues in the flow example of Figure 1lB. At some point after exclusion data indicating time-based exclusion has been received by the data collection agent 1195, one or more CTI providers 1194 may, in item 1141, send CTI data indicating the Yth IOC of the endpoint www.typ345.c. Based on the transmission of the CTI data, the data collection agent 1195 may, in item 1143, receive and store the CTI data (for example, as part of one or more processes for receiving, storing, and / or processing the CTI data, as described in relation to Figure 1).

[0242] Based on the CTI data, the data collection agent 1195 may determine endpoint data (for example, by performing a method similar to example 600 of the method in Figure 6A). In item 1145, the data collection agent 1195 may send the endpoint data to the threat analysis agent 1196. In item 1147, the threat analysis agent 1196 may ingest and analyze the endpoint data (for example, as part of one or more processes for ingesting and analyzing endpoint data, as described in relation to Figure 1).

[0243] The threat analysis agent 1196 may determine threat differential data based on endpoint data and a determination that a change has occurred for the endpoint www.typ345.c (for example, by performing a procedure similar to Example 700 of the method in Figure 7). This threat differential data may indicate that the Yth IOC has occurred for the endpoint www.typ345.c. In item 1149, the threat analysis agent 1196 may send the threat differential data to each disposal feed agent 1198 currently operating in the computer environment.

[0244] Based on threat differential data, at least one of the currently operating disposition feed agents 1198 in the computer environment may determine the disposition feed in item 1155. In particular, this disposition feed agent may be a new disposition feed agent configured for the new feed criteria in item 1113. This new disposition feed agent may determine whether the IOC indicated by the threat differential data is within the threshold window time for time-based exclusion, and based on that determination, may include or exclude the disposition for endpoint www.typ345.c in the temporary disposition feed. In fact, as shown in the example flow in Figure 1lB, the new disposition feed agent may include the disposition for endpoint www.typ345.c in the temporary feed based on the threshold window time and the threat differential data. In this way, the new disposition feed agent may temporarily allow the disposition to be sent based on time-based exclusion. While the temporary feed is being built, the disposition feed agent for that disposition feed may continue to prevent the disposition feed from including dispositions for endpoints. In fact, while the temporary feed is being built, the temporary feed may be the only disposition feed that can include dispositions for endpoints. In other words, while a temporary feed is being built, all other disposal feeds may be prevented from containing disposals for endpoints. After the temporary feed is dismantled (for example, based on the expiration of a threshold window time), all disposal feeds may be prevented from containing disposals for endpoints (for example, until the criteria for a threshold window time are met and the temporary feed is rebuilt).

[0245] In some cases, a temporary feed may contain dispositions for multiple endpoints subject to time-based exclusions. A temporary feed may remain constructed as long as at least one time-based exclusion remains that has not expired. Furthermore, time-based exclusions may have different expiration dates (for example, the threshold window times for two time-based exclusions may differ from each other). Thus, a temporary feed may contain dispositions for different sets of endpoints over time (for example, at time tl, a temporary feed may contain dispositions for a set of endpoints represented by {endpoint A, endpoint B, endpoint C}, but at time t2, the temporary feed may contain dispositions for a second set of endpoints represented by {endpoint B, endpoint C} because the time-based exclusion for endpoint A has expired).

[0246] Referring to Figure 12-14, we will discuss how to address cyber threats through the potential impact of blocking potentially legitimate network traffic. Network traffic can be potentially legitimate if, for example, one or more IOCs shown in received CTI data are false positives. Considering the impact of blocking potentially legitimate network traffic represents a new way of thinking about cyber threats, beyond the severity of the potential threat or the reliability of the received cyber threat intelligence. As will be explained in more detail below, this new approach to addressing cyber threats introduces the concept of shimmability. The determination of whether traffic is shimmable involves evaluating the adverse impact on the entity's operations (such as business operations) that would result from blocking network traffic that is not known to be malicious, nor is it known to be harmless, but could be legitimate network traffic for the customer. Such traffic may be determined to be shimmable, even if it is legitimate, if blocking that traffic would not adversely impact the entity's operations. Thus, the concept of shimmability recognizes the relationship between exposure and risk. Allowing a large amount of traffic to an entity's network exposes that entity to greater risk, while reducing the traffic allowed to that entity's network reduces the entity's overall risk profile. Identifying traffic that can be blocked without disrupting the entity's operations can reduce the entity's overall risk profile without requiring reliable cyber threat intelligence in all scenarios.

[0247] For many entities, network traffic can be classified into three categories. The first (often larger) portion of an entity's network traffic can be identified as legitimate (e.g., harmless) network traffic. The second (often smaller) portion of an entity's network traffic can be identified as rogue (e.g., malicious) network traffic. The third (also often small) portion of an entity's traffic may not be identifiable as either legitimate or rogue network traffic (with a favorable level of certainty). As an example of a hypothetical scenario, 98% of an entity's network traffic may be known or determined to be legitimate, 1% of that entity's network traffic may be known or determined to be rogue, and the remaining 1% of that entity's network traffic may not be definitively determined as legitimate or rogue. Endpoints associated with such traffic can similarly be classified as known malicious endpoints (e.g., known to be associated with rogue network traffic) or known harmless endpoints (e.g., known to be associated with legitimate network traffic). For ease of reference, endpoints that are not known to be malicious, and endpoints that are not known to be non-malicious, are sometimes called ambiguous endpoints (or, additionally or alternatively, non-conclusive endpoints, uncertain endpoints, suspicious endpoints, indeterminate endpoints, etc.). Legitimate network traffic may be called "whitelisted" traffic because the associated endpoint is a known non-malicious endpoint, and therefore can be added to the "whitelist" (allow list) of entities for permitted network traffic.Similarly, malicious network traffic is called "blacklisted" traffic because the associated endpoint is a known malicious endpoint, and therefore may be added to a "blacklist" (blocklist) of entities of network traffic that are blocked. The remaining traffic is sometimes called "graylisted" or "grayzone" traffic because the associated endpoint falls in the "gray zone" between known non-malicious endpoints of "whitelisted" traffic and known malicious endpoints of "blacklisted" traffic.

[0248] CTI data can be used to identify certain endpoints as either known non-malicious endpoints or known non-malicious endpoints, as described herein, and to determine appropriate action for such endpoints (e.g., permit or permit / monitor traffic from known non-malicious endpoints and block traffic from known malicious endpoints), as described herein. However, in the case of “graylist” traffic, the received CTI data may be insufficient to draw a definitive conclusion about the malicious or non-malicious nature of the endpoint if the associated endpoint falls into the “gray area” of ambiguous malice. For example, the received CTI data may not contain information about the endpoint associated with the “graylist” traffic, or the received CTI data for that endpoint may be unreliable, failing to meet the reliability threshold and thus not reaching an actionable intelligence level, and / or the CTI data may not contain a threshold amount of IOCs for that endpoint. As a result, an entity may apply default rules (e.g., default block rules or default permit rules) to “graylist” traffic associated with ambiguous endpoints. While default permission rules may align with the ideal of free communication over the internet, they can potentially expand an entity's risk profile, as mentioned above. Conversely, default blocking rules may reduce an entity's risk profile, but at the cost of potentially disrupting its operations. By considering the possibility of shielding "graylist" traffic, entities can assess the potential impact that blocking such traffic would have on their operations.

[0249] As will be explained in more detail below, the impact status may be determined based on the potential impact of blocking legitimate network traffic between the ambiguous endpoint and the entity's network. The impact status may be used to determine an alternative disposition for the ambiguous endpoint, different from the default disposition that applies. The impact status may be considered in conjunction with the threat context to determine the alternative disposition. For example, for an ambiguous endpoint, a composite status may be determined based on both the impact status and the threat status. For ease of reference, the composite status may be referred to as the composite occludability status or simply the occludability status. The threat status of an endpoint may be determined by a threat analysis agent and / or a threat monitoring agent, as described herein. The threat status may be based on received CTI data that does not contain information about the ambiguous endpoint, or on unreliable CTI data received about the ambiguous endpoint. For example, the threat status may indicate the reliability of the received CTI data about the endpoint and / or the reliability of one or more IOCs indicated in the received CTI data. The reliability of an IOC may depend, for example, on the number of CTI providers that have certified the same IOC, or on one or more reliabilitys that have been assigned to the IOC by one or more CTI providers. For example, CTI data / IOCs may be relatively unreliable if they are received from a relatively small number of CTI providers (e.g., one or fewer than a threshold number). Therefore, the concept of occlusion can also facilitate or improve unreliable CTI data into actionable intelligence. Furthermore, an entity's network may be, or contain, one or more physical networks and / or one or more logical networks.

[0250] As will be explained in more detail below, machine learning models may be used to help assess the potential impact of blocking legitimate network traffic associated with ambiguous endpoints. The impact status is determined based on the entity's historical network traffic activity (e.g., network traffic patterns). The machine learning model is trained to assist in processing the historical network traffic data for each entity. Once trained, the machine learning model can be used as part of the process of determining the impact status of ambiguous endpoints. The impact status may be based on an assessment of the entity's own historical network traffic activity and the entity's resources (human resources, organizational resources, computing resources, etc.) associated with such network traffic. When determining the impact status of ambiguous endpoints, the historical network activity and / or resources of one or more other entities may also be evaluated.

[0251] Figure 12 shows a block diagram of an exemplary computer environment 1200, which may be configured to provide impact status based on the potential impact of blocking legitimate network traffic between an entity's network and one or more ambiguous endpoints. Although not shown in Figure 12, computer environment 1200 may include one or more components of computer environment 100 shown in Figure 1, which may be configured to detect cyber threats based on threat context and / or changes in threats. For example, computer environment 1200 may include a data repository (e.g., data repository 110) and various agents (e.g., agents 104, 130, 140, 150) working together to detect cyber threats based on threat context and / or changes in threats, as shown in Figure 1. In brief, computer environment 1200 is shown as including a data repository 1205 and various agents 1210, 1215 working together to determine alternative dispositions for ambiguous endpoints based on the potential impact of blocking legitimate network traffic to and from endpoints. The data repository 1205 is implemented in one or more storage devices and includes the impact analysis data repository 1220. The data repository 1205 may also include a provider feed repository (e.g., provider feed repository 112) and / or a threat analysis data repository (e.g., threat analysis data repository 120), as shown in Figure 1. The various agents 1210, 1215 may be located locally or remotely from one another. Furthermore, some or all of the data repository 1205 and the various agents 1210, 1215 may be implemented using cloud computing services (for example, the data repository 1205 may be implemented using cloud computing services, and the various agents 1210, 1215 may be implemented on one or more computer devices that communicate with the cloud computing services over one or more networks). The computer environment 1200, although not shown in Figure 12, may include additional components, such as additional data repositories, additional computer devices, and / or additional networks.

[0252] In relation to the decision on alternative disposal of ambiguous endpoints, the exemplary computer environment 1200 shows the impact analysis data repository 1220 as containing certain types of data repositories, such as the network traffic data repository 1225 and the resource data repository 1230. The network traffic data repository 1225 may store historical network traffic data 1235 (e.g., records of network traffic) for one or more entities. Historical network traffic data 1235 may include network traffic information such as endpoint identifiers, source IP addresses, destination IP addresses, source ports, destination ports, protocols, and / or other information that characterizes, or otherwise associates, network traffic between one or more networks and one or more endpoints. Historical network traffic data 1235 may include network traffic information for both inbound and outbound traffic in the entities' networks. The resource data repository 1230 may store resource data 1240 that provides information about resources related to network traffic. Resources may include computer resources, as well as other types of resources related to entities. Computer resources may include devices configured to communicate over a network, such as desktop computers, laptop computers, mobile computer devices (e.g., mobile phones, tablet computers), rack-mount computer devices (e.g., servers), Internet of Things (IoT) devices, monitoring devices (e.g., sensors, cameras), and home appliances (e.g., refrigerators, HVAC devices). Computer resources also include executable files such as software applications, computer programs, and services. Other types of resources associated with an entity include non-computer resources such as individuals associated with the entity (e.g., network users), user groups, departments of the entity (e.g., business units), and offices of the entity (e.g., geographical offices of the entity).Resources may also be network addresses (for example, the IP address of a computer device connected to a network). Resource data 1240 may also include an indication of the importance (for example, priority) associated with the resource. As further described below, the impact analysis agent 1210 uses the network traffic data 1235 and resource data 1240 to determine the impact status of the endpoints and provides impact data 1245 with the determined impact status to the disposal feed agent 1215, which uses the received impact data 1245 to determine alternative disposals for ambiguous endpoints that would otherwise receive default disposals.

[0253] Furthermore, in relation to the decision on alternative disposition for ambiguous endpoints, the exemplary computer environment 1200 is shown as having various agents 1210, 1215 that send and / or receive data to or for the data repository 1210, receive specific input data, and provide specific output data. In fact, one or more impact analysis agents 1210 are shown to receive network traffic data 1235, resource data 1240, and endpoint data 1270, and to perform one or more processes to ingest and analyze the network traffic data, resource data, and endpoint data, and to output impact data 1245. One or more disposal feed agents 1215 are shown to receive impact data (e.g., impact data 1245 from one or more impact analysis agents 1210) and threat data received from threat analysis agents (e.g., threat analysis agent 130); perform one or more processes to determine a disposal feed; and output feed notifications 1250 and disposal feeds 1255 (e.g., disposal feeds 1 to Z), some of which may be received by computer equipment 1260 via network 1265. The illustrated agents 1210, 1215, processes, input data, and output data are provided as examples that may be used when determining impact status and alternative disposals for ambiguous endpoints. In some variations, compared to the computer environment example shown in Figure 12, the agents are different, added, or fewer; the processes are different, added, or fewer; the types of input data are different, added, or fewer; and the types of output data are different, added, or fewer.

[0254] Furthermore, in relation to the determination of alternative disposals for ambiguous endpoints, in the exemplary computer environment 1200, various agents 1210, 1215 are shown as including machine learning models and / or rules / policies. For example, as shown in Figure 12, one or more impact analysis agents 1210 may include one or more machine learning models 1210-1 and rules / policies 1210-2. One or more disposal feed agents 1215 may include machine learning model 1215-1 and rules / policies 1215-2. The machine learning models and rules / policies shown are provided as examples that may be used by various agents 1210, 1215 in relation to the determination of impact status and / or alternative disposals for ambiguous endpoints. In some variations, the machine learning models and / or rules / policies are different, added, or fewer compared to those shown in Figure 12.

[0255] The following description will begin with an impact analysis agent 1210 that receives endpoint data 1270, in order to provide a more detailed explanation of an exemplary computer environment 1200 and how the exemplary computer environment 1200 determines alternative disposal for ambiguous endpoints. The endpoint data 1270 may contain information about the ambiguous endpoint (e.g., endpoint identifier). The impact analysis agent 1210 may receive the endpoint data 1270 from a threat analysis agent (e.g., threat analysis agent 130). The threat analysis agent may be configured to provide the endpoint data 1270 to the impact analysis agent 1210 based, for example, on a determination that the endpoint is an ambiguous endpoint and / or on a determination that default rules apply to the endpoint. The threat analysis agent may provide the endpoint data 1270 to the impact analysis agent 1210 periodically or irregularly and / or on request / request, in response to such a determination (e.g., immediately, in real time) (as a result of such a determination). Endpoint data 1270 may include information for one ambiguous endpoint or multiple ambiguous endpoints (for example, for a collective determination of their respective impact statuses). Endpoint data 1270 may be provided by one or more threat analysis agents described herein, including threat analysis agents 130, 1096, and 1196, which are described in relation to Figures 1, 10, and 11. Endpoint data 1270 may indicate the threat context determined for the endpoint (for example, by threat analysis agents 130, 1096, and 1196) and / or may include available threat data for the endpoint or threat data determined (for example, by threat analysis agents 130, 1096, and 1196). Threat data may include additional contextual information retrieved by the threat analysis agents (for example, by expanding the degree of isolation as described herein).In this way, the threat analysis described herein (for example, by threat analysis agents 130, 1096, and 1196) can be employed as part of a process to determine the occludability of network traffic.

[0256] Different analyses may be performed in relation to determining the impact status of ambiguous endpoints. As mentioned above, occlusion attempts to assess whether blocking potentially legitimate traffic to and / or from an ambiguous endpoint would have a detrimental impact on the entity's operations (e.g., business operations). Therefore, whether blocking potentially legitimate traffic to and / or from an ambiguous endpoint would adversely impact a particular entity's network may depend on the nature (volume, frequency, etc.) of the entity's network traffic to that ambiguous endpoint, and / or the resources associated with that network traffic. Different criteria, or combinations of different criteria, may influence the impact status and, consequently, the occlusion potential when determining whether blocking potentially legitimate traffic would have a detrimental impact on the entity's network traffic, resources, and overall operations. Therefore, criteria for determining the impact status may include the volume of traffic to and / or from the ambiguous endpoint (e.g., volume of network communication, size of data transfer, volume and / or size over a specified period), the frequency of network communication to and / or from the ambiguous endpoint, and / or the amount of potentially affected resources related to network traffic from the ambiguous endpoint. As mentioned above, resources may include both computing resources and non-computing resources (users, user groups, etc.).Therefore, the impact analysis agent 1210 may determine the impact status based, for example, by comparing a traffic volume threshold with the historical traffic volume between the ambiguous endpoint to determine whether the historical traffic volume meets the traffic volume threshold, by comparing a traffic frequency threshold with the historical traffic frequency between the ambiguous endpoint to determine whether the historical traffic volume meets the traffic frequency threshold, and / or by comparing a resource threshold with the amount of resources associated with network traffic between the ambiguous endpoint to determine whether the amount of those resources meets the resource threshold. Depending on the method of execution, the threshold may be met if the traffic volume, traffic frequency, or amount of affected resources are above or below the threshold, or if they are below the threshold. Other criteria for determining the impact status may include the type of resource (for example, the type of user associated with network traffic, the type of computer equipment associated with network traffic). Therefore, the impact analysis agent 1210 may determine the type of user associated with network traffic between the ambiguous endpoint and determine the impact status based on the determined user type, and / or determine the type of computer resource associated with network traffic between the ambiguous endpoint and determine the impact status based on the determined computer resource type. User types can be based on job title, function, set of permissions, inclusion in user groups, department assignments, office assignments, etc. Criteria for determining impact status may include time periods related to network traffic between the entity's network and the ambiguous endpoint. Thus, impact analysis agent 1210 may determine impact status based on the time the entity's network sent network traffic to and / or received network traffic from the ambiguous endpoint. Criteria for determining impact status may include the severity (priority, etc.) of potentially affected resources.The impact analysis agent 1210 may then determine the impact status based on whether the severity threshold is met by comparing the severity threshold with the severity of the resources associated with network traffic to the ambiguous endpoint. The severity of potentially affected resources may be indicated using numerical values ​​(e.g., 1 to 10) and / or textual values ​​(e.g., "High", "Medium", "Low"). The severity threshold may then be used as a numerical threshold (e.g., greater than 5 (5 or more) or less than 5 (5 or less)) or as a textual threshold (e.g., "Medium" or "High" (higher than "Medium", lower than "High")).

[0257] Impact status can be implemented in various ways. For example, impact status may indicate (e.g., include) an impact score. The impact score may be a numerical or text value, as described herein. Impact status may also include an indication of whether the impact score meets an impact score threshold. Thus, impact analysis agent 1210 may determine an impact score (e.g., based on the criteria described herein), compare the impact score to an impact score threshold, and include an indication in the impact status of whether the impact score meets the impact score threshold. As another example, impact status may indicate only an impact score, and disposal feed agent 1215 may compare the impact score to an impact score threshold to determine whether the impact score meets the impact score threshold. The impact score threshold can similarly be implemented as a numerical or text threshold, as described herein. The impact score threshold can also be a configurable setting or parameter for individual entities. For example, an entity that is relatively risk-averse may set a relatively low impact score threshold, resulting in an alternative measure that blocks relatively more potentially legitimate network traffic to and from ambiguous endpoints. Conversely, an entity that is relatively risk-tolerant may set a relatively high impact score, resulting in an alternative measure that blocks relatively less potentially legitimate network traffic to and from ambiguous endpoints. In this method, determining the impact status and the corresponding alternative measure can be customized and / or tailored for specific entities (e.g., according to their preferences, risk tolerance, etc.). More generally, the alternative measure may be determined specifically for a particular entity, or it may be a global alternative measure determined for all or more entities.

[0258] Machine learning models (e.g., machine learning models 1210-1, 1215-1) and / or rules / policies (e.g., rules / policies 1210-2, 1215-2) may be used to determine impact status. Historical network traffic data (e.g., network traffic data 1235) and / or resource data (e.g., resource data 1240) may be used as training data and provided as input to one or more machine learning models 1210-1 of the impact analysis agent 1210. Once trained, the machine learning models may provide impact status for ambiguous endpoints as output. Historical network traffic data used to train machine learning models 1210-1 may include historical network traffic data and / or resource data relating to a single entity or multiple entities. In this way, the impact analysis agent 1210-1 may determine the impact of blocking potentially legitimate traffic based only on historical network traffic in the network of a given entity and / or historical network traffic across multiple entities. Depending on the scenario, historical network traffic across multiple entities may be used to determine the impact status of a given entity in the same or similar way as a single entity.

[0259] For example, if a given entity's network traffic data indicates that network traffic to and from an ambiguous endpoint is infrequent, constitutes a relatively small portion of the entity's overall network traffic, is associated with relatively few resources, or is associated with low-priority resources, the impact analysis agent 1210 may determine an impact status indicating that blocking potentially legitimate network traffic to and from that ambiguous endpoint would have a relatively low impact on the entity's operations. Such network traffic may, for convenience, be referred to as low-impact network traffic. Network traffic to and from an ambiguous endpoint may include, for example, communication with a new grocery store website that a curious employee is interested in, communication with news-focused websites or network-enabled devices that send beacons to manufacturer servers to check for and receive updates, in addition to unrecognized hosts exploring the entity's network (e.g., potential vulnerabilities) (these are legitimate network traffic for that ambiguous endpoint), communication with geographical regions with which the entity has no reason to communicate (e.g., hostile countries), and network communications that only occur outside of normal business hours (these are malicious network traffic for that ambiguous endpoint). As this example illustrates, considering the nature of potentially legitimate network traffic, it is possible to block traffic to and from ambiguous endpoints without disrupting the entity's operations, while reducing the risk of exposure to potentially malicious network traffic. Low-impact network traffic may also include network traffic that is relatively frequent, relatively high in volume, and / or related to a relatively large number of resources. Examples of such low-impact network traffic include communication with sports-themed websites (e.g., during playoffs or tournaments) and communication with social media services.

[0260] Among network traffic to and from ambiguous endpoints, there may be some that are relatively infrequent, relatively small in volume, and / or associated with relatively few resources, but which, even if blocked, could have a significant impact on the entity's operations. The impact analysis agent 1210 may then determine an impact status indicating that blocking potentially legitimate network traffic to and from ambiguous endpoints would have a relatively significant impact on the entity's operations. For example, the impact status could be determined using the importance of the resources associated with the network traffic to and from ambiguous endpoints. For instance, such traffic may include network traffic associated with relatively high-priority users (e.g., CEO, CTO, etc.), relatively high-priority user groups (e.g., network administrators), relatively high-priority computing resources (e.g., business-critical machinery and software applications, etc.), and other relatively high-priority resources, as described herein.

[0261] Among the network traffic to and from ambiguous endpoints, there is network traffic that is relatively infrequent and / or relatively new, yet has a significant impact. For example, such traffic may include network traffic related to a new service (e.g., hosting service, messaging service, etc.) that has recently been implemented (e.g., installed, deployed, or subscribed to) by the entity. This example illustrates how the impact analysis agent 1210 uses historical network traffic data related to different entities to determine the impact status of a given entity's communication with an ambiguous endpoint. An entity may have never communicated with that ambiguous endpoint in the past (or only infrequently), but historical network traffic data from other entities implementing the same service may indicate that network communication with that endpoint is common and frequent due to the use of the service. In this case, the impact analysis agent 1210 may determine that the impact status for that ambiguous endpoint is relatively significant.

[0262] Next, we provide examples of how the impact status may be based on whether traffic to and from an ambiguous endpoint is abnormal. Network traffic may be abnormal due to a given entity, or one or more resources associated with that entity, or the entity as a whole. For example, network traffic may be characterized as abnormal if it deviates from a baseline network traffic pattern (such as that shown by historical network traffic data, as described herein). Deviations in network traffic patterns may occur, for example, across multiple entities, for a single entity, for one or more groups (e.g., one or more users, user groups, one or more departments, one or more geographical locations, etc.), and / or for a specific computer resource (e.g., a program, application, device, machine, service, etc.). Abnormal network traffic includes not only legitimate network traffic but also malicious network traffic. For example, widespread abnormal and malicious network traffic across multiple entities may be related to an attack on multiple systems and / or networks from one or more malicious endpoints. Localized malicious and abnormal network traffic may be related to an attack on a single system and / or network from one or more malicious endpoints. As a further example, widespread, unusual, and legitimate network traffic may also be related to new or newly popular services that are seeing increased use across multiple entities (such as social media services or content delivery services).Localized anomalies and legitimate network traffic may relate to, for example, new or newly utilized computer resources (e.g., newly installed machines or devices on the network; new or newly utilized programs, applications, and / or services on the entity's network; new or newly utilized websites inside or outside the entity's network, such as a new time-stamping website). Network traffic may be anomaly for one group but not for another. For example, one user group of an entity may regularly use a particular computer resource (e.g., an accounting program, an instant messaging program), while another user group of the entity does not regularly use that computer resource (e.g., due to preference, access restrictions, etc.). In this example, the threat context of the network traffic related to that computer resource may be very small when considering only the potential threat from the anomaly network traffic to and / or from the latter user group. Even if blocking such anomaly network traffic to the latter user group does not adversely affect that user group, the entity can reduce its overall risk profile by considering the anomaly nature of such traffic when determining the impact status of such network traffic. In this method, the entity can block the pathways of potential attacks that originate through legitimate computer resources and rely on resource exploitation or vulnerabilities for malicious activity (i.e., using legitimate computer resources for illegal purposes).

[0263] The machine learning model 1210-1 in the impact analysis agent 1210 may include different types of models and perform different types of modeling. For example, the machine learning model 1210-1 may include single-entity models, where each single-entity model is specific to a particular entity, and / or multi-entity models, where modeling is performed for a set of multiple entities. The machine learning model 1210-1 in the impact analysis agent 1210 may also include single-resource models, where each single-resource model is specific to a particular resource (e.g., the resources described herein), and / or multi-resource models, where modeling is performed for a combination of resources (e.g., any combination of two or more resources). The machine learning model 1210-1 in the impact analysis agent 1210 may also include models that perform modeling based on a combination of network traffic data and resource data. By modeling in this way, an indicator may be provided of whether network traffic to and from ambiguous endpoints is abnormal. This method may, if necessary, take into account past network data at other entities to provide an indicator of whether the anomalous network traffic to the ambiguous endpoint is due to the anomalousness of that entity alone, or to the anomalousness of multiple entities, when determining the impact status and the corresponding alternative action. The criteria for determining whether network traffic is anomalous may include the same or similar criteria as those for determining the impact status described herein (e.g., frequency, regularity, quantity, quantity per unit time, etc.). Machine learning models 1210-1 and / or 1215-1 may include models that take the outputs of other models for further modeling.

[0264] After determining the impact status for one or more ambiguous endpoints, the impact analysis agent 1210 may provide impact data 1235 to the disposal feed agent 1215. The impact data 1235 may include the impact status for a single ambiguous endpoint or for multiple ambiguous endpoints. The impact data 1235 may include a display of the determined impact status (e.g., impact value, impact score). The impact data 1235 may also include a display of whether the determined impact status meets the impact threshold. The disposal feed agent 1210 may then determine a disposition for the ambiguous endpoint (e.g., using one or more of the machine learning model 1215-1 and / or rules / policies 1215-2). The disposition determined for an ambiguous endpoint may be an alternative disposition (e.g., block) that differs from the default disposition (e.g., default allow) selected or applied to that ambiguous endpoint. In some situations, the disposal feed agent may determine, based on the impact status, that the default disposition applied to the ambiguous endpoint is the most appropriate disposition, and therefore may not determine an alternative disposition for that endpoint that differs from the default disposition. An example scenario is described in detail below.

[0265] The disposal feed agent 1215 may also determine an alternative disposal for an ambiguous endpoint based solely on the determined impact status, or based on both the determined impact status and the determined threat status. The disposal feed agent 1215 may then receive threat data 1275 from a threat analysis agent (e.g., threat analysis agent 130). As described herein, the threat data 1275 may indicate, for example, that no CTIs have been received for the ambiguous endpoint, that any CTIs received for the ambiguous endpoint are unreliable (e.g., due to being received from a single provider or a relatively small number of providers), or that the CTIs received for the ambiguous endpoint contain a relatively small number of IOCs for the endpoint (e.g., below a threshold amount of IOCs for the endpoint). The disposal feed agent 1215 may determine an alternative disposal, for example, by comparing the determined impact status and the determined threat status to impact thresholds and threat thresholds, respectively. As an example, the disposal feed agent 1215 may determine an alternative disposal for an ambiguous endpoint based on whether the impact status meets the impact threshold and the threat status meets the threat threshold. The disposal feed agent 1215 may not determine an alternative disposal for an ambiguous endpoint (and therefore use the default disposal applicable to the endpoint) based on whether either or both of the impact status and threat status meet their respective thresholds. As described herein, the impact status and / or threat status may be used as numerical values ​​(e.g., "Impact: 50" and "Threat: 25") and / or text values ​​(e.g., "Impact: Low" and "Threat: Medium").

[0266] By combining impact status and threat status, a composite status can be obtained that indicates the occlusion potential of network traffic to and from an ambiguous endpoint. As mentioned above, impact status may include an impact score. Similar to impact status, threat status may include a threat score, which can quantify or characterize a threat. A threat score may indicate, for example, one or more risks associated with a threat or potential damage to an entity (e.g., the entity's network, the entity's network operations, the entity's data, the entity's general operations, etc.). The composite status is sometimes referred to as a composite occlusion status for convenience. As an example, the impact score of an impact status and the threat score of a threat status can be combined to obtain a composite occlusion score (or simply an occlusion score). The impact score and / or threat score may be weighted (e.g., based on the reliability of the impact score, the reliability of the threat score, or some other metric) when determining the composite occlusion score. The composite occlusion score can be a numerical value (e.g., the sum, average, or weighted average of the impact score ("10") and the threat score ("25")), a text value (e.g., a concatenation of the impact status ("low") and the threat status ("medium") ("low-medium")), a combination of numerical and text values ​​(e.g., a concatenation of the impact status ("10") and the threat status ("medium") ("10-medium")), or a data structure that pairs impact status and threat status (e.g., a vector or array (e.g., "

[10]

[25] ", "

[10] [medium]", or "[low][medium]")). The disposal feed agent 1215 may determine an alternative disposal for an ambiguous endpoint based on whether the composite occlusion status satisfies the composite occlusion threshold (or simply the occlusion threshold), or may not determine an alternative disposal for an ambiguous endpoint based on whether the composite occlusion status does not satisfy the composite occlusion threshold. Similar to impact thresholds and threat thresholds, the status of composite shielding potential can be a configurable setting or parameter for each entity, based on each entity's risk tolerance.

[0267] The disposition feed agent 1215 may provide feed data 1280 based on the alternative disposition determined for the endpoint. Feed data 1280 may be similar to feed data 155. For example, the format of feed data 1280 similarly depends on the type of feed (e.g., DNS feed, ACT feed, RPZ feed, composite feed). Feed data 1280 may include a representation of one alternative disposition determined for one ambiguous endpoint, or a representation of multiple alternative dispositions determined for each of multiple ambiguous endpoints. An alternative disposition may indicate monitoring the ambiguous endpoint (e.g., allowing and logging network traffic associated with the ambiguous endpoint), blocking the ambiguous endpoint (e.g., blocking network traffic associated with the endpoint), or allowing the ambiguous endpoint (e.g., without monitoring network traffic associated with the ambiguous endpoint). Once received, an alternative disposition may cause the receiving device to filter the traffic based on the threat level. For example, if an alternative disposition indicates monitoring the ambiguous endpoint, the receiving device (e.g., device 1260) may log network traffic associated with the endpoint. This record may be sent back to the computer environment 1200 for storage and / or to determine statistics, calculations, and other data based on the record, and to be stored in a data discovery repository (not shown) (for example, the same as or similar to the discovery data repository 123). If the disposal indicates that an endpoint is blocked, the receiving device (for example, device 1260) may block network traffic associated with the ambiguous endpoint. An indication of what network traffic is being blocked may be sent back to the computer environment 1200 for storage and / or to determine statistics, calculations, and other data based on the record, and to be stored in the discovery data repository.

[0268] One or more disposal feed agents 1215 may be configured to provide the same or similar functionality as the disposal feed agent 150 described herein. For example, one or more disposal feed agents 1215 may include any number of the above types of feeds, or combinations of the above types of feeds. The exact number or combination may also change over time. As a way of notifying the device which feeds are currently configured, one or more disposal feed agents 1215 may identify the configured feeds in a feed notification 1250. The feed notification 1250 may include the name of the currently configured disposal feed and / or an indication of what feed criteria are used for the configured disposal feed. In this manner, the receiving device (e.g., device 1260) may determine which disposal feeds it wishes to receive and begin receiving the desired disposal feeds.

[0269] The machine learning models 1210-1, 1215-1 and / or rules / policies 1210-2, 1215-2 in the impact analysis agent 1210 and the disposal feed agent 1215 may be similar to the machine learning models (e.g., machine learning models 130-1, 150-1) and rules / policies (e.g., rules / policies 130-2, 150-2) described herein with reference to Figures 1, 2, 3A–3D, 4, and 5A–5D, respectively. For example, machine learning models 1210-1, 1215-1 may include machine learning models that are neural networks of various configurations (e.g., feedforward neural networks, multilayer perceptron neural networks, radial basis function neural networks, recurrent neural networks, modular neural networks). These machine learning models may be trained using a controlled training process that uses human-labeled training data. These examples should be understood as variations that can operate in the example computer environment 1200 in Figure 12, similar computer environments, and any combination thereof. Also, for simplicity, the rules / policies shown in these examples are described as receiving output, making decisions, and / or performing analysis. This expression is used to simplify the idea that a computer device configured with an agent or rule / policy applies the rule / policy based on received input, making decisions, and / or performing analysis. In other words, this expression is used as a simplified way of saying that a rule / policy is used in relation to an agent or computer device that receives input, makes decisions, and / or performs analysis.

[0270] As described herein, the impact analysis agent 1210 may use a machine learning model 1210-1 to determine the impact status of an ambiguous endpoint based on data from the impact analysis data repository 1220 (e.g., network traffic data 1235, resource data 1240). For example, the machine learning model 1210-1 of the impact analysis agent 1210 may be trained using a corpus of data that includes historical network traffic data (e.g., network traffic data 1235) and / or resource data (e.g., resource data 1240). In this method, the trained machine learning model 1210-1 may be configured to receive network traffic data as input, relating to network traffic between the entity's network and the ambiguous endpoint. The trained machine learning model 1210-1 may also be configured to receive resource data associated with the entity as input. The trained machine learning model 1210-1 may further be configured to provide an impact status as output, based on the received network traffic data and, if provided, the received resource data. Machine learning model 1210-1 may determine the impact status based on a confidence value indicating whether blocking potentially legitimate network traffic between the entity's network and an ambiguous endpoint would adversely impact the entity's operations (e.g., business operations). The corpus of network traffic data and / or resource data may include human-labeled combinations of network traffic data and / or resource data, where the labels indicate whether blocking network traffic to and / or from the ambiguous endpoint would adversely impact the entity's operations.As described herein, the impact analysis agent 1210 may use rules / policies 1210-2 to determine (or facilitate the determination of impact status) ambiguous endpoints based on data from the impact analysis data repository 1220 (e.g., network traffic data 1235, resource data 1240). In this manner, rules / policies 1210-2 may be created to determine impact status and / or to provide additional data used to determine impact status.

[0271] As described herein, the disposal feed agent 1215 may use a machine learning model 1215-1 to determine alternative disposals for ambiguous endpoints based on impact data (e.g., impact data 1245) from the impact analysis agent 1210. The disposal feed agent 1215 may also use threat data (e.g., threat data 1275) to determine alternative disposals for ambiguous endpoints. For example, the machine learning model 1215-1 of the disposal feed agent 1215 may be trained using a corpus of data containing impact data and / or threat data. In this way, the trained machine learning model 1215-1 may be configured to receive impact data for one or more ambiguous endpoints as input. The trained machine learning model 1215-1 may also be configured to receive threat data for one or more ambiguous endpoints as input. The trained machine learning model 1215-1 may further be configured to provide alternative disposals for each of the one or more ambiguous endpoints as output, based on the received impact data and, if provided, on the received threat data. The machine learning model 1215-1 may determine alternative dispositions based on confidence values ​​associated with impact status. The impact data and / or threat data corpus may include human-labeled combinations of impact data and / or threat data, where the labels indicate alternative dispositions for impact data and / or combinations of impact data and threat data. As also described herein, the disposition feed agent 1215 may use rule / policy 1215-2 to determine (or facilitate the determination of alternative dispositions for) ambiguous endpoints based on impact data (e.g., impact data 1245) and received threat data, if provided. In this manner, rule / policy 1215-2 is created to determine alternative dispositions and / or to provide additional data used to determine alternative dispositions.

[0272] Having described the exemplary computer environment 1200 in Figure 12 along with examples of machine learning models and / or rules / policies for various agents 1210, 1215 in Figure 12, we will now describe examples of methods and flows that can be implemented by those agents with respect to how they determine alternative dispositions that, upon receipt, cause the device to filter network traffic to and / or from ambiguous endpoints. Figure 13 shows an example of a method 1300 that can be implemented by the impact analysis agent 1210 and the disposition feed agent 1215 in Figure 12 (for example, as part of one or more processes for determining alternative dispositions for ambiguous endpoints). Figure 14 shows an example of a flow in which an alternative disposition is determined for ambiguous endpoints and the device filters network traffic based on the alternative disposition. Figure 15 shows an example of a method 1500 that can be implemented by the impact analysis agent 1210 and the disposition feed agent 1215 in Figure 12 (for example, as part of one or more processes for determining alternative dispositions for ambiguous endpoints). Examples of methods 1300 and 1500 are merely examples of processes that may be carried out by the various agents 1210 and 1215 in Figure 12. Other variations may include omitting steps from example methods 1300 and / or example methods 1500, adding new steps to example methods 1300 and / or example methods 1500, and / or changing the order of steps from example methods 1300 and / or example methods 1500. Furthermore, for simplicity, examples of methods 1300 and 1500 and the example flows are described in terms of being carried out by one or more computer devices (for example, within the computer environment 1200 or a similar computer environment in Figure 12).

[0273] Beginning with Example 1300 of the method in Figure 13, in step 1305, one or more computer devices may receive endpoint data. Endpoint data may indicate one or more endpoints as described herein. Endpoint data may also include CTI data received or determined about endpoints. In step 1310, endpoint data may be analyzed to determine whether any of the identified endpoints are known malicious endpoints (i.e., blacklisted endpoints) or known non-malicious endpoints (e.g., whitelisted endpoints). As described herein, known malicious endpoints or known non-malicious endpoints may be referred to as unambiguous endpoints for ease of reference. For each endpoint (step 1310: Y) that is indicated and determined to be an unambiguous endpoint in the endpoint data, a disposition may be determined in step 1312. For example, a disposition determined for a known malicious endpoint may be a block disposition, and a disposition determined for a known non-malicious endpoint may be a permit disposition or a permit / monitoring disposition. Of the endpoints identified in the endpoint data, all endpoints that have not been determined (or cannot be determined) to be known malicious or known non-malicious endpoints (step 1310:N) may be determined to be ambiguous endpoints as described herein (e.g., graylisted endpoints). In step 1314, a default disposition for ambiguous endpoints may be determined. For example, a high-risk-tolerance entity may determine a default allow / monitoring disposition for ambiguous endpoints, while a high-risk-averse entity may determine a default block disposition for ambiguous endpoints. In step 1316, the impact status for ambiguous endpoints may be determined as described herein.In step 1318, the threat status for the ambiguous endpoint may be determined as described herein. The threat status may indicate, for example, that no CTI data has been received or determined for the ambiguous endpoint. The threat status may indicate, for example, that all CTI data received or determined for the ambiguous endpoint is unreliable CTI data. Step 1320 is optional, and here the occlusion status may be determined based on the threat status and impact status determined for the ambiguous endpoint as described herein. In step 1325, an alternative disposition for the ambiguous endpoint may be determined based on the impact status and threat status determined for the ambiguous endpoint. Determining an alternative disposition for the ambiguous endpoint may include determining an alternative disposition based on the occlusion status determined for the ambiguous endpoint. In step 1330, the disposition feed may be updated based on the alternative disposition determined for each ambiguous endpoint indicated in the endpoint data, as described herein. Steps 1305-1330 may be repeated when new (or additional) endpoint data is received.

[0274] Looking at the example flow in Figure 14, the CTI provider 1402, data collection agent 1404, threat analysis agent 1406, impact analysis agent 1408, disposal feed agent 1410, and computer device 1412 may be the same as or similar to the CTI providers 101-1 to 101-X, data collection agent 105, threat analysis agent 130, impact analysis agent 1210 and disposal feed agent 1215 in Figure 12, and computer device 170 in Figure 1, respectively. The example flow in Figure 14 is merely one example of how to determine alternative disposal and / or how to have the device filter network traffic based on the alternative disposal. In step 1420, one or more CTI providers 1402 may transmit CTI data 1422 to the data collection agent 1404. Based on the transmission of the CTI data 1422, the data collection agent 1404 receives, classifies, and stores the CTI data 1422 in step 1424, as described herein. Based on the received CTI data 1422, the data collection agent 1404 may determine the endpoint data 1426 as described herein. In step 1428a, the data collection agent 1404 may send the endpoint data 1426 to the threat analysis agent 1406. In step 1428b, the data collection agent 1404 may send the endpoint data 1426 (or a portion thereof) to the impact analysis agent 1408. In step 1430, the threat analysis agent 1406 may ingest and analyze the received endpoint data for one or more endpoints. For example, the endpoint data 1426 may indicate that no CTI data has been received for an endpoint (e.g., an ambiguous endpoint), or that the CTI data received for an endpoint is unreliable CTI data. The endpoint data 1426 may indicate whether an endpoint is an ambiguous endpoint or an unambiguous endpoint. The endpoint data provided to the impact analysis agent 1408 may indicate only ambiguous endpoints.Based on the received endpoint data 1426, the threat analysis agent 1406 may determine threat data 1434. In step 1432, the threat analysis agent 1406 may provide the threat data 1434 to the disposal feed agent 1410 as described herein. In step 1436, the data repository 1438 may provide the retained data 1440 to the impact analysis agent 1408 as described herein. For example, the retained data 1438 may include network traffic data and / or resource data. In step 1440, the impact analysis agent 1408 may ingest and analyze the received endpoint data 1426 and the received retained data 1438 to determine the impact of blocking potentially legitimate network traffic between the entity's network and one or more ambiguous endpoints, as described herein. In step 1442, the impact analysis agent 1408 may provide the impact status 1444 to the disposal feed agent 1410. In step 1446, the disposal feed agent 1410 may ingest and analyze the received impact status 1444 and received threat data 1434 to determine alternative disposals for ambiguous endpoints, as described herein. As a result, the disposal feed agent 1410 may determine feed data 1448 showing one or more alternative disposals determined for one or more ambiguous endpoints, as described herein. In step 1450, the disposal feed agent may transmit the feed data to the computer device 1412 via the disposal feed. Based on the alternative disposals shown in the feed data 1448, the computer device 1412 may be configured in step 1452 to filter network traffic to and / or from ambiguous endpoints based on the alternative disposals determined for those endpoints.

[0275] Referring here to Example 1500 of the method in Figure 15, in step 1502, one or more computer devices may receive CTI data for one or more endpoints as described herein. The CTI data may include one or more indicators that one or more endpoints were in a dangerous state. The reliability of the CTI data is determined, received, or retrieved. In step 1504, the reliability of the CTI data may be analyzed to determine whether it is sufficient to assign any of the identified endpoints to a list of known malicious endpoints (e.g., a blacklist) or to a list of known non-malicious endpoints (e.g., a whitelist). In this method, endpoints are identified as unambiguous or ambiguous endpoints based on the reliability of the CTI data. If the reliability of the CTI data is sufficient to assign an endpoint to a list of known malicious endpoints or a list of known non-malicious endpoints (step 1504: Y), then in step 1506, a disposition for that ambiguous endpoint may be determined. If the reliability of the CTI data is insufficient to assign the endpoint to a list of known malicious endpoints or a list of known non-malicious endpoints (step 1504:N), in step 1508, a default disposition for the ambiguous endpoint may be determined as described herein. In step 1510, the impact status for the ambiguous endpoint may be determined as described herein. The impact status may be used to adjust the reliability threshold for the CTI data. For example, if the impact status indicates a relatively high likelihood of adversely impacting the entity's operations by blocking potentially legitimate traffic between the entity's network and the ambiguous endpoint, the reliability threshold may be increased to require relatively reliable CTI data before blocking such traffic.As another example, if the impact status indicates that the likelihood of adversely impacting the entity's operations by blocking potentially legitimate traffic between the entity's network and the ambiguous endpoint is relatively low, the reliability threshold may be lowered to allow relatively unreliable CTI data to result in blocking such traffic. In other words, the reliability of the CTI data required to block network traffic to and / or from the ambiguous endpoint may depend on the impact of blocking potentially legitimate network traffic to and / or from that ambiguous endpoint. In step 1514, the reliability of the CTI data may be compared to an adjusted reliability threshold. If the reliability of the CTI data does not meet the adjusted reliability threshold (step 1514:N), in step 1516, the disposal feed may be updated based on the default disposal determined for the ambiguous endpoint. If the reliability of the CTI data meets the adjusted reliability threshold (step 1514: Y), then in step 1518, an alternative disposition for the ambiguous endpoint may be determined based on the impact status as described herein (for example, using the composite occlusion status for the ambiguous endpoint in combination with the threat status determined for the ambiguous endpoint). In step 1520, the disposition feed may be updated based on the alternative disposition determined for each ambiguous endpoint associated with the received CTI data. Steps 1502-1520 may be repeated when new (or additional) CTI data is received.

[0276] Next, we will describe examples of use cases for filtering network traffic based on impact status. For simplicity, the example use cases will be described in terms of being implemented by one or more computer devices (for example, within computer environment 1200 in Figure 12 or a similar computer environment). In the first use case, an endpoint is identified as an ambiguous endpoint, and a default allow disposition may be determined for that ambiguous endpoint. Determining the impact status of an ambiguous endpoint may include determining that the potential impact of blocking network traffic between the endpoint and the entity's network is a low potential impact that does not meet the threshold for a large impact. Thus, the alternative disposition determined for an ambiguous endpoint may be a block disposition. In the second use case, an endpoint is again identified as an ambiguous endpoint, and a default block disposition may be determined for that ambiguous endpoint. Determining the impact status of an ambiguous endpoint may include determining that the potential impact of blocking network traffic between the endpoint and the entity's network is a high potential impact that does not meet the threshold for a small impact. Thus, the alternative disposition determined for an ambiguous endpoint may be an allow disposition. Furthermore, it should be understood that, depending on the circumstances, even if the threat status is determined to be relatively high (e.g., a relatively serious threat), alternative measures may still allow network traffic between the entity's network and ambiguous endpoints. Such circumstances include stress testing of the entity's network and special projects involving high-priority individuals associated with the entity (e.g., the entity's chief technology officer, senior network administrators).

[0277] In addition to (or instead of) using impact status to update the disposition feed, impact status may be used to filter live network traffic in real time. In this case, when an entity's network receives network traffic, the corresponding endpoint may be identified as either an unambiguous or ambiguous endpoint, as described herein. Network traffic associated with an unambiguous endpoint may then be described as either malicious network traffic (if associated with a known malicious endpoint) or non-malicious network traffic (if associated with a known non-malicious endpoint). The remaining network traffic that is neither malicious nor non-malicious is called ambiguous network traffic. The impact status of ambiguous network traffic may be determined, as described herein. Disposition of ambiguous network traffic may be determined based on the determined impact status (for example, using or not using threat status, using or not using composite occludability status), as described herein. Computer equipment may be configured to filter ambiguous network traffic based on the determined disposition.

[0278] Furthermore, it should be understood that the disclosures herein relating to determining impact status do not need to be provided by the same system or within the same computer environment as determining the disposition of the endpoint. For example, a separate, independent system and / or computer environment may be configured to receive and ingest data from the computer environment 100 in Figure 1, determine the impact status, and provide the determined impact status as output to the computer environment 100 in Figure 1. In this example, the disposition feed agent 150 may be configured to receive and ingest the impact status from this separate system and / or computer environment and determine the alternative disposition of the endpoint as described herein.

[0279] Thus, the disclosures herein also provide techniques for clarifying and discovering hidden (or buried) attacks by focusing on ambiguous network traffic that is neither known to be malicious nor known to be non-malicious, and by providing additional context for making more careful judgments about ambiguous network traffic that is relatively likely to be malicious. In this way, the disclosures herein also enable the discovery of potential attacks in network traffic that is more difficult to analyze in a timely and computationally efficient manner. As described herein, an example of ambiguous network traffic that can be addressed more deliberately based on impact status is ambiguous network traffic to a particular geographic region and / or country where an alternative disposition may be decided if it is determined that such network traffic is unexpected in the sense that the entity has no legitimate reason for such network traffic, and blocking such network traffic would not have a meaningful adverse effect on the entity's operations. More specifically, certain TLDs may contain a statistically relatively large number of relatively malicious endpoints. Such TLDs may be called suspicious TLDs. Collecting and analyzing data from legitimate TLDs could result in an enormous amount of data. Therefore, it may be possible to collect and analyze network traffic originating from or destined for suspicious TLDs to determine the likelihood of that traffic being legitimate and take appropriate action.

[0280] The disclosures herein also provide techniques for uncovering hidden (or buried) attacks by retrospectively analyzing and clarifying historical network traffic. The techniques described herein (for example, Figures 1-2, 3A-3D, 4, 5A-5B, 6A-6B, 7-9, 10A-10E, 11A-11B, and the threat monitoring agents 130, 1096, 1196 and 140, 1097, 1197 shown therein) can be applied to historical network traffic. Historical network traffic and / or data relating to historical network traffic may be received from one or more entities. Data relating to network traffic may characterize or summarize historical network traffic, such as network traffic records (e.g., firewall records). As an example, network traffic blocked by an entity's firewall is an example of historical network traffic that can be retrospectively analyzed using the techniques described herein. Therefore, when considered in combination with additional contextual information (e.g., newly received CTI data, and / or data obtained during forensic analysis that expands the degree of isolation when acquiring additional contextual information as described herein), the entity's firewall records can be a source of additional data that can help discover past, present, or future network attacks. Thus, the threat analysis techniques described herein can be employed retrospectively in a forensic manner with respect to past network traffic to discover previously undetected network attacks. For example, past network traffic alone is insufficient to detect that a network attack has occurred. Additional contextual information that recontextualizes the past network traffic data in a manner that suggests the network's past network traffic was associated with an undetected network attack can be acquired after the receipt of that past network traffic data (e.g., by expanding the degree of isolation as described herein).The results of such retrospective analysis (e.g., a re-evaluation of past network traffic) may also be used to determine one or more alternative dispositions for an endpoint that differ from the occludability status for the endpoint and / or the default disposition that was determined to apply to the endpoint.

[0281] Figure 16 shows an example of a computer device 1601 that may be used to carry out one or more embodiments of the herein. For example, in some embodiments, the computer device 1601 may carry out one or more embodiments of the present disclosure by reading and / or executing instructions and performing one or more actions based on those instructions. The computer device 1601 may represent, be incorporated into, and / or include various devices such as desktop computers, computer servers, mobile devices (e.g., laptop computers, tablet computers, smartphones, and other types of mobile computer devices), and / or other types of data processing devices.

[0282] In some embodiments, the computer device 1601 may operate in a standalone environment. Alternatively, the computer device 1601 may operate in a network environment. As shown in Figure 12, various network nodes 1601, 1605, 1607, and 1609 are interconnected via a network 1603, such as the Internet. Other networks may also be used, such as a private intranet, entity network, LAN, wireless network, or personal network (PAN). Network 1603 is for illustrative purposes only and can be replaced with fewer or additional computer networks. A local area network (LAN) may have one or more known LAN topologies and may use one or more different protocols, such as Ethernet. Devices 1601, 1605, 1607, 1609, and other devices (not shown) may be connected to one or more networks via twisted pair wire, coaxial cable, optical fiber, radio waves, or other communication media.

[0283] As can be seen from Figure 16, the computer device 1601 may include a processor 1611, RAM 1613, ROM 1615, a network interface 1617, an input / output interface 1619 (e.g., keyboard, mouse, display, printer, etc.), and memory 1621. The processor 1611 may include one or more computer processing units (CPUs), a graphical processing unit (GPU), and other processing units such as processors adapted to perform calculations related to cyber threat detection and / or forms of machine learning. The input / output 1619 may include various interface units and drives for reading, writing, displaying, and / or printing data or files. The input / output 1619 may be coupled with a display such as display 1620. The memory 1621 may store software for configuring the computer device 1601 into a special-purpose computer device in order to perform one or more of the various functions described herein. Memory 1621 may store an operating system 1123, which is software for controlling the overall operation of the computer device 1601; control logic 1625 for instructing the computer device 1601 to perform the embodiments described herein; threat analysis software 1627 configured to perform any of the processes and / or methods described above; training data 1629 available for training any or all of the machine learning models described above; and other applications 1631. The control logic 1625 may be incorporated into or be part of the dataset processing software 1627. In other embodiments, the computer device 1601 may include two or more of any and / or all of these components (e.g., two or more processors, two or more memories, etc.), and / or other components and / or subsystems not shown herein.

[0284] Devices 1605, 1607, and 1609 may have similar or different architectures as described with respect to computer device 1601. Those skilled in the art will understand that the functions of computer device 1601 (or devices 1605, 1607, and 1609) described herein may extend to multiple data processing devices, for example, to distribute processing loads across multiple computers, to isolate transactions based on geographical location, user access levels, and quality of service (QoS), and to use cloud-based computing services. For example, devices 1601, 1605, 1607, 1609, and others may work together to provide parallel computing capabilities that support the operation of control logic 1625, threat analysis software 1627, and / or other applications 1631.

[0285] One or more embodiments described herein may be embodied in computer-readable or computer-executable data and / or computer-executable instructions, such as within one or more program modules executed by one or more computers or other devices, as described herein. Generally, program modules include routines, programs, objects, components, data structures, etc., which, when executed by a processor of a computer or other device, perform a particular task or implement a particular abstract data type. Modules may be written in a programming language for source code that is subsequently compiled for execution, or in a scripting language such as HTML or XML (but not limited to these). Computer-executable instructions may be stored in computer-readable media such as hard disks, optical discs, removable storage media, solid-state memory, RAM, etc. As will be understood by those skilled in the art, the functions of program modules may be combined or distributed as desired in various embodiments. Furthermore, all or part of the functions may be embodied in firmware or hardware equivalents such as integrated circuits, field-programmable gate arrays (FPGAs), etc. Certain data structures may be used to more effectively implement one or more embodiments described herein, and such data structures are intended within the scope of computer executable instructions and computer usable data described herein. The various embodiments discussed herein may be embodied as methods, computer devices, data processing systems, or computer program products.

[0286] While this subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter as defined in any of the following exemplary embodiments is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms for carrying out any of the statements or attached statements.

[0287] The following describes various features in a series of numbered statements or paragraphs. These characteristics are not to be construed as limiting the invention or inventive concept, but are provided merely to identify some of the characteristics described herein, and do not imply a particular order of importance or relevance of such characteristics.

[0288] In a first example, the statement may relate to refining threat data based on received CTI data, for example, determining threat difference data based on received CTI data that indicates a change in threat to one or more endpoints.

[0289] (Statement 1) A method comprising receiving cyber threat intelligence (CTI) data including a first indicator of compromise (IOC) for an endpoint from a first provider of a plurality of providers.

[0290] (Statement 1A) The method of statement 1, further comprising determining first endpoint data indicative of the first IOC for the endpoint based on the first CTI data.

[0291] (Statement 1B) The method according to any one of statements 1 and 1A, further comprising determining that a change has occurred for the endpoint based on an analysis of the first endpoint data and stored event data related to the endpoint, the stored event data indicating one or more second IOCs for the endpoint received from the plurality of providers.

[0292] (Statement 1C) Based on determining that the change has occurred for the endpoint, determining threat difference data for the endpoint that indicates one or more attributes that have changed for the endpoint between the stored event data and the first endpoint data, the threat difference data being between the stored event data and the first endpoint data for the endpoint, The method according to any one of statements 1, lA to lB, further comprising.

[0293] (Statement 1D) Based on the threat difference data, determining a disposition for the endpoint, The method according to any one of statements 1, lA to lC, further comprising.

[0294] (Statement lE) The first computer device transmitting the disposition to the second computer device to cause the second computer device to filter network traffic based on the disposition, The method according to any one of statements 1, lA to 1D, further comprising.

[0295] (Statement 2) Determining the disposition for the endpoint is based on how many of the plurality of providers indicated an IOC for the endpoint, The method according to any one of statements 1, lA to E.

[0296] (Statement 3) Determining the disposition for the endpoint is based on which of the plurality of providers indicated an IOC for the endpoint, The method according to any one of statements 1, lA to 1E, 2.

[0297] (Statement 4) Determining the disposition for the endpoint is based on showing that one or more of the plurality of providers have repeatedly indicated the same IOC for the endpoint, The method according to any one of statements 1, lA to 1E, 2.

[0298] (Statement 5) Determining the disposition against the endpoint is done according to one of statements 1, lA-1E, 2-4, based on how many of the multiple providers showed the same IOC to the endpoint.

[0299] (Statement 6) The method according to any one of statements 1, lA-E, 2-5, wherein determining the disposition against the endpoint is done based on one or more of the following: a first confidence value related to the first IOC, a second confidence value related to one or more attributes, and a third confidence value related to the first provider.

[0300] (Statement 7) The method according to any one of statements 1, lA-1E, 2-6, wherein the first endpoint data is in a second format, and further comprises training multiple machine learning models for the multiple providers, each configured after training to receive input in the format in which the providers transmit CTI data and provide output in the second format, and determining the first endpoint data is performed on the basis of using a first machine learning model among the multiple machine learning models to provide the first CTI data as input to the first machine learning model.

[0301] (Statement 8) The method according to any one of statements 1, lA-1E, 2-7, further comprising training a machine learning model, which after training is configured to output an indication of whether the endpoint has changed based on input data related to the endpoint, wherein determining that the change has occurred with respect to the endpoint is performed by using the machine learning model and providing the first endpoint data and the stored event data as input to the machine learning model.

[0302] (Statement 9) The method of any one of statements 1, 1A-1E, 2-8, further comprising training a machine learning model, which after training is configured to output criteria for a new feed, receiving a first criterion for a feed based on providing the threat differential data as input to the machine learning model, constructing the feed, and transmitting the disposition, which is performed via the feed.

[0303] (Statement 10) One or more non-transient computer-readable media storing computer executable instructions that, when executed, cause one or more computer devices to: receive cyber threat intelligence (CTI) data from a first provider among a plurality of providers, including a first indicator of compromise (IOC) for an endpoint; determine first endpoint data indicating the first IOC for the endpoint based on the first CTI data; determine that a change has occurred with respect to the endpoint based on an analysis of the first endpoint data and stored event data related to the endpoint, which included one or more second IOCs for the endpoint received from the plurality of providers; determine threat differential data for the endpoint, which included one or more attributes that have changed for the endpoint between the stored event data and the first endpoint data, based on the determination that the change has occurred with respect to the endpoint; determine a disposition for the endpoint based on the threat differential data; and transmit the disposition to the device to filter network traffic based on the disposition.

[0304] (Statement 11) The computer-executable instruction, when executed, causes one or more computer devices to determine the disposition to the endpoint based on how many of the multiple providers have indicated an IOC to the endpoint, as described in Statement 10.

[0305] (Statement 12) When the aforementioned computer executable instruction is executed, it will be executed on one or more computer devices. A non-transitory computer-readable medium according to any one of statements 10 and 11, causing the determination of the disposition for the endpoint based on which of the plurality of providers indicated an IOC for the endpoint.

[0306] (Statement 13) The computer-executable instructions, when executed, cause the one or more computer devices to A non-transitory computer-readable medium according to any one of statements 10 to 12, causing the determination of the disposition for the endpoint based on indication that one or more of the plurality of providers repeatedly indicated the same IOC for the endpoint.

[0307] (Statement 14) The computer-executable instructions, when executed, cause the one or more computer devices to A non-transitory computer-readable medium according to any one of statements 10 to 13, causing the determination of the disposition for the endpoint based on how many of the plurality of providers indicated the same IOC for the endpoint.

[0308] (Statement 15) A non-transient computer-readable medium as described in any one of statements 10 to 14, wherein the first endpoint data is in a second format, and the computer-executable instruction causes one or more computer devices, when executed, to train a plurality of machine learning models for the plurality of providers, and after training, train a plurality of machine learning models, each configured to receive input in a format in which the providers transmit CTI data and provide output in a second format, and the computer-executable instruction causes one or more computer devices, when executed, to determine the first endpoint data based on providing the first CTI data as input to the first machine learning model using the first machine learning model among the plurality of machine learning models.

[0309] (Statement 16) A non-transient computer-readable medium as described in any one of statements 10 to 15, wherein the computer-executable instruction causes one or more computer devices to train a machine learning model, which after training is configured to output an indication of whether the endpoint has changed, based on input data associated with the endpoint, and the computer-executable instruction causes one or more computer devices to determine that the change has occurred with respect to the endpoint, based on using the machine learning model to provide the first endpoint data and the stored event data as input to the machine learning model.

[0310] (Statement 17) The computer-executable instruction, when executed, causes one or more computer devices to train a machine learning model, which after training is configured to output criteria for a new feed; to receive a first criterion for a feed based on providing the threat differential data as input to the machine learning model; and to construct the feed; and the computer-executable instruction, when executed, causes one or more computer devices to transmit the disposition via the feed, a non-transient computer-readable medium as described in any one of statements 10 to 16.

[0311] (Statement 18) One or more computer devices comprising one or more processors and memory, wherein the memory stores computer executable instructions that, when executed by the one or more processors, cause the one or more computer devices to: receive cyber threat intelligence (CTI) data from a first provider among a plurality of providers, including a first indicator of compromise (IOC) for an endpoint; determine first endpoint data indicating the first IOC for the endpoint based on the first CTI data; determine that a change has occurred for the endpoint based on an analysis of the first endpoint data and stored event data related to the endpoint, which has been received from the plurality of providers and indicates one or more second IOCs for the endpoint; determine threat differential data for the endpoint, which indicates one or more attributes that have changed for the endpoint between the stored event data and the first endpoint data, based on the determination that the change has occurred for the endpoint; determine a disposition for the endpoint based on the threat differential data; and transmit the disposition to the device to filter network traffic based on the disposition.

[0312] (Statement 19) The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine the disposition to the endpoint based on how many of the multiple providers have indicated an IOC to the endpoint, as described in Statement 18.

[0313] (Statement 20) The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine the disposition to the endpoint based on the fact that one or more of the multiple providers have repeatedly shown the same IOC to the endpoint, as described in any one of statements 18 to 19.

[0314] In the second example, the statement may relate to refining threat data based on received CTI data, for example, determining additional context for an endpoint based on received CTI data indicating the occurrence of additional IOCs for that endpoint.

[0315] (Statement 21) A method comprising receiving first cyber threat intelligence (CTI) data, including a first indicator of compromise (IOC) for an endpoint, from a first provider among multiple providers.

[0316] (Statement 21A) The method according to statement 21, further comprising determining first endpoint data indicating the first IOC for the endpoint based on the first CTI data.

[0317] (Statement 21B) The method according to any one of statements 21, 21A, further comprising determining first threat differential data indicating that a first IOC to the endpoint is the first occurrence of an IOC to the endpoint, based on the first endpoint data.

[0318] (Statement 21C) The method according to any one of statements 21, 21A to 21B, further comprising determining a first disposition for the endpoint that indicates a first threat level for the endpoint, based on the first threat differential data.

[0319] (Statement 21D) The method according to any one of statements 21, 21A to 21C, further comprising the first computer device transmitting the first disposition to the second computer device in order to filter the first network traffic based on the first disposition. ...

Claims

1. Receiving cyber threat intelligence (CTI) data from a first provider among multiple providers, including a first indicator of compromise (IOC) for an endpoint; determining first endpoint data indicating the first IOC for the endpoint based on the first CTI data; Based on an analysis of the first endpoint data and the stored event data related to the endpoint, which includes one or more second IOCs for the endpoint received from the multiple providers, it is determined that a change has occurred with respect to the endpoint. Based on the determination that the aforementioned change has occurred with respect to the endpoint, threat differential data for the endpoint is determined, which shows one or more attributes that have changed for the endpoint between the stored event data and the first endpoint data. Based on the aforementioned threat differential data, a decision is made regarding the action to be taken against the endpoint. The first computer device transmits the disposal to the second computer device in order to filter the network traffic based on the disposal, A method that includes this.

2. The method according to claim 1, wherein determining the disposition for the endpoint is based on how many of the plurality of providers have shown an IOC for the endpoint.

3. The method according to claim 1, wherein determining the disposition for the endpoint is based on which of the plurality of providers has shown an IOC for the endpoint.

4. The method according to claim 1, wherein determining the disposition against the endpoint is based on the fact that one or more of the multiple providers have repeatedly shown the same IOC to the endpoint.

5. The method according to claim 1, wherein determining the disposition for the endpoint is based on how many of the multiple providers showed the same IOC for the endpoint.

6. The method according to claim 1, wherein determining the disposition for the endpoint is based on one or more of the following: a first reliability value related to a first IOC, a second reliability value related to one or more attributes, and a third reliability value related to the first provider.

7. The first endpoint data is in the second format, The method further includes training multiple machine learning models for the aforementioned multiple providers, wherein, after training, each of the multiple machine learning models is configured to receive input in the format in which the providers transmit CTI data and provide output in the second format, The method according to claim 1, wherein determining the first endpoint data is performed by using a first machine learning model among the plurality of machine learning models and providing the first CTI data as input to the first machine learning model.

8. Training a machine learning model, further comprising training a machine learning model configured to output, after training, an indication of whether the endpoint has changed, based on input data related to the endpoint, The method according to claim 1, wherein determining that the change has occurred with respect to the endpoint is performed by using the machine learning model and providing the first endpoint data and the stored event data as input to the machine learning model.

9. Training a machine learning model, which, after training, is configured to output a criterion for a new feed. Receiving a first criterion for a feed based on providing the aforementioned threat differential data as input to the machine learning model, Further including constructing the aforementioned feed, The method according to claim 1, wherein transmitting the disposition is performed via the feed.

10. When executed, it will be performed on one or more computer devices. Receiving cyber threat intelligence (CTI) data, including a first indicator of compromise (IOC) for an endpoint, from a first provider among multiple providers. Based on the first CTI data, determine the first endpoint data indicating the first IOC for the endpoint. Based on an analysis of the first endpoint data and the stored event data related to the endpoint, which includes one or more second IOCs for the endpoint received from the multiple providers, it is determined that a change has occurred with respect to the endpoint. Based on the determination that the aforementioned change has occurred with respect to the endpoint, threat differential data for the endpoint is determined, which shows one or more attributes that have changed for the endpoint between the stored event data and the first endpoint data. Based on the aforementioned threat differential data, a decision is made regarding the action to be taken against the endpoint. To transmit the said disposition to the device in order to filter network traffic based on the said disposition, One or more non-transient computer-readable media containing computer executable instructions that perform the following actions.

11. The non-transient computer-readable medium according to claim 10, wherein the computer-executable instruction, when executed, causes one or more computer devices to determine the disposition to the endpoint based on how many of the plurality of providers have indicated an IOC to the endpoint.

12. The non-transient computer-readable medium according to claim 10, wherein the computer-executable instruction, when executed, causes one or more computer devices to determine the disposition to the endpoint based on which of the plurality of providers has indicated an IOC to the endpoint.

13. The non-transient computer-readable medium according to claim 10, wherein the computer-executable instruction, when executed, causes one or more computer devices to determine the disposition to the endpoint based on the fact that one or more of the multiple providers have repeatedly shown the same IOC to the endpoint.

14. The non-transient computer-readable medium according to claim 10, wherein the computer-executable instruction, when executed, causes one or more computer devices to determine the disposition to the endpoint based on how many of the plurality of providers have shown the same IOC to the endpoint.

15. The first endpoint data is in the second format, The computer executable instruction, when executed, causes one or more computer devices to train multiple machine learning models for the multiple providers, and after training, train multiple machine learning models, each configured to receive input in a format in which the providers transmit CTI data and provide output in a second format. When executed, the computer executable instruction causes one or more computer devices to determine the first endpoint data by using a first machine learning model among the plurality of machine learning models and providing the first CTI data as input to the first machine learning model. The non-transient computer-readable medium according to claim 10.

16. The computer executable instruction, when executed, causes one or more computer devices to train a machine learning model, which is configured to output, after training, an indication of whether the endpoint has changed, based on input data related to the endpoint. The non-transient computer-readable medium according to claim 10, wherein the computer-executable instruction, when executed, causes one or more computer devices to determine that the change has occurred with respect to the endpoint, based on providing the first endpoint data and the stored event data as input to the machine learning model using the machine learning model.

17. The aforementioned computer executable instruction, when executed, instructs one or more computer devices to train a machine learning model, wherein the machine learning model is configured to output a criterion for a new feed after training. Receiving a first criterion for a feed based on providing the aforementioned threat differential data as input to the machine learning model, To construct the aforementioned feed, This is to have them do it, The non-transient computer-readable medium according to claim 10, wherein the computer-executable instruction, when executed, causes one or more computer devices to transmit the disposition via the feed.

18. One or more computer devices comprising one or more processors and memory, wherein the memory, when executed by the one or more processors, Receiving cyber threat intelligence (CTI) data, including a first indicator of compromise (IOC) for an endpoint, from a first provider among multiple providers. Based on the first CTI data, determine the first endpoint data indicating the first IOC for the endpoint. Based on an analysis of the first endpoint data and the stored event data related to the endpoint, which includes one or more second IOCs for the endpoint received from the multiple providers, it is determined that a change has occurred with respect to the endpoint. Based on the determination that the aforementioned change has occurred with respect to the endpoint, threat differential data for the endpoint is determined, which shows one or more attributes that have changed for the endpoint between the stored event data and the first endpoint data. Based on the aforementioned threat differential data, a decision is made regarding the action to be taken against the endpoint. To transmit the said disposition to the device in order to filter network traffic based on the said disposition, One or more computer devices that store computer-executable instructions for performing the following actions.

19. The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine the disposition to the endpoint based on how many of the plurality of providers have indicated an IOC to the endpoint, according to one or more computer devices of claim 18.

20. The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine the disposition to the endpoint based on which of the plurality of providers indicated an IOC to the endpoint, according to claim 18.

21. The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine the disposition to the endpoint based on the fact that one or more of the plurality of providers have repeatedly shown the same IOC to the endpoint, according to one or more computer devices of claim 18.

22. The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine the disposition to the endpoint based on how many of the multiple providers have shown the same IOC to the endpoint, according to one or more computer devices of claim 18.

23. The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine the disposition to the endpoint based on one or more of the following: a first reliability value related to a first IOC, a second reliability value related to one or more attributes, and a third reliability value related to a first provider, according to claim 18.

24. The first endpoint data is in the second format, The computer executable instruction, when executed by one or more processors, causes one or more computer devices to train multiple machine learning models for multiple providers, and after training, further train multiple machine learning models, each configured to receive input in a format in which the providers transmit CTI data and provide output in a second format, The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine the first endpoint data by providing the first CTI data as input to the first machine learning model using the first machine learning model among the plurality of machine learning models.

25. The computer executable instruction, when executed by one or more processors, causes one or more computer devices to further train a machine learning model, which after training is configured to output an indication of whether the endpoint has changed, based on input data related to the endpoint. The computer executable instruction, when executed by one or more processors, causes one or more computer devices to determine that the change has occurred with respect to the endpoint, based on using the machine learning model to provide the first endpoint data and the stored event data as input to the machine learning model.

26. When the aforementioned computer executable instruction is executed by one or more processors, it is executed on one or more computer devices. Training a machine learning model, which, after training, is configured to output a criterion for a new feed. Receiving a first criterion for a feed based on providing the aforementioned threat differential data as input to the machine learning model, This further involves constructing the aforementioned feed, The computer executable instruction, when executed by one or more processors, causes one or more computer devices to transmit the disposal via the feed, as described in claim 18.

Images