Method, Packet Filtering Appliance, and Computer-Readable Medium for Efficient Threat-Context-Aware Packet Filtering for Network Protection - Patent application

Abstract

Threat indicator rules are applied to packets in transit crossing the boundary between a protected network and a public network, such as the Internet. [Solution] The packet filtering appliance receives one or more threat indicators from one or more cyber threat intelligence (CTI) providers, determines a plurality of packet filtering rules associated with the one or more threat indicators, configures the packet filtering appliance with the plurality of packet filtering rules, receives an in-flight packet, determines that the in-flight packet matches one of the plurality of packet filtering rules, determines threat context information based on the rule that was not previously determined prior to receiving the in-flight packet, determines a disposition and / or one or more directives based on the threat context information, and applies the disposition and / or one or more directives to the in-flight packet.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Application No. 17 / 235,544, filed April 20, 2021 (now U.S. Patent No. 11,159,546), U.S. Application No. 17 / 508,596, filed October 22, 2021 (now U.S. Patent No. 11,316,876), U.S. Application No. 17 / 508,614, filed October 22, 2021, U.S. Application No. 17 / 695,047, filed March 15, 2022, and U.S. Application No. 17 / 713,827, filed April 5, 2022, the contents of which are expressly incorporated by reference in their entireties herein.

[0002] Aspects of the invention (hereinafter sometimes referred to as the "disclosure") described herein generally relate to computer hardware and software and network security. In particular, one or more aspects of the disclosure generally relate to computer hardware and software for efficient filtering of in-transit packets that determine the action (e.g., disposition and / or directive) to be taken for each in-transit packet (e.g., whether to block or allow each in-transit packet) depending on the threat context at the time the in-transit packet is observed. Other aspects and features are also described herein. [Background technology]

[0003] As the information age advances, Transmission Control Protocol / Internet Protocol (TCP / IP) network security becomes increasingly important. Network threats / attacks can take many forms (e.g., unauthorized requests or data transfers, viruses, malware, floods of traffic designed to overwhelm resources, etc.).

[0004] To combat these threats and attacks, various cyber defense methods and systems have been developed and deployed. A key component of cyber defense is a network appliance (e.g., packet filtering appliance) that applies a set of packet filtering rules to Layer 3 / Internet Protocol (L3 / IP) packets in transit and determines whether to allow / prompt each packet toward its destination or block / drop the packet. These packet filtering appliances may be inserted in-line on links located at the border between a private network, such as a corporate network, and the public Internet, and may be configured with a set of packet filtering rules, i.e., policies, that may be designed to protect or otherwise defend the private network in some manner. For example, early generation network firewalls were typically configured with packet filtering rules that enforced the access control policies of the private network, such as which Internet services (i.e., well-known ports associated with Internet hosts) that internal hosts may be allowed to access, and conversely, which internal resources may be accessed by which (unauthorized) Internet hosts. In another example, current generation packet filtering appliances include threat intelligence gateways (TIGs), which can be configured with packet filtering rules having packet matching criteria that correspond to network addresses, e.g., IP addresses, 5-tuple values, domain names, URIs, etc., of cyber threats identified by a cyber threat intelligence (CTI) provider.

[0005] Although there are no formal standards required for packet filtering rule syntax and semantics, packet filtering appliances typically support packet filtering rules that generally conform to this high-level exemplary representative schema. <disposition><directives><matching-criteria>;<metadata>, where <disposition> is, for example, one of blocking / rejecting / dropping (commonly referred to herein as "blocking") or allowing / passing / forwarding (commonly referred to herein as "allowing") packets that match the rule, the <matching criteria> corresponds to some combination of internet layer (L3), transport layer (L4), and application layer header field values, e.g., source and destination IP addresses, protocol, source and destination ports, domain names, Uniform Resource Identifiers (URIs) such as Uniform Resource Locators (URLs) or Uniform Resource Names (URNs), and the <metadata> is information associated with the rule that may be used to inform applications about the packet and / or the rule, for example, the metadata may indicate the source of the matching criteria and may be included in associated logs that may be processed by applications for, for example, cyber situational awareness, cyber analysis, cyber defense, cyber network protection, etc. A <command> may be a signal that instructs the operational application logic of the packet filtering appliance to process matching packets according to the logic associated with the command.For example, this logic may be additional packet and / or policy processing actions that may be applied to matching packets (e.g., signaled by directives such as "log," "flow log," "capture," "mirror," "redirect," "spoof-tcp-rst," etc.), whether to (conditionally) apply the rule when the packet flows in ("in"), when the packet flows out ("out"), or both ("in-out"), whether to (conditionally) continue to apply subsequent rules in the policy to matching packets ("continue" or "quick"), associating the rule with a particular interface of the packet filtering appliance, etc.

[0006] One approach to cyber defense is to filter packets associated with Internet threats, which are Internet hosts and / or resources managed by Internet hosts that may be associated with malicious activity. These threats may be investigated and identified by Cyber ​​Threat Intelligence (CTI) provider organizations that publish CTI reports on threats. The CTI reports may include threat indicators, which may be network addresses in the form of IP addresses, 5-tuples, domain names, URIs, etc., associated with Internet hosts and / or resources that may be involved in malicious activity. The threat indicators may be collected from multiple CTI provider organizations and used to create a set / policy comprised of packet filtering rules having matching criteria corresponding to the threat indicators. Such packet filtering rules generated from the threat indicators are hereinafter referred to as "threat indicator rules" and the set of threat indicator rules comprises a "CTI-based policy" for protecting the network from Internet threats. Packet filtering appliances placed at the boundary between a protected network (e.g., a private network) and a potentially unprotected network (e.g., a public network such as the Internet) may be configured with these policies and may apply them to all in-flight packets traversing the boundary, thereby protecting the private network from Internet threats, for example, by blocking / dropping packets associated with the threat. Because a gateway is an interface at the boundary between two different networks, such as between a CTI-protected network and an unprotected network, such packet filtering appliances configured with CTI-based policies and logic to enforce policies may be referred to as threat intelligence gateways (TIGs).

[0007] While this CTI-based cyber defense approach may seem simple, it is not for several reasons. One reason is that the threat risk associated with a threat indicator may not be deterministic in the sense that, for example, a packet in transit having a header field value matching a threat indicator may not necessarily be associated with malicious activity, but instead may be associated with legitimate business activity or some benign activity. For example, a website hosting service may use a single IP address to host multiple domains. One of the domains may be involved in malicious activity, while the other domains are only involved in legitimate activity. A CTI provider may detect malicious activity associated with one malicious domain, but may expose every single IP address of the domain as a threat indicator, and further, the CTI provider may assign / associate such threat indicator with a high confidence, a high risk score, a recommended disposition (e.g., "block"), etc. A consumer of such CTI and associated threat indicators may consider it undesirable and wish to filter out such CTI before applying it. Furthermore, what is filtered out may be relative / contextual to a given consumer. For example, an enterprise may subscribe to a CTI provider's threat indicator feed, but discover that the enterprise's own networked hosts and resources are listed as threat indicators, e.g., because the hosts may be compromised by malware and / or malicious actors.

[0008] Thus, when creating threat indicator rules, consumers may not select the "block" disposition by default, even if the associated CTI provider recommends "block" with high confidence, for example, because it may block legitimate business traffic. This may lead to consumers falsely designating actual attacks against their network as not being a threat, potentially blocking the attack rather than blocking it. Conversely, other more risk-averse consumers may choose not to select the "allow" disposition by default (and packet processing actions / directives configured to monitor for potential threats) because it may allow malicious traffic to attack and damage networked resources, which may be considered a false negative. In addition, there are other reasons why consumers may be uncertain about whether to select a "block" or "allow" disposition despite the published recommendation of the CTI provider, and other reasons why the disposition (block or allow) that may best protect the network may not be easily determined when creating threat indicator rules that include network protection policies intended to be applied to future packets that have not yet been received.

[0009] Thus, it may be problematic to predetermine the disposition of a threat indicator rule, i.e., "block" or "allow," when the threat risk associated with a threat indicator is uncertain, subjective, and / or probabilistic in nature (e.g., risk probability in the range [0, 1]) versus deterministic, objective, or binary in nature (i.e., "all risk" risk probability = 1, or "no risk" risk probability = 0). Similarly, it may be problematic to predetermine any directive for a threat indicator rule, since these directives are often correlated with the disposition, the risk associated with the threat indicator, and / or other factors.

[0010] Thus, a need exists for improved network protection computer logic and techniques associated with the application of threat indicator rules to packets in transit traversing the boundary between a protected network and a public network, such as the Internet, with the aim of improving cyber defense against Internet threats. Summary of the Invention

[0011] The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosure. It is not intended to identify key or critical elements of the disclosure or to delineate the scope of the disclosure. The following summary merely presents some concepts of the disclosure in a simplified form as a prelude to the detailed description that follows.

[0012] According to certain aspects as described herein, a determination of a disposition (e.g., “block” or “allow”) of a threat indicator rule, as well as a determination of one or more directives to apply to an in-transit packet, may be delayed until an in-transit packet matching the threat indicator rule is observed. Based on observation of the rule-matching packet by the packet filtering appliance (e.g., in response to receiving or observing the packet and / or after receiving or observing the packet), logic of the packet filtering appliance may efficiently calculate, access, and / or otherwise determine threat context information that was not available, applicable, or otherwise unknown at the time the threat indicator rule was created. Such threat context information may be used as input to the logic that determines a disposition (e.g., block or allow), as well as a directive to apply to the in-transit packet. This may result in a disposition and / or directive being applied to the in-transit packet that better protects the network compared to solely using a disposition and directive that was pre-determined prior to receiving the in-transit packet. Threat context information that may be used to determine disposition and instructions for packets in transit may include, for example, the time of packet observation / filtering, the location and / or administrator of the associated packet filtering appliance, and / or whether the packet is associated with an active attack on the associated network and / or other networks connected to the Internet.

[0013] Thus, a new disposition may be specified for a threat indicator rule at the time the threat indicator rule is created / generated. For convenience and illustrative purposes only, this new disposition is referred to herein as a "protect" disposition, however, this new disposition may be assigned any name as desired, such as "undefined," "neutral," "TBD," and / or any other name as desired. Alternatively, the rule may not be assigned any disposition and / or directive (e.g., null, blank, or missing disposition and / or directive) and based on the missing disposition and / or directive, the logic of the network appliance may determine (e.g., calculate) a disposition and / or one or more directives using threat context information associated with the observed in-flight packets. In a further example, the rule may be assigned a first disposition (e.g., "block," "allow," etc.) and / or first one or more directives and based on the observed in-flight packets and based on the threat context information, the logic of the network appliance may determine (e.g., calculate) a different second disposition and / or different second one or more directives. In still further examples, a rule may be assigned a "block" or "allow" disposition (or any other disposition) and may also be assigned or otherwise associated with an indicator, such as a flag or signal, that indicates that threat context information is used for that rule at in-flight packet observation time. An example of such an indicator may be, for example, a simple one-bit flag, where one value of the indicator signals to the TIG or other packet filtering appliance that the assigned disposition is correct and another value of the indicator signals to the TIG or other packet filtering appliance that the assigned disposition is flexible and / or that the disposition is calculated at in-flight packet observation time.In any of these situations, upon receiving and observing an in-flight packet that matches a rule that does not indicate a predetermined disposition (e.g., a rule with a "protect" disposition, or a rule with a missing / blank disposition, or a rule with an indicator as discussed above), the packet filtering appliance (e.g., TIG) may use the threat context information to efficiently calculate the disposition, e.g., "block" or "allow," to actually apply to the packet, as well as one or more directives that may best protect the network from the associated threat, and may then apply the calculated disposition and directive to the in-flight packet. The efficient determination (e.g., calculation) and application of the in-flight packet's disposition and / or directive may be completed before processing / filtering the next in-flight packet and / or before the next in-flight packet is received by the packet filtering appliance. Thus, the calculation and application may be sufficiently efficient such that, regardless of packet transmission speed and associated traffic load, the in-flight packet may be processed / filtered without incurring excessive latency and / or packet drops that may significantly impact the performance of the associated networked application.

[0014] Further aspects described herein are directed to receiving, by a packet filtering appliance, one or more threat indicators from one or more cyber threat intelligence providers, determining a plurality of packet filtering rules associated with the one or more threat indicators, configuring the packet filtering appliance with the plurality of packet filtering rules, receiving an in-flight packet, determining that the in-flight packet matches a rule of the plurality of packet filtering rules, determining threat context information based on the rule that was not pre-determined prior to receiving the in-flight packet, determining a disposition and / or one or more directives based on the threat context information, and applying the disposition and / or one or more directives to the in-flight packet.

[0015] Further aspects described herein are directed to receiving, by a packet filtering appliance, one or more threat indicators from one or more cyber threat intelligence providers, determining a plurality of packet filtering rules associated with the one or more threat indicators, configuring the packet filtering appliance with the plurality of packet filtering rules, receiving an in-flight packet, determining that threat context information is determined based on a rule of the plurality of rules that matches the in-flight packet, determining the threat context information, where the threat context information has not been pre-determined prior to receiving the in-flight packet, determining a disposition based on the threat context information, and applying the disposition to the in-flight packet.

[0016] Further aspects described herein are directed to receiving, by a packet filtering appliance, a plurality of packet filtering rules determined based on a plurality of threat indicators determined based on cyber intelligence reports from a plurality of cyber threat intelligence providers; configuring the packet filtering appliance with the plurality of packet filtering rules; receiving an in-flight packet from a first network destined for a second network; determining threat context information based on determining that the in-flight packet matches a first packet filtering rule of the plurality of packet filtering rules; determining a disposition based on the threat context information; and applying the disposition to the in-flight packet. Non-limiting examples of threat context information may include one or more of: in-flight packet observation time, appliance location and / or appliance identifier / ID, administrator and / or associated security policy preferences, type of network protected and / or type of network associated with the in-flight packet, type of active threat or active attack associated with the in-flight packet, an indication of whether the in-flight packet is a member of an active multi-packet multi-flow attack (and / or information about such an attack), flow origin of the in-flight packet, flow direction of the in-flight packet, flow state of the in-flight packet, flow connection state of the in-flight packet, global threat context, a domain name associated with (e.g., identified by) the in-flight packet, popularity of the domain name, registration status of the domain name, a URI associated with the in-flight packet, a data transfer protocol method associated with (e.g., identified by) the in-flight packet, a protocol risk associated with the in-flight packet, and / or contextual CTI noise, etc.

[0017] Further aspects described herein are directed to receiving, by a packet filtering appliance, a plurality of packet filtering rules determined based on a plurality of threat indicators determined based on cyber intelligence reports from a plurality of cyber threat intelligence providers, configuring the packet filtering appliance with the plurality of packet filtering rules, receiving an in-flight packet from a first network destined for a second network, determining that the in-flight packet matches a first packet filtering rule of the plurality of packet filtering rules, where the first packet filtering rule does not indicate a predetermined disposition to be applied to the matching packet, determining threat context information based on determining, determining a disposition based on the threat context information, and applying the disposition to the in-flight packet. The threat context information may be based on various information available after the in-flight packet is observed, such as based on a time of observation of the in-flight packet.Non-limiting examples of threat context information may include one or more of: in-flight packet observation time, appliance location and / or appliance identifier / ID, administrator and / or associated security policy preferences, type of network protected and / or type of network associated with the in-flight packet, type of active threat or active attack associated with the in-flight packet, an indication of whether the in-flight packet is a member of an active multi-packet multi-flow attack (and / or information about such an attack), flow origin of the in-flight packet, flow direction of the in-flight packet, flow state of the in-flight packet, flow connection state of the in-flight packet, global threat context, a domain name associated with (e.g., identified by) the in-flight packet, popularity of the domain name, registration status of the domain name, a URI associated with the in-flight packet, a data transfer protocol method associated with (e.g., identified by) the in-flight packet, a protocol risk associated with the in-flight packet, and / or contextual CTI noise, etc.

[0018] Further aspects described herein are directed to determining in real time (and / or with low latency) a disposition and / or directive (and / or another type of action) for a packet being forwarded that matches one or more rules that do not include a predetermined disposition or that include a disposition other than allow dispositions and block dispositions, such as a “protective” disposition, and applying the disposition, directive, and / or other type of action to the packet being forwarded.

[0019] Further aspects described herein are directed to determining and applying dispositions and / or directives (and / or other types of actions) for in-flight packets based on information that is determined (e.g., calculated) and / or available (e.g., in real-time and / or with relatively low latency) after the in-flight packet is received and that was not determined and / or available prior to receiving the in-flight packet, which information may be different (and / or may be independently determined) from information received from another source (e.g., a CTI provider) prior to receiving the in-flight packet.

[0020] Further aspects described herein are directed to assigning a particular action, such as a particular disposition (such as a protective disposition or another disposition that is not allowed or blocked) and / or a particular instruction, to a rule based on a determination that the rule would potentially match a desired packet, such as a packet expected to be legitimate. The determination that the rule would potentially match a desired packet is based on CTI noise rejection and / or autoimmune information.

[0021] Further aspects described herein are directed to performing attack detection (such as detecting port scan attacks) based on multiple packet flows (e.g., multi-packet multi-flow attack detection). For efficiency, attack detection may be performed using efficient data structures, such as an LRU cache data structure, and / or an efficient attack packet rate estimator.

[0022] Further aspects described herein are directed to using global threat context information to determine dispositions and / or directives for packets in transit, which may be based on information provided by one or more other TIGs (or other types of packet filtering devices) and collected, consolidated, and / or distributed on a subscription basis.

[0023] Further aspects described herein are directed to using machine learning to determine dispositions and / or instructions for in-flight packets. For example, the determinations may be efficiently implemented by a machine learning configured artificial neural network (ANN) of a packet filtering device such as a TIG. The ANN may be configured to determine (e.g., calculate) dispositions and / or instructions in real-time after an in-flight packet is received by the TIG and before the next in-flight packet in the same direction is received by the TIG.

[0024] These and other features will be explained in the following detailed description with reference to the various drawing figures.

[0025] The present disclosure is pointed out with particularity in the appended claims. Features of the present disclosure will become more apparent upon consideration of the entire disclosure, including the drawings provided herein.

[0026] Certain features of the present specification are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which like reference numerals refer to like elements and in which: [Brief description of the drawings]

[0027] [Figure 1] 1 illustrates an exemplary environment for efficient threat-context-aware packet filtering. [Diagram 2] 11 is a flow chart illustrating an example use by a packet filtering appliance of threat context information associated with an active attack to compute dispositions and / or directives for packets in transit. [Diagram 3] 1 is a flow chart illustrating an exemplary use by a packet filtering appliance of threat context information associated with an active attack to calculate dispositions and directives for packets in transit. [Figure 4]1 is a flow chart illustrating an example use of threat context information to filter out cyber threat intelligence (CTI) noise and prevent autoimmune problems. [Diagram 5] 11 is a flow chart illustrating an example use of global threat context information to calculate dispositions and orders. [Figure 6] An exemplary artificial neural network (ANN) is presented that uses threat context information to efficiently compute in-flight packet dispositions and directives. [Figure 7] 1 illustrates an example computing device that may be used to implement any of the packet filtering appliances, other devices, systems, and methods described herein. [Figure 8] 1 illustrates an exemplary packet filtering appliance, such as a Threat Intelligence Gateway (TIG). [Figure 9] FIG. 1 is an example timing diagram according to aspects described herein. [Figure 10] 4 is another example timing diagram according to aspects described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] In the following description of various exemplary embodiments, reference is made to the accompanying drawings, which form a part of this specification and show, by way of example, various embodiments in which aspects of the present disclosure can be implemented.It should be understood that other embodiments can be utilized and structural and functional changes can be made without departing from the scope of the present disclosure.In addition, reference is made to specific applications, protocols, and embodiments in which aspects of the present disclosure can be implemented.It should be understood that other applications, protocols, and embodiments can be utilized and structural and functional changes can be made without departing from the scope of the present disclosure.

[0029] Various connections between elements are discussed in the following description. These connections are general and, unless otherwise specified, may be direct or indirect, wired or wireless, physical or logical (e.g., virtual or software-defined), in any combination. The specification is not intended to be limiting in this respect.

[0030] An important component of cyber defense is a packet filtering appliance that applies a set of packet filtering rules to in-flight L3 / IP packets traversing a network boundary and determines whether to allow / promote each packet towards its destination or block / drop the packet, i.e., determine the disposition of the packet. In this context, in-flight packets may be, for example, L3 / IP packets with source IP addresses corresponding to hosts upstream from the packet filtering appliance, or L3 / IP packets with destination IP addresses corresponding to hosts downstream from the packet filtering appliance. These network appliances may be inserted in-line on links located at the boundary between a private network, such as a corporate network, and the public Internet, and may be configured with a set of packet filtering rules designed to protect or otherwise defend the private network in some manner. These (in-line) network appliances may also be configured such that their network interfaces do not have L3 / IP addresses and / or L2 / MAC addresses, which may be referred to as a bump-in-the-wire (BITW) configuration; in a virtual environment, a physical BITW configuration may be emulated by a virtual BITW configuration. Thus, the set of packet filtering rules defines the (network security) policy, and the network appliance enforces the policy. U.S. Provisional Patent Application No. 63 / 071,174, filed August 27, 2020, and entitled "Methods and Systems for Efficient Virtualization of Inline Transparent Computer Networking Devices," is incorporated herein by reference for its disclosure of how a physical BITW configuration may be emulated by a virtual BITW configuration.

[0031] For example, early generation network firewalls and edge routers may be configured with packet filtering rules that enforce access control policies of private networks, such as which Internet services internal hosts are allowed to access, and conversely, which internal resources and services may be accessed by which (unauthorized) Internet hosts. Internet services and internal resources and services may be identified by their IP addresses, Transmission Control Protocol / User Datagram Protocol (TCP / UDP) ports, and / or protocol types, and thus the match criteria of packet filtering rules often correspond to the packet's 5-tuple values, i.e., (L3) source and / or destination IP address values, and / or (L4) source and / or destination port values, and / or (L3) protocol values ​​(IPv4) or next header values ​​(IPv6). Access control policies and their associated packet filtering rules are often static, in the sense that the rate of change of access control policies / rules is low enough that a human may manage them manually. Similarly, the number of rules in a typical policy is small enough that a human may manage them manually.

[0032] In another example, referring to FIG. 1 illustrating a representative network environment 100 of the present disclosure, one or more current generation packet filtering appliances, such as threat intelligence gateways (TIGs) 120a, 120b, 120c, may be configured with packet filtering rules. When any one of these TIGs 120a, 120b, 120c is referred to herein, or when they are referred to collectively, they are referred to herein simply as TIG 120 or TIGs 120. Furthermore, references to TIGs herein are merely examples, and it is understood that all references to TIGs herein are applicable to other types of packet filtering appliances. The packet filtering rules may identify packet matching criteria that correspond to network addresses and / or identifiers, such as IP addresses, 5-tuple values, domain names, URIs, etc., or indicators of cyber threats that may have been identified by a cyber threat provider (CTIP) 140 in an associated cyber threat intelligence (CTI) report. CTIP 140 may continually identify Internet threats, generate threat intelligence reports on the threats, determine indicators associated with the threats, and publish (e.g., stream) lists or feeds of threat indicators. The indicators may identify particular Internet hosts and / or particular resources controlled by Internet hosts.A subscriber to these feeds may be, for example, a Security Policy Management Server / Service (SPMS) 150, which may continually / iteratively consume multiple different feeds from multiple different CTIPs 140, aggregate relevant indicators (which may be in the millions), e.g., by removing duplicates and resolving overlapping address ranges, create a set of packet filtering rules (i.e., policies) with packet matching criteria corresponding to the threat indicators, rule metadata corresponding to the CTIPs 140 and feeds that supplied the indicators (and other relevant information about the threat, e.g., type of threat / attack, credibility, risk score, recommended disposition, etc.), dispositions of either “block” or “allow,” and directives, e.g., to log and / or flow log matching packets, capture matching packets, etc., and publish the policies to subscribers. A subscriber may be or may control / manage a packet filtering appliance (e.g., a Threat Intelligence Gateway (TIG) 120) that may be located at the boundary between private networks (e.g., 102, 104, ... 108) that are protected by threat intelligence and public networks such as the Internet 110 that may not be protected by threat intelligence, and may include an interface or gateway therebetween, and may receive policies and enforce policies by applying the policies to network traffic (e.g., packets in transit) that may pass through the TIG 120.

[0033] Due to the volume and dynamics of CTI provided by CTIP 140 (e.g., in aggregate, millions of threat indicators that are continuously updated and published at high rates, e.g., hourly or continuously as a stream), policy creation and associated threat indicator rule construction (e.g., by SPMS 150) is often an automated process. Thus, the selection of the disposition and one or more directives for each threat indicator rule is often performed automatically. Thus, the selection of a “Block” or “Allow” disposition may be automatically determined before a rule-matching packet is observed by TIG 120, and thus without taking into account any threat context that may be associated with the packet (e.g., time of observation, location, operating environment, current (local and global) threat situation, TIG operator / administrator policies, and associated (private) networks, etc.). Furthermore, in practice, due to the uncertainty of threat risk, the “Allow” disposition is often selected by default instead of the “Block” disposition, e.g., even when CTIP 140 may recommend the “Block” disposition with high confidence, so that legitimate traffic is not blocked. Also, different subscribers, e.g., different enterprises, may have different policies / requirements regarding the selection of "block" vs. "allow" and related directives, and these policies / requirements may change over time. A possible negative effect of pre-determining dispositions and directives at policy creation time is that network protection from threats / attacks may be significantly reduced.

[0034] This disclosure describes methods for improving network protection, for example, by calculating the best disposition and instructions to apply to a rule-matching in-flight packet to protect the network at the time the rule-matching in-flight packet is actually observed / filtered by TIG 120. At packet observation / filtering time, logic in TIG 120 uses current threat context information to calculate the disposition and instructions of the matching rule that can be applied to the observed in-flight packet. Threat context information may include, for example, local threat context information, which may be stored, for example, in a memory (e.g., main memory) of TIG120 and / or one or more efficient data structures, and may be readily available to and / or easily accessible by TIG120 and / or may be efficiently calculated by TIG120 (e.g., time / observation time, active attacks to which a packet may be associated, threat context information associated with matching rules, etc.), and / or global threat context information, which may be collected and disseminated, for example, by a global threat context system / service comprised of one or more Global Threat Context Servers (GTCS) 170. In order to reduce the possibility that TIG120 will have a detrimental impact on network performance (either all of the time or more than an acceptable amount), it may be desirable for threat context information to be available, accessible, computable, and / or otherwise determined for packets being forwarded by TIG120 with a speed (and efficiency) such that the threat context information can be determined (and possibly applied) before the next in-flight packet is received by TIG120 or before the next in-flight packet is processed by TIG120.For example, if packets are received at a rate of X packets per second, TIG 120 may be expected to be able to analyze each incoming packet and determine its associated threat context information in a time frame of 1 / X seconds or less per packet. It may be further desirable for TIG 120 to be able to further determine (e.g., calculate) the disposition of the packet and / or one or more directives within that same 1 / X time frame. Thus, in support of these high processing rates, it may be desirable that any information relied upon by TIG 120 to formulate threat context information for already received in-flight packets in real-time (e.g., by a scheduled deadline corresponding to the 1 / X time frame) can be easily and efficiently accessed, as described herein. For example, the information relied upon to formulate the threat context information may already be local to TIG 120 and / or may be at a remote location that is quickly accessible on demand, in real-time and with relatively low latency. Also, as described herein, exemplary processing structures that may be particularly suited for such high speed complex decisions (for processing threat context information and / or computing dispositions and / or one or more directives) may be (bounded) artificial neural networks, data structures with logarithmic or constant time complexity, task-efficient or task-optimized parallel processing algorithms, and related structures, etc. However, other time frames (e.g., greater than 1 / X) and other processing structures may be used as appropriate or desirable for the situation.

[0035] In operation, and with reference to the example of FIG. 1 , communications over a public network, such as the Internet 110, may occur between hosts connected to private networks 102, 104, ... 108 that may be protected by the TIG 120 enforcing network protection policies, and hosts connected to networks 130, 132, ... 138 that may be associated with threats. Note that hosts connected to TIG-protected private networks 102, 104, ... 108 may also be associated with threats (e.g., the hosts may be compromised by malicious actors), and any host connected to any network may communicate with any other host connected to any other network. Threat hosts may be associated with threat indicators that may be known to the CTIP 140. Thus, the TIG 120 may be enforcing policies that include packet filtering rules derived from threat indicators, e.g., packet filtering rules having matching criteria that correspond to the threat indicators.

[0036] When a packet in transit flows into the TIG 120 and matches a packet filtering rule, the TIG applies the rule's disposition to the packet, e.g., a block disposition (e.g., block or drop) or a permit disposition (e.g., permit or forward), and also applies logic associated with the rule's directive, such as "log," "flow log," "capture," or "spoof-tcp-rst." The rule's directive may be correlated to a disposition. For example, a "spoof-tcp-rst" directive, which may generate / spoof a TCP RST packet to terminate the associated TCP connection, may only be used with a "block" disposition (and only if the L4 / transport layer protocol is TCP). If "spoof-tcp-rst" is a directive in a rule with an "allow" disposition, the associated TCP connection may be terminated, which may not be the desired behavior and may be interpreted as an attack. In another example, for a rule with a "block" disposition, it may or may not be desirable to have a "capture" or "log" directive depending on the threat context. For example, consider a TIG observing a typical port scanning attack on a protected network, which may generate hundreds of thousands or millions of TCP SYN packets, at a rate of hundreds or thousands of packets per second. If a matching “block” rule includes a “capture” directive, each TCP SYN packet that constitutes the port scanning attack will be captured, which may use as much as 100MB of storage, but of little or no value to a cyber analyst who may be investigating the attack. Similarly, if a matching “block” rule includes a “log” directive, each TCP SYN packet that constitutes the attack will be logged, which may use even more storage space than a “capture” directive. A cyber analyst may need to observe only a few of the many packet logs to fully understand the attack in order to determine effective protective / defensive / corrective actions.Conversely, there are other types of attacks, e.g., some types of advanced persistent attacks (APTs), that may match a "block" rule, but each captured and / or logged packet may provide a lot of value to a cyber analyst investigating the attack. In either case, pre-determining the rule's disposition and directive without additional context, before a matching packet is observed, and therefore without current threat context (either when or after the in-flight packet is received), to guide the selection of the disposition and directive may be problematic and / or inefficient.

[0037] Furthermore, for some threats / attacks, the best disposition and directives may change during the threat / attack lifetime as the threat context changes. For example, considering a typical port scan attack on a network that may be protected by the TIG 120 of the present disclosure, this may include logic to (efficiently) detect port scan attacks as they occur. A typical port scan attack may send many TCP SYN packets at a high rate from the same source IP address towards many different ports on each (public) IP address of the target network. Assume that the port scan attack detection logic includes a packet arrival rate threshold that, when exceeded, may switch the associated threat context between "no active attack" and "active attack." Calculating the packet arrival rate includes observing at least two (arriving / receiving) packets, and for a given port scan attack, three or more packets may be observed before the threshold may be exceeded, which may cause the threat context to change. Thus, at or near the start of a port scanning attack, the best disposition may be "allow" for relevant packets that arrive before the threshold is exceeded and thus when the threat context is "no active attacks", but the best disposition may then change to "block" after the threshold is exceeded and the threat context changes to "active attacks". When the packet arrival rate falls below the threshold, the threat context may change to "no active attacks" and the best disposition may change to "allow".

[0038] Accordingly, this disclosure describes a new disposition, referred to herein as a “protect” disposition, and associated TIG 120 logic associated with the “protect” disposition. At the time a threat indicator rule is created / generated, e.g., when a CTI-based policy is being automatically created by SPMS 150, a “protect” disposition may be specified for the rule as an alternative to “block” or “allow”. Upon later observing an in-flight packet that matches a rule with a “protect” disposition, TIG 120 may use the threat context information to determine (e.g., calculate) a disposition for the in-flight packet, e.g., “block” or “allow”, and associated directive that best protects the network from the associated threat. The determined disposition may then be applied to the in-flight packet. Thus, the disposition of such in-flight packets may be undefined (e.g., unknown) before and until the packet is observed in-flight. Further, the disposition of such an in-flight packet may remain undefined (e.g., unknown) during the period from when TIG 120 determines that an observed in-flight packet satisfies a rule having a "protect" disposition until the disposition of the in-flight packet is subsequently determined based on the threat context information. The name "protect" for this disposition is merely an example, and any name may be assigned to this disposition as desired. Other non-limiting examples of names that may be used for this disposition include "protect," "guard," "undefined," "undetermined," "null," "flexible," "TBD," "other," "3," "ABC," and the like. Regardless of the name, this disposition may indicate a state in which threat context information is used to determine (e.g., calculate) the actual disposition to be applied to the in-flight packet in response to the observed in-flight packet. By way of example only, such a disposition is referred to herein as a "protect" disposition, regardless of the actual assigned name.

[0039] For example, the TIG120 logic may determine that even if an observed in-flight packet matches a threat indicator rule (e.g., a "protect" disposition), the computed disposition to be applied to the in-flight packet is "allowed" (and may be monitored, e.g., logged and captured for subsequent cyber analysis) because, for example, the in-flight packet is determined to be associated with legitimate business communications, or benign communications, or low-risk communications, or even with threat context information, there is still a lot of uncertainty about the threat, and thus the in-flight packet and associated communications may be allowed, but may be monitored and tracked to support subsequent cyber analysis. Note that in practice, the total time required to (a) access or compute any threat context information, (b) compute / select dispositions and directives to apply, and (c) apply the computed / selected dispositions and directives to the current in-flight packet must be short enough relative to the packet transmission rate so that the in-flight packet is not dropped by the packet filtering appliance (e.g., due to in-flight / arrival packet buffer overflow), which may result in a violation of the "Transparency Rule" of RFC2979. Conventional packet transmission rates may be measured in millions or tens of millions of packets per second on a single link. This means that the computation of threat context information should be very efficient, e.g., may have constant time (i.e., O(1)) or logarithmic time (i.e., O(logN)) complexity. Note that one skilled in the art may also expect that: (1) packet filtering rules follow the general schema described above and associated syntax and semantics, which may resemble, e.g., the schema of iptables or BSD PF; (2) rules in a policy are searched in the spatial order in which they appear in the policy file, i.e., from the top / head of the file to the bottom / tail of the file; and (3) TIG120's application of packet filtering rules is stateless (e.g., memoryless) in the following sense:(a) each in-flight packet is filtered through the policy in order of arrival, before the next in-flight packet in order of arrival is filtered through the policy (and has a disposition determined and applied), and (b) the disposition applied to a packet, e.g., block or allow, is not dependent on or related to the disposition applied to any preceding in-flight packets or to the disposition applied to any subsequent in-flight packets, and (4) the packet filtering appliance / TIG 120 (a) exits the appliance in the same order as the packets entered the appliance, and (b) any latency added by the appliance is negligible, e.g., the additional latency is a (small) fraction of the end-to-end packet transmission time between the host endpoints (e.g., one or more orders of magnitude less than the transmission time), the associated application is not affected by the latency, and packets are not dropped due to internal buffer overflow (e.g., the appliance behaves like a wire, or "bump-in-the-wire" (BITW), with respect to packet transmission). Those skilled in the art may refer to filtering packets as "stateless" packet filtering, regardless of how another packet is filtered. RFC 2979 "Internet Firewall Behavior and Requirements," RFC 2544 "Benchmark Methodology for Network Interconnection Devices," etc. may formalize some of these potential characteristics of packet filtering appliances that one skilled in the art might expect / assume.

[0040] For an in-flight packet that matches a threat indicator rule, (a) examples of threat context information that may be factored into the TIG 120 decision logic (hereinafter, "decision logic") for selecting dispositions and directives that best protect the network, and (b) examples of threat context information that may be efficiently accessed and / or otherwise determined (e.g., calculated) by the TIG 120 in response to the in-flight packet being observed (or otherwise after the in-flight packet is observed), may include, but are not limited to, one or more of the following, alone or in any combination or subcombination:

[0041] Packet Observation Time: The time at which an in-flight packet is observed by a packet filtering appliance (e.g., TIG 120) (e.g., when the in-flight packet is received, or when the in-flight packet is determined to match a rule, or when the in-flight packet is read). For example, the packet observation time may be the local time (and / or day of the week, date, month, season, etc.) at which the in-flight packet is observed by TIG 120. Additionally, a packet filtering appliance such as TIG 120 may determine whether the time of observation of the in-flight packet by TIG 120 occurs within a predetermined time period (which may be pre-determined prior to observing the in-flight packet), such as during normal business hours, weekends, holidays, and / or any other desired time period associated with TIG 120 and / or the owner / operator / administrator of TIG 120, e.g., an enterprise. For example, TIG 120 may determine an in-flight packet observation time that matches a rule, determine whether the in-flight packet observation time is within a predetermined time period, and determine (e.g., calculate) a disposition and / or one or more instructions for the in-flight packet based on the rule and / or based on whether the in-flight packet observation time is determined to be within the predetermined time period (e.g., disposition 1 (e.g., allow or block) if it is within the predetermined time period and a different disposition 2 (e.g., block or allow) if it is not within the predetermined time period), and the determined disposition and / or instructions may then be applied to the in-flight packet by TIG 120, and the disposition and / or one or more instructions may be determined and / or applied before the next in-flight packet is observed. As another example, TIG 120 may determine an in-flight packet observation time and determine (e.g., calculate) a disposition and / or one or more instructions for the in-flight packet based on the rule and / or based on the in-flight packet observation time (e.g., disposition 1 (e.g., allow or block) or a different disposition 2 (e.g., block or allow)), and the determined disposition and / or instructions may then be applied to the in-flight packet by TIG 120, and the disposition and / or one or more instructions may be determined and / or applied before the next in-flight packet is observed.

[0042] Appliance Location and / or Appliance Identifier / ID: The location of the packet filtering appliance (e.g., TIG 120) that observed the in-flight packet. The location may be, for example, a geopolitical location (e.g., which country, state, region, etc.), a time zone, a network location (e.g., network border or peering point, inside or outside a corporate security stack / network firewall, etc.). The appliance ID may be used alone (without a subscriber location) or to disambiguate between multiple appliances operating at the same or similar appliance locations. For example, in response to observing an in-flight packet that matches a rule, a packet filtering appliance such as TIG 120 may determine its own location and / or identifier, determine (e.g., calculate) a disposition and / or one or more directives to apply to that in-flight packet based on the rule, location, and / or identifier, and then apply that disposition and / or directive to that in-flight packet, which may be determined and / or applied before the next in-flight packet is observed. For example, if the location and / or identifier is a first location and / or first identifier, the disposition and / or instruction for the in-flight packet may include a first disposition (e.g., block or allow) and / or a first instruction, and if the location and / or identifier is a different second location and / or a different second identifier, the disposition and / or instruction for the in-flight packet may include a different second disposition (e.g., allow or block) and / or a different second instruction.

[0043] Administrator and / or associated security policy preferences: The enterprise or organization that owns, operates, manages, or otherwise controls the packet filtering appliance (e.g., TIG 120) that observed the packets in transit and / or its associated network. For example, different enterprises may have different enterprise security policies regarding threat indicators. For example, one enterprise X may consider a given threat indicator to be a significant threat while another enterprise Y may consider the same threat indicator to be low risk or benign. For example, enterprise X may be subject to US ITAR compliance / regulations while enterprise Y may not, or enterprise X may allow communication with anonymizing networks like Tor while enterprise Y may not. For example, the combination of (public) IP addresses and ports (and associated services) that an enterprise intends to be open / available for unsolicited internet communication. Additionally, an enterprise may change its security policies regarding a given threat indicator over time. For example, a packet filtering appliance such as TIG 120 may determine the administrator and / or (e.g., the administrator's) security policy preferences of TIG 120 in response to observing packets in transit that match a rule. TIG120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the in-flight packet based on rules, based on an administrator of TIG120, and / or based on security policy preferences, and then apply the disposition and / or directive to the in-flight packet, which may be determined and / or applied before the next in-flight packet is observed.

[0044] Network Type: The type of network associated with a packet filtering appliance (e.g., TIG 120) observing an in-flight packet (e.g., private network, public network (Internet), LAN, WAN, etc.). For example, a packet filtering appliance such as TIG 120 may determine the type of network associated with (e.g., protected by) TIG 120 in response to observing an in-flight packet that matches a rule. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the in-flight packet based on the rule and / or the type of network, and then apply the disposition and / or directive to the in-flight packet, which may be determined and / or applied before the next in-flight packet is observed.

[0045] Active Threat / Attack Type: The type of active threat / attack with which the observed in-flight packet may be associated (e.g., port scan, port sweep, leak, distributed denial of service (DDoS), spam, phishing, malware, etc.). See FIG. 3 and related discussion below for an example process for protecting a network by using this type of threat context information and the methods of the present disclosure. For example, a packet filtering appliance such as TIG 120 may determine an active threat / attack type with which the observed in-flight packet is associated in response to observing an in-flight packet that matches a rule. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to that in-flight packet based on the rule and / or the active threat / attack type, and the disposition and / or one or more directives may be determined and / or applied before the next in-flight packet is observed.

[0046] Whether the packet is a member of an active multi-packet, multi-flow attack: Determining whether the packet is a member of an active multi-packet, multi-flow attack by performing attack detection (e.g., detecting a port scan attack) based on multiple packet flows (e.g., multi-packet, multi-flow attack detection). For efficiency, attack detection may be performed using an efficient data structure, e.g., an LRU cache data structure, and / or an efficient attack packet rate estimator. See, e.g., the flowchart of FIG. 3. For example, a packet filtering appliance such as TIG 120 may determine whether the in-flight packet is a member of an active attack, such as an active multi-packet, multi-flow attack, in response to observing an in-flight packet that matches a rule. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the in-flight packet based on the rule and / or the type of network and / or based on whether the in-flight packet is a member of such an active attack, and then apply the disposition and / or directive to the in-flight packet, which may be determined and / or applied before the next in-flight packet is observed.

[0047] Multi-Packet, Multi-Flow Threat / Attack Analysis Results: Some systems analyze one or more (bi-directional) flows to determine threats and / or attacks associated with one or more flows consisting of one or more observed in-flight packets. Examples of such systems (generally referred to herein as "threat analysis systems") include Intrusion Detection / Prevention Systems (IDS / IPS), Network Behavior Analysis (NBA) systems, etc. Analysis results or output of such threat analysis systems may be threat context information that may be used by a packet filtering appliance to compute dispositions and directives for in-flight packets. Analysis results may be efficiently accessed by a packet filtering appliance that maintains, for example, a flow tracking table indexed by a hash of a 5-tuple value that (uniquely) characterizes each observed flow. Results / outputs of threat analysis associated with a flow may be posted to the flow tracking table. To ascertain whether an in-flight packet has a threat context based on these analysis results, the flow tracking table may be indexed by a hash value of the in-flight packet's 5-tuple value, and threat analysis results, if any, may be accessed for the flow associated with the in-flight packet. The hashing and indexing may be performed efficiently with constant time O(1) complexity using, for example, well-known algorithms. For example, a packet filtering appliance (e.g., TIG120) may observe an in-flight packet that matches a rule and determine whether the observed in-flight packet is part of a flow associated with an analysis result posted, for example, by an IDS (e.g., compare a hash of the in-flight packet's 5-tuple value to an index in a flow tracking table to determine the in-flight packet's flow and whether the determined flow is associated with the analysis result).Based on the rule and / or based on whether the flow of packets being forwarded is associated with the analysis result, the TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the packets being forwarded, and the disposition and / or one or more directives may be determined and / or applied before the next packets being forwarded are observed. For example, if the flow of packets being forwarded is determined to be associated with the analysis result, a first disposition and / or one or more directives may be determined and applied, and if the flow of packets being forwarded is determined to not be associated with the analysis result, a different second disposition and / or one or more directives may be determined and applied. As another example, if the flow of packets being forwarded is determined to be associated with a first analysis result, a first disposition and / or one or more directives may be determined and applied, and if the flow of packets being forwarded is determined to be associated with a different second analysis result, a different second disposition and / or one or more directives may be determined and applied.

[0048] Origination and / or direction of the flow: whether the flow associated with the observed packet originated from a public network (e.g., the Internet) or from within a private / protected network, and / or the direction of the packet. For example, a packet filtering appliance such as TIG 120 may determine whether the flow associated with the in-flight packet originated from a public network (e.g., the Internet) or from within a private / protected network, and / or the direction of the packet, in response to observing an in-flight packet that matches a rule. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the in-flight packet based on the rule and / or based on whether the flow associated with the in-flight packet originated from a public network (e.g., the Internet) or from within a private / protected network, and / or the direction of the in-flight packet, and then apply the disposition and / or directive to the in-flight packet, which may be determined and / or applied before the next in-flight packet is observed.

[0049] Flow state and / or connection state: e.g., whether a flow associated with an observed in-flight packet successfully established a TCP connection, the current number of transmitted bytes for the flow, etc. For example, a packet filtering appliance such as TIG 120 may determine a flow state and / or connection state associated with an observed in-flight packet in response to observing an in-flight packet that matches a rule. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the in-flight packet based on the rule, the flow state, and / or the connection state, and the disposition and / or one or more directives may be determined and / or applied before the next in-flight packet is observed.

[0050] Global Threat Context: Global threat status and awareness information, e.g., regarding threats / attacks that may be active or occurring recently on other networks other than the network protected by the (local) TIG 120. Contextual information associated with threats / attacks that are simultaneously attacking networks that are widely and globally distributed over the Internet, such as port sweep attacks. See the discussion associated with FIG. 5 for an example of how the global threat context may be used by TIG 120 to efficiently determine (e.g., calculate) a disposition and / or one or more directives for an observed in-flight packet. For example, a packet filtering appliance (e.g., TIG 120) may observe an in-flight packet that matches a rule, determine whether the in-flight packet is associated with an attack that may be occurring elsewhere (e.g., on one or more networks other than the local network protected by TIG 120), and determine (e.g., calculate) a disposition and / or one or more directives based on the rule and / or based on whether the in-flight packet is associated with such an attack, which may be determined and / or applied before the next in-flight packet is observed.

[0051] Threat characteristics of domain names and / or URIs (e.g., URLs or URNs): Any domain names and / or URIs contained in an observed packet may display lexical and / or syntactic features that may indicate a threat risk. For example, a large proportion of URL-encoded characters in a URI are often correlated with attacks and domain names that are randomly generated alphanumeric strings or are otherwise not well correlated with words in a human language, such as English words. The latter (i.e., correlation with words in a human language) may be efficiently calculated using, for example, an information entropy measure. For example, a packet filtering appliance such as the TIG 120 may determine a URI (e.g., a URL or URN) contained in the observed in-flight packet in response to observing an in-flight packet that matches a rule having a match criterion that is not a URI, and may determine whether the URI is associated with a threat. TIG120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the in-flight packet based on the rule and / or URI (e.g., based on whether the URI is determined to be associated with a threat), and the disposition and / or one or more directives may be determined and / or applied before the next in-flight packet is observed.

[0052] Domain Name Popularity: The popularity of a domain name that may be observed in an in-flight packet (e.g., as measured by the speed (e.g., average speed) of DNS requests to resolve the domain (e.g., current popularity). Generally / heuristically, the less popular a domain name is, the higher the threat risk may be associated with it. Databases of domain name popularity data and related services are readily and publicly available. The domain popularity data may be stored locally in efficient data structures and may be quickly / efficiently accessed by TIG logic for use as threat context information. For example, a packet filtering appliance such as TIG 120 may determine the domain name contained in the in-flight packet in response to observing an in-flight packet that matches a rule and determine the popularity of the domain name. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to that in-flight packet based on the rule and / or the domain name (e.g., based on the popularity of the domain name), and the disposition and / or one or more directives may be determined and / or applied before the next in-flight packet is observed.

[0053] Domain Name Registration Status: Domain names included in observed in-flight packets, but that are not registered in DNS, may be attack vectors. DNS registration domain name data may be stored locally in an efficient data structure and may be quickly / efficiently accessed by TIG logic for use as threat context information. For example, a packet filtering appliance such as TIG 120 may determine the domain name included in the in-flight packet and determine the registration status of the domain name in response to observing an in-flight packet that matches a rule. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to that in-flight packet based on the rule and / or the domain name (e.g., based on the registration status of the domain name), and the disposition and / or one or more directives may be determined and / or applied before the next in-flight packet is observed. U.S. Patent Application Publication No. 2020 / 0351245-A1, filed as U.S. Patent Application No. 16 / 399,700 on April 30, 2019, and U.S. Patent Application Publication No. 2020 / 0351244-A1, filed as U.S. Patent Application No. 16 / 692,365 on November 22, 2019, are both incorporated by reference into this specification for their teachings of examples of methods for efficiently determining whether a domain name is registered in the DNS.

[0054] Data Transfer Protocol Method: Observed packets that match a threat indicator and include a particular data transfer protocol method, e.g., a HyperText Transfer Protocol (HTTP) PUT, POST, or CONNECT method request, may indicate a malicious data transfer and thus may be threat context information used as input to the TIG logic. For example, a packet filtering appliance such as TIG 120 may determine a data transfer protocol method associated with (e.g., identified by) the in-flight packet in response to observing an in-flight packet that matches a rule. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the in-flight packet based on the rule and / or data transfer protocol method, and the disposition and / or one or more directives may be determined and / or applied before the next in-flight packet is observed.

[0055] Protocol Risk: Some (application layer) protocols associated with the observed packet may be insecure or easily exploitable by malicious actors. For example, Telnet, SSLv2, SSLv3, and older versions of TLS are known to be insecure. For example, a packet filtering appliance such as TIG 120 may, in response to observing an in-flight packet that matches a rule, determine a protocol (e.g., an application layer protocol) associated with (e.g., identified by) the in-flight packet. TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to that in-flight packet based on the rule and / or protocol, and the disposition and / or one or more directives may be determined and / or applied before the next in-flight packet is observed, and / or Contextual CTI Noise: In some cases, threat indicators in the CTI that are generally / globally considered to be associated with real threats may instead be considered by some locations / administrative domains to be associated with legitimate, (very) low-risk, and / or benign communications. Such indicators are referred to herein as “CTI noise.” Depending on the local context, such indicators may be excluded by some administrative domains from the CTI used to protect the network. For example, in some scenarios, public / WAN IP addresses assigned to the border / interface between private network X and the Internet may be included in the globally distributed CTI as threat indicators. These threat indicators may correspond to real threats for many / most subscribers, but may correspond to legitimate traffic of private network X. Thus, an administrator of private network X may want to exclude these indicators from the CTI used to protect private network X. Such an action is referred to herein as performing CTI noise exclusion. Thus, these indicators are threat context information for private network X and may be factored into the logic of any TIG protecting private network X to achieve CTI noise exclusion. For example, a packet filtering appliance such as TIG 120 may, in response to observing an in-flight packet that matches a rule, determine whether a threat indicator associated with the in-flight packet is excluded (e.g., determined to be included in a CTI noise data set). TIG 120 may determine (e.g., calculate) a disposition and / or one or more directives to apply to the in-flight packet based on the rule and / or whether the threat indicator is excluded, and the disposition and / or one or more directives may be determined and / or applied before the next in-flight packet is observed.For example, if the threat indicator is determined to be excluded, the disposition may be determined to be "allow," and if the threat indicator is determined to not be excluded (e.g., not included in the CTI noise data set), the disposition may be determined to be "block." See FIG. 4 and related discussion below for an explanation of how this threat context information is used to calculate dispositions and directives that will best protect the network from threats / attacks.

[0056] The additional information may be used to help calculate the disposition and / or directive of the packet during forwarding. Examples of such additional information include, but are not limited to, the following: CTI Provider and / or Associated Information: The CTI provider that supplied the threat indicator for the matching threat indicator rule, as well as associated information such as the CTI feed name / identifier and associated information, the threat / attack type / category associated with the threat indicator, the quantity and / or identity of the CTI provider that provided the threat indicator for the CTI provided by the provider, etc. Such information may be included in the <metadata> of the threat indicator rule and may be used as input to the TIG120 logic when a rule matching in-transit packet is observed. This information may be included at the time of rule creation. <metadata>Note that when included in a rule, it may be available before a matching in-flight packet is observed / filtered, and thus this information may not strictly be considered threat context information in and of itself, but may be combined with threat context information, for example, by logic determining the disposition and / or directive of an observed in-flight packet, to provide context for the threat context information.

[0057] Threat Indicator Type and / or Fidelity: The fidelity of a threat indicator correlates with the type of threat indicator, for example (and listed in order of increasing fidelity): URL / URI, Fully Qualified Domain Name (FQDN), domain name, 5-tuple, IP address, subnet address range, etc. Threat indicators may also include certificates and certification authorities used to secure communications, such as TLS-secured communications. Indicators with indicator type value "URL / URI" may have the highest fidelity because a URL maps to a single networked (malicious) resource (e.g., a malware executable), i.e., there is no uncertainty regarding the threat risk / maliciousness of a URL / URI type indicator. On the other hand, threat indicators with indicator type value "FQDN" may have lower fidelity than "URL / URI" because a single FQDN may map to multiple URLs or multiple networked resources, some of which may be non-malicious. Thus, there may be some uncertainty regarding the threat risk / maliciousness of an FQDN type indicator. Similarly, threat indicators with an Indicator Type value of "IP Address" have lower fidelity than "FQDN" because a single IP address can map to multiple domains / FQDNs, some of which may be benign. For example, domain hosting services often have many domains associated with a single IP address, and only a small / small percentage of the domains may be associated with threat activity. Similar to the CTI provider information above, threat indicator type and fidelity information is provided during rule creation. <metadata>rules. Thus, this information may not strictly be considered threat context information in and of itself. However, it may be combined with threat context information by logic that determines the disposition and directives of packets in transit, and / or Threat indicator longevity: In general, a threat indicator that has been identified more recently may be considered to pose a higher threat risk than an older threat indicator. The longevity of an indicator is determined by: <metadata>It may be included in the rule or may be associated with a local efficient data structure containing recently registered domain names that can be accessed by the TIG logic, for example, when an observed in-flight packet contains the domain name. As with the CTI provider information above, indicator lifetime information may be added at rule creation time. <metadata>This information may be included in rules and therefore may not strictly be considered threat context information in and of itself, however it may be combined with threat context information by logic that determines the disposition and directives of packets in transit.

[0058] Any of the above threat context information and / or other information, in any combination or sub-combination, as well as other types of threat context information that may not be listed above, may be efficiently accessed and / or otherwise determined (e.g., calculated) by TIG120 and used by TIG120 logic to determine the disposition and / or one or more instructions of a packet in transit observed by TIG120.

[0059] As described above, a new disposition (e.g., referred to herein as a "protect" disposition) may be added to the packet filtering rule syntax / schema to signal the packet filtering appliance (e.g., TIG 120) to use the threat context information to calculate dispositions and directives for observed in-flight packets. For example, a threat indicator 12.34.56.00 / 24 ​​(i.e., a subnet address prefix covering the range of Internet Protocol version 4 (IPv4) addresses [12.34.56.00, 12.34.56.255]) may be provided by the CTI provider 140 with an intelligence report that associates the subnet address range 12.34.56.00 / 24 ​​with some malicious activity, including port scanning, but there may also be legitimate traffic associated with 12.34.56.00 / 24. Thus, an enterprise may want to block malicious activity but allow legitimate activity (e.g., for business activity). However, using this single threat indicator and traditional packet filtering appliances (i.e., without the threat context information processing capabilities of the present disclosure), businesses seeking to protect their networks must choose between enforcing the rule "block log 12.34.56.00 / 24" and thereby risking loss of legitimate business communications, or enforcing the rule "allow log 12.34.56.00 / 24" and thereby risking business damage from a cyber attack.

[0060] As described throughout this disclosure, with a threat-context-aware packet filtering appliance (e.g., TIG 120) of the present disclosure, an enterprise may instead protect its network from unprotected resources (e.g., an unprotected network such as public network 110). The threat-context-aware packet filtering appliance may use one or more rules, and in response to determining that one or more of the rules apply to an observed in-flight packet, the threat-context-aware packet filtering appliance may determine threat context information associated with the in-flight packet, determine (e.g., calculate) a disposition and / or one or more directives based on the threat context information, such as by using logic associated with the one or more rules, and apply the calculated disposition and / or one or more directives to the in-flight packet. Moreover, determining the disposition and / or directive (and potentially also applying the disposition and / or directive) may be efficiently completed before the next in-flight packet is received (e.g., observed) by the threat-context-aware packet filtering appliance. For example, consider the rule "protect 12.34.56.00 / 24". The TIG 120 logic associated with a rule may be configured, for example, as follows: "If a matching packet for this rule is observed during normal business hours [or some other time frame], then 'Allow, log, and capture'. If a matching packet for this rule is observed outside of normal business hours and the packet may be associated with a current port scanning attack, then 'Block and log'." In this example, the time of in-flight packet observation, the result of comparing "normal business hours" to the time of in-flight packet observation, and the determination of whether the observed in-flight packet is associated with a current port scanning attack may each be threat context information that was not available to TIG 120 before the in-flight packet was observed by TIG 120.Other TIG120 logic may apply any other types of threat context information in any other combinations or sub-combinations, as desired, based on one or more matching rules.

[0061] The directives determined and applied by the threat context-aware packet filtering appliance may be of any relevant type. For example, threat context-aware logging directives may be used, which further improve the efficiency and effectiveness of CTI-based cyber analysis and defense. Traditional logging directives may be used, including, for example, basic "log" directives that create a log of a single packet, and "flow log" directives that aggregate logs of packets from the same flow (i.e., same bidirectional 5-tuple) into a single log of the flow. In the context of CTI-driven network protection, where logs may be used to improve network protection, these basic packet log and flow log directives may be ineffective and inefficient in some scenarios. For example, a typical port scanning attack against an enterprise network perimeter may involve thousands or millions of packets in a relatively short period of time, generating a log for each packet. Furthermore, because each packet in a port scanning attack may have different 5-tuple flow characteristics, flow logging may not significantly reduce the number of logs and / or the amount of log data. Thus, for example, a log directive may be implemented that collects statistical / aggregate / cumulative data on correlated packets (e.g., packets containing the same port scanning attack) and generates a single log of many packets and / or flows in such a manner that a single log of the attack is much more effective for cyber analysis and defense than the many packet logs and / or flow logs that would otherwise be generated. Similarly, a log directive designed for a particular cyber attack, such as the exemplary port scanning attack described above, may be implemented to generate a single log for the attack incident that has high information value for cyber analysis and defense. Examples of such log directives are described in U.S. Provisional Patent Application No. 63 / 106,166, filed October 27, 2020, and entitled "Methods and Systems for Efficient Adaptive Logging of Cyber ​​Threat Incidents," which is incorporated herein by reference for the disclosure of the above log directives.These log directives may be determined (e.g., calculated) based on any or all of the threat context information collected or otherwise determined by TIG 120 for one or more observed in-flight packets.

[0062] The sheer number of potential sources and types of threat context information, and the many possible combinations, may make it impractical for a human to design an accurate and efficient decision logic. In this scenario, depending on data rates, TIG capacity, and / or other factors, machine learning may be desirable or necessary to design some or all of the decision logic. For example, a machine learning artificial neural network may be created having multiple input nodes corresponding to sources of threat context information and information derived from the in-flight packets being filtered, and multiple output nodes corresponding to dispositions and directives to be applied to the in-flight packets. The neural network may be created in such a manner that the decision logic is highly efficient (e.g., has constant time complexity), for example as a bounded-depth classifier. In addition to or as an alternative to artificial neural networks, other machine learning algorithms and methodologies, for example, evolutionary algorithms, genetic algorithms, genetic programming, etc., may be used to design the decision logic.

[0063] Those skilled in the art will appreciate that in some embodiments of the present disclosure, the "protect" disposition may be treated as a procedure call by the TIG application logic and thus may be parameterized with variables, for example, to specify default values ​​for the disposition and directives and / or to specify which threat context information may be used to compute the disposition and directives, etc. For example, "protect(disposition="allow",directives="log,capture",threat-context::"port-scan-attack,time-of-day")" may signal the TIG application logic to compute the disposition and directives using threat context information associated with the port scan attack detection logic and the current time (which may correspond to the observation time of the packet in transit). Additionally, default values ​​for the disposition and directives may be used, for example, when the TIG logic for computing the disposition and directives is indeterminate, which may occur, for example, when the TIG is running a version of the application logic that may not support the specified threat context logic. The TIG application logic may also treat the dispositions "block" and "allow" as procedure calls that may implicitly execute logic that may transparently use some threat context information to cause useful / desired side effects when the "block" or "allow" action is applied to a packet. A specific example of such logic is described below in connection with FIG. 4.

[0064] As an example of using multiple types and combinations of threat context information available only at the time, location, and environment of in-flight packet observation to calculate in-flight packet dispositions and directives, consider the following: A key problem associated with protecting networks using cyber threat intelligence (CTI) that can be addressed by using the threat context filtering method of the present disclosure is dealing with port scanning attacks (as well as some attacks that have similar characteristics to port scanning attacks, e.g., some DDoS attacks such as port sweep attacks, reflected spoofing attacks, etc.). In a typical port scanning attack, a malicious actor may use a port scanner application to search a target / victim network for any services that may be "open," i.e., accessible by Internet hosts at L3 (IP) and accepting connections at L4 (TCP), and therefore potentially exploitable or attackable. Those skilled in the art may refer to such malicious activity as (cyber) reconnaissance. For each public IP address of the target network, e.g., 256 contiguous IP addresses that form a / 24 IPv4 subnet address block, there may potentially be 48K known or registered ports open. For example, an enterprise may host its public-facing web server on port 80 (a well-known port for HTTP services) and / or port 443 (a well-known port for HTTP Secure (HTTPS)) of a public IP address assigned to the enterprise. A port scanner typically sends a TCP SYN handshake flag to a given {IP address, port} pair. If the {IP address, port} responds with a TCP SYN-ACK handshake flag, the port scanner knows that the {IP address, port} pair is "open," i.e., accepting (unsolicited) TCP connection requests. This information may then be used to attack or otherwise exploit the service associated with the open port.During a typical attack on a target network, a port scanner can send hundreds or thousands of TCP SYNs to different {IP address, port} pairs per second for several minutes.

[0065] To avoid attribution, for example, a malicious actor may compromise an otherwise legitimate host computer connected to an otherwise legitimate corporate network, such as network 130 of Figure 1, and install a port scanner program on the host computer. To avoid notification of the attack on the target's business operations and / or to mitigate the adverse impact of the attack on the target's business operations, the malicious actor may also launch a port scanning attack during non-business hours of the enterprise operating the target network.

[0066] CTI provider 140 may determine that a (compromised) host computer connected to network 130, e.g., host C, is the source of port scanning attacks and / or other malicious activity, and may include one or more IP addresses associated with host C in a CTI feed that is exposed to subscribers. For example, host C may be assigned one or more public IP addresses associated with the boundary of network 130 and the Internet 110. For example, an ISP for network 130 may assign the IPv4 subnet address range 22.22.22.00 / 24 ​​to network 130, and a network address translation (NAT) device at the boundary may temporarily assign any of the 256 IPv4 addresses in 22.22.22.00 / 24 ​​to host C when host C is conducting Internet communications, including when host C is conducting port scanning attacks. Thus, a CTI provider may include one or more IP addresses from 22.22.22.00 / 24 ​​as a threat indicator in one or more CTI feeds, and / or may include the subnet address prefix 22.22.22.00 / 24 ​​as a single threat indicator in one or more CTI feeds.

[0067] During policy construction, and for systems not enabled with the disclosed methods / techniques, such CTIs / threat indicators are often translated into packet filtering rules with "allow" dispositions (and "quick", "log", and / or "flow log" directives) instead of "block" dispositions. This is because enterprises that protect their networks with such CTIs and associated packet filtering rules may not want to take the risk of blocking / dropping legitimate business traffic, especially during normal business hours. Thus, enterprises may detect such port scanning attacks (by "log" and / or "flow log" directives and associated log analysis / threat awareness applications), but the malicious actor's mission is accomplished (e.g., open ports may be known by the malicious actor and associated services may be characterized by the malicious actor). In addition, the associated logs generated by the port scanning attacks may flood / overwhelm any log analysis / threat awareness applications and associated resources (e.g., computational resources, storage, network bandwidth, and / or cyber analysts operating the applications).

[0068] 2 is a flow chart illustrating an example use by a packet filtering appliance (e.g., TIG 120) of threat context information associated with an active attack to compute in-flight packet dispositions and / or directives. In step 2-1, TIG 120 may be configured with threat context filtering logic and / or other information that may be used to compute in-flight packet dispositions and / or directives. This configuration step may be performed before TIG 120 comes online in its network and / or as part of an update after TIG 120 is already online. As described below, this configuration may include traditional threat context filtering logic as well as other types of logic, such as configuration of TIG 120's artificial neural network classifiers.

[0069] Next, in step 2-2, the SPMS 150 may receive one or more CTI feeds provided by one or more CTIPs 140.

[0070] In step 2-3, the SPMS 150 may aggregate relevant threat indicators based on those one or more CTI feeds and create one or more sets of packet filtering rules (which may be associated with one or more policies) having packet matching criteria corresponding to the threat indicators. The rules in the policies may be further associated with various dispositions. For example, at least some of the rules in a given policy may be associated with a “protect” disposition, while other rules in the policy may be associated with an “allow” or “block” disposition. In a further example, the rules may be associated with information indicating that the disposition is determined at the in-transit packet observation time. For example, as previously discussed, the rules may have or otherwise be associated with a value (such as a flag) indicating whether the disposition is calculated at the in-transit packet observation time using threat context information. For example, the rules may have a field that includes a flag, where a flag value of 0 means “use the rule's predefined disposition” and a flag value of 1 means “calculate the rule's disposition at the in-transit packet observation time.” More generally, each rule may be associated with a pre-defined disposition that is applied by default at the in-flight packet observation time, or may be associated with an indication (such as a "protective" disposition, or a null disposition, or a flag as described above) that the disposition is calculated at the in-flight packet observation time and / or is not pre-defined prior to the in-flight packet observation time.

[0071] In steps 2-4, SPMS 150 may publish the policy to its subscribers, which may include TIG 120. Thus, TIG 120 may receive the policy from SPMS 150, which may include one or more rules associated with various dispositions, as described above. TIG 120 may use the received policy to self-configure so that it can enforce the policy on future observed packets in transit.

[0072] In step 2-5, TIG 120 may receive a next in-flight packet P from another network, such as, for example, network 110. For example, FIG. 9 shows a series of in-flight packets P0, P1, etc. received by TIG 120. The next in-flight packet P in step 2-5 may be considered to be, for example, packet P0 in FIG.

[0073] In step 2-6 of FIG. 2, TIG 120 may perform a lookup of its policies (received and configured in step 2-4) to ascertain which rules may match (apply) to the in-flight packet P. If TIG 120 determines that the in-flight packet P matches a rule associated with a "block" or "allow" disposition, and / or is not otherwise indicated as being associated with a subsequent disposition computation, TIG 120 may apply the predetermined disposition identified by the rule. This is shown as an example in FIG. 9, where a rule match may result in an immediately determined disposition (and / or directive) indicated by the matching rule, which may be applied to the in-flight packet P0. However, if TIG 120 determines that the in-flight packet P matches a rule associated with a "protect" disposition or is otherwise associated with an indication that the disposition (and / or other action to be applied to the in-flight packet) is calculated at the in-flight packet observation time and / or is not predetermined prior to the in-flight packet observation time, steps 2-7 and 2-8 (FIG. 2) may be performed and processing may return to step 2-5 to receive the next packet.

[0074] In steps 2-7, in response to the in-flight packet P and / or in response to a determination that the in-flight packet P matches a rule associated with a “protect” disposition (or is otherwise associated with a computation of a disposition at the in-flight packet observation time), the TIG 120 may determine threat context information associated with the in-flight packet P and / or associated with the rule. The determined threat context information may include one or more types of threat context information, such as any type of threat context information described herein (e.g., observation time, appliance location, etc.). The TIG 120 may determine (e.g., calculate) a disposition (e.g., “allow” or “block”) and / or one or more instructions (e.g., “log”, “quick”, etc.) based at least in part on the threat context information. The calculated disposition and / or instructions may be determined to best protect the network associated with the TIG 120 (e.g., network 102 associated with the TIG 120a). This is shown as an example in FIG. 9, where if the rule matching process determines that an in-flight packet (in this example, P0) matches a "protect" disposition rule or is otherwise associated with a disposition calculation at the in-flight packet observation time, a disposition, directive, and / or some other action to be applied to the in-flight packet may be calculated based on threat context information associated with the in-flight packet P0, and the calculated disposition, directive, and / or other action may be applied to the in-flight packet P0.

[0075] In the example of FIG. 9, the entire processing of a given in-flight packet, from rule match to computing the disposition and / or directive and applying the computed disposition and / or directive, may all be completed by TIG 120 before the next in-flight packet (P1) is received by TIG 120. However, other timings may be feasible. For example, processing up to computing the disposition and / or directive may be completed before TIG 120 receives the next in-flight packet P1. As another example, the disposition and / or directive for in-flight packet P0 may be partially or completely computed before the disposition and / or directive is computed or otherwise determined for the next in-flight packet P1, even though the disposition and / or directive for in-flight packet P0 is not applied prior to observing the next in-flight packet P1. An example of the latter timing is shown in FIG. 10. Such timing may function efficiently, for example, if TIG 120 can compute a disposition and / or directive (and / or application of a disposition and / or directive) for an in-flight packet (e.g., P0) simultaneously (e.g., in parallel or simultaneously) with performing a rule match for the next in-flight packet (e.g., P1). This timing may be feasible, for example, if different (and / or independent) processing resources are used for the rule match as compared to processing resources used for computing the disposition and / or directive based on threat context information and / or if computing the disposition and / or directive for the in-flight packet P0 does not interfere with the rule match for the next in-flight packet P1 (and vice versa).

[0076] In steps 2-8 of FIG. 2, TIG 120 may apply the calculated disposition and / or the calculated one or more instructions to packet P in transit. Processing may then return to steps 2-5 to receive the next packet (e.g., packet P1 of FIG. 9 or FIG. 10). In some examples, the processing of FIG. 2 may be performed in a pipelined manner, one step at a time. In further examples, the processing of FIG. 2 may be performed where the execution of one or more steps may overlap in time and / or may be simultaneous.

[0077] The disclosed methods and techniques that take into account threat context information when calculating dispositions and / or directives at in-transit packet observation time may be used to protect networks in the above scenarios and / or other (e.g., similar) scenarios. For example, in the above scenario involving a port scan attack, the relevant threat context information associated with the in-transit packet at the observation time may include other threat context information such as (a) whether the packet is part of or includes an active port scan attack, (b) whether the observation time is during normal business hours, and (c) which combination of IP addresses and ports (and associated services) the enterprise intends to be open / available for unsolicited Internet communications. For example, assume that (in-transit) packet P0 matches rule R0 with a "protect" disposition, and that the associated TIG120 logic for calculating dispositions and directives takes into account the above threat context information (a) and (b). An example of such TIG120 logic written in pseudocode as a procedure may be as follows: Compute-Disposition-and-Directives(in packet:P0,out string:Disposition-and-Directives): IF(Member-Active-Port-Scan-Attack(P0)AND NOT(Normal-Business-Hours(current-time()))) THEN Disposition-and-Directives:="block,log,quick"; ELSE Disposition-and-Directives:="allow,log,quick"; End Compute-Disposition-and-Directives;

[0078] With reference to the pseudocode above, the procedure Compute-Disposition-and-Directives() accepts as input a (in-flight) packet named P0 that matched a "protect" rule associated with the CTI, and outputs a string Disposition-and-Directives that contains the computed disposition and directives that may be applied to P0 to best protect the network. The Boolean function Member-Active-Port-Scan-Attack(P0) determines whether packet P0 is part of an active port scan attack. The Boolean function Normal-Business-Hours() determines whether the input time, which is the function call current-time() in the exemplary pseudocode above, occurs during normal business hours (which may be locally configurable). Overall, the procedure in this example combines multiple types and values ​​of threat context information that may only be available at (and not before) the observation or filtering time of the in-flight packet to compute dispositions and directives that will best protect the network from threats associated with the in-flight packet that matched a packet filtering rule derived from the CTI.

[0079] As discussed above, when taking threat context information into account and incorporating it into logic for computing in-flight packet dispositions and / or directives, it may be desirable that the threat context information and associated TIG logic be computed efficiently, e.g., such that (time) latency resulting from threat context information determination (e.g., computation) does not adversely affect the packet processing performance of the packet filtering network appliance (e.g., TIG 120), thereby causing a violation of the "Transparency Rule" of RFC 2979. For example, (relatively) high latency may overflow in-flight packet buffers of the packet filtering network appliance, thereby causing packet drops, which may affect the performance of associated networked applications. Referring to the above example, the threat context information (a) and (b) would be, namely, (a) "Is the packet a member of an active port scan attack?", which may be implemented as the function Member-Active-Port-Scan-Attack() above, and (b) "Is the packet observed during normal business hours?", which may be implemented as the function Normal-Business-Hours() above, and thus the threat context information (a) and (b) would be efficiently calculated. Those skilled in the art would know how to efficiently calculate a function such as (b) / Normal-Business-Hours() or other similar time-based functions, for example, by accessing a timestamp() function in TIG 120, which may be available in a software development kit, e.g., Data Plane Development Kit (DPDK), designed to efficiently process TCP / IP packets in transit. Accessing a time function, which may be a system call to the relatively slow operating system kernel of TIG 120, could also generate this information, although this may be a less efficient (and therefore potentially impractical) way of doing so.However, one of ordinary skill in the art may not know how to efficiently compute (a) / Member-Active-Port-Scan-Attack() and certain functions similar to (a), e.g., when the (in-transit) packet is a member of an active port scan attack and / or is a member of an active attack that may be structurally similar to a port scan attack, e.g., a port sweep attack.

[0080] Therefore, we now describe an exemplary efficient method for computing (a) / Member-Active-Port-Scan-Attack() and functions similar to (a). The exemplary method includes, for each attack type of interest: (1) Characterizing the structure (e.g., architecture) and related information of an attack type; (2) Identifying attack information or keys that can uniquely characterize each instance of an attack type and that can be used as or correspond to an index and / or unique identifier of a set of (potential) attack instances (wherein an attack instance can be considered an element of the set); (3) Creating an (efficient) set data structure for managing (potential) instances of an attack type, where each distinct instance of an attack type, or equivalently each element in the set, is identified by / associated with its (unique) key. The set data structure may be associated with at least an operation Insert(key,element) that inserts a new element identified by the key, e.g., a new (potential) attack instance, into the set, and an operation Member(key) that tests whether an element already exists in the set corresponding to the key. Furthermore, for attack types that consist of multiple packets, such as port scan attacks, an additional (Boolean) operation Is-Active-Attack(key) may be associated with the set data structure. For example, for some attacks that consist / may consist of multiple packets, a potential attack instance may not be determined to be an active / actual attack until certain multi-packet criteria are met. For example, multiple different ports are being scanned, and only after the attack packet rate exceeds a threshold, a potential port scan attack instance, i.e., an actual potential attack instance, may be determined to be active. Both criteria in this example require that multiple packets associated with the (potential) attack instance have been observed, (4) configuring and operating the TIG with logic for determining whether a packet being forwarded is part of or contains an instance of an (active) attack;

[0081] For illustrative purposes, the above methodology, which may be applied to a port scanning attack but may be readily adapted to other types of attacks, may be described as follows.

[0082] Regarding (1), characterizing the structure or architecture and related information of a single, separate instance of a (typical) port scanning attack on a target network protected by TIG, which may include: - Inbound packets (to TIG and protected networks) containing the TCP SYN flag, - packets originating from the same Internet IP address and / or the same (smaller) subnet address range (e.g., the same / 24, / 28, or / 32 IPv4 subnet address prefix); - the packets are destined for multiple different public IP addresses of the target network and / or multiple different ports; and / or --The packets are sent at a rate that exceeds a (heuristic, configurable, and / or predetermined) threshold, such as 10 packets per second, although other thresholds may be used.

[0083] Thus, for example, a collection of information that may (effectively) describe an instance of a (potential) port scanning attack may include any one or more of the following, in any combination or subcombination: - The source IP address or subnet address prefix, which may be identified / associated with a key that uniquely characterizes or represents the attack; - the current number of packets that make up the attack, The number of different ports scanned by the attack, and a (configurable and / or predetermined) threshold that may determine that a potential attack may be an active port scanning attack (e.g., in practice the threshold may be four different ports, although other thresholds may be used); - the number of different destination IP addresses scanned by the attack, - the current rate of the attack (e.g., packets / second), and a (configurable and / or predetermined) threshold that may determine that the attack may be an active port scan attack, e.g., in practice the threshold may be 10 packets per second, but other thresholds may be used. The estimate of the current rate may be calculated efficiently, e.g., by using logic based on the Exponentially Weighted Moving Average (EWMA) method, or logic based on the Exponential Smoothing method. The Exponential Smoothing method is time and space efficient, since each update to the estimate after each (attack) packet is received requires only a constant time O(1) computation (typically two multiplications and one addition), and typically only three values ​​need to be stored at any time (per managed attack, e.g., per potential attack in each set data structure for managing attacks of a given type); The start time of the attack, e.g., the observation time of a particular (e.g., the first) packet that contains the attack; - a unique identifier for a (potential) port scanning attack instance, - and / or similar.

[0084] Regarding (2), identify attack information or a key that can uniquely characterize each instance of the attack: In the case of a typical port scanning attack, the unique key could be the source IP address of the (inbound TCP SYN) packet, or an associated (smaller) subnet address range, such as a / 24 or / 28 IPv4 subnet address prefix that corresponds to the source IP address of the packet.

[0085] With regard to (3), creating an (efficient) set data structure for managing (potential) instances of port scanning attacks, an exemplary efficient set data structure may be a least recently used (LRU) cache. The LRU cache may be implemented as a (bounded) doubly linked list with an associated hash map for efficient indexing into the linked list. The elements of the set are instances of potential port scanning attacks, which may be represented by the information described above in (1) and uniquely identified / associated by a key described above in (2). Set operations / functions for a set data structure such as an LRU cache may include Insert(key,element), a procedure that may insert a new (potential) port scanning attack instance or element identified by the key into the set of (potential) port scanning attacks; Member(key), a function that determines whether the (potential) attack uniquely represented / identified by the key is an element in the set; and Delete(key), a function that removes the associated element from the set. For an LRU cache, both Insert() and Member() may have constant time complexity, i.e., O(1) complexity, and thus may be examples of efficient operations. For a (bounded) LRU cache, the Delete() function may also have O(1) complexity, but may not be transparent / implicit / exposed, since elements are automatically removed from the LRU cache when the associated set reaches its size limit and when the element is the oldest or most recently least recently used (LRU) element in the set. In this way, attacks that have been terminated, subsided, or otherwise become dormant may be efficiently removed from the set of potential and / or active attacks.For the example port scan attack, where a single instance may consist / be made up of multiple packets and / or multiple packets may be required to determine if an attack instance is currently active, an additional input parameter named “packet” may be added to both the Member(key,packet) function and the Insert(key,packet,element) procedure associated with the set, using the following example semantics / logic:

[0086] Member(key, packet) searches the set for an element that matches the key (e.g., the source IP address of the packet), e.g., an instance of a (potential) port scan attack. If there is no match (e.g., there is no element / port scan attack instance in the set to which the packet corresponds / associated), it returns false. If there is a match (e.g., there is an element / port scan attack instance in the set to which the packet corresponds / associated), it (1) merges the packet with the matching instance, e.g., increments the current number of packets that constitute the attack, updates the number of distinct ports that constitute the attack if necessary or appropriate, updates the number of distinct destination IP addresses that constitute the attack if necessary or appropriate, updates the current rate of the attack, etc., and (2) returns true. Note that the packet merge computation can be sufficiently efficient, e.g., has O(1) complexity, such that the overall efficiency of the computation of dispositions and directives is not reduced.

[0087] Insert(key,packet,element) inserts a new (unique) element, e.g., a new instance of a (potential) port scanning attack, which may be identified by a key and initialized with a packet. For example, for the new element / new attack instance, the source IP address or subnet address range associated with the new attack instance may be set to the key, which may be, e.g., the source IP address of the packet or the corresponding subnet address prefix, the current number of packets constituting the attack may be initialized, e.g., to 1, the number of different ports scanned by the attack may be initialized, e.g., to 1, the number of different destination IP addresses scanned by the attack may be initialized, e.g., to 1, the current rate may be set, e.g., to 0, the start time of the attack may be set to the current time (which may use, e.g., a wall clock, local, or other time standard), a (unique) identifier of the port scanning attack, which may be automatically generated by a local or global service, etc.

[0088] To manage port scan attacks, the set data structure may also be associated with a Boolean function Is-Active-Attack(key), which returns true if the element / attack instance corresponding to the key is currently active and returns false if the element / attack instance corresponding to the key is not currently active (or there is no element / attack instance corresponding to the key in the set). Whether a port scan attack is active may be determined, for example, by comparing the value of the parameter number of different ports scanned to a (preconfigured) threshold, e.g., four different ports scanned, and the value of the parameter current speed to a (preconfigured) threshold, e.g., 10 packets per second. If both parameter values ​​exceed (or meet or exceed) the respective threshold, the attack may be considered active, otherwise the attack may be considered inactive.

[0089] Using efficient implementations of Insert(), Member(), and Is-Active-Attack(), the efficient function Member-Active-Port-Scan-Attack() referenced in the above pseudocode for Compute-Disposition-and-Directives() can be written, for example, as follows (in pseudocode):

[0090] Boolean Member-Active-Port-Scan-Attack(in packet:P0):

[0091] Key key:=Port-Scan-Attack-Key(P0); / *eg,the source IP address of packet P0* /

[0092] IF NOT(Member(key,P0))THEN Insert(key,P0,new element());ENDIF;

[0093] Return Is-Active-Attack(key); / *end of function Member-Active-Port-Scan-Attack* /

[0094] For illustrative purposes, the above functions assume that a set data structure for managing potential port scanning attacks has been initialized and is transparently available, e.g., by the TIG application logic. Also, for different types of attacks other than port scanning attacks, such as port sweep attacks, functions similar to the Member-Active-Port-Scan-Attack() above may be utilized with the appropriate and corresponding subfunctions, data structures, keys, and elements for the attack type.

[0095] Whether an attack is determined to be active (when an in-flight packet is observed) can be important threat context information for determining dispositions and directives for packets that may match packet filtering rules. Figure 3 is a flow chart illustrating an example use of threat context information associated with an active attack, e.g., a port scanning attack (an example of a multi-packet, multi-flow attack), to calculate an example in-flight packet disposition and / or directive. Although the process illustrated by the flow chart is described with reference to a particular device (e.g., TIG 120a), the process may be performed by or in communication with any other device (e.g., any of the other TIGs 120).

[0096] Referring to FIG. 3, in step 3-1, the TIG 120a protecting the network 102 may be configured with logic that may use one or more types of threat context information to compute dispositions and directives for packets in transit to protect the network 102 from threats associated with Internet hosts. Threat context information may include, for example, attack detection logic, such as logic to detect port scan attacks, port sweep attacks, DDoS attacks, reflected spoofing attacks, etc., a current time that may be used to determine when a packet in transit is observed or otherwise received and / or to compare the observation / reception time to a predefined time frame such as normal business hours, context information specific to the owner / operator of the TIG, and / or the like. Other examples of threat context information are described elsewhere in this document. In one particular example, the port scan attack detection logic of the TIG 120a may be configured with thresholds of (for example) 2 for the number of distinct ports indicative of a port scan attack, and 10 packets per second for the attack rate. Other thresholds may be used. Normal business hours of TIG 120a may be configured to be the time interval between (for example) 7:00 AM and 6:00 PM (local time). TIG 120a may also be configured with policies received from SPMS 150, which may include packet filtering rules having match criteria corresponding to threat indicators derived from the CTI. One of these threat indicators may correspond to a particular IP address, e.g., 22.22.22.22, of a host connected to a particular network, such as network 130. The disposition of the rule with match criteria 22.22.22.22 may be “protect,” which signals TIG 120a to apply logic that takes into account threat context information to compute a disposition and / or instructions to apply to packets being forwarded.

[0097] In a first iteration of step 3-2, TIG 120a receives a first inbound packet (referred to herein as "P0") containing a TCP SYN at a particular time, e.g., 3:00 a.m. local time (i.e., 03:00:00:00), on one of its Internet interfaces. This packet P0 may have been sent by a particular host, e.g., host 22.22.22.22, which may be infected with malware for conducting a port scan attack. Thus, the source IP address of P0 is 22.22.22.22, the destination IP address is 11.11.11.11, which may be a public IP address associated with network 102, and the destination port is 22 (a well-known port for the Secure Shell (SSH) service). TIG 120a searches the policies for a rule that matches P0 and finds a rule with matching criteria 22.22.22.22 and a "protect" disposition. The "protect" disposition signals TIG120a to use the threat context information and associated logic to calculate dispositions and / or directives that may be applied to the packet (assuming other matching rules do not override the calculated disposition).

[0098] In the first iteration of step 3-3, the TIG 120a logic may determine that P0 is inbound (with respect to the network 102) and contains the TCP SYN flag, and based on this, may determine that P0 may be a component of a port scan attack. Thus, the logic calls the procedure Compute-Disposition-and-Directives(P0, new string disposition_and_directives). The procedure calls the Boolean function Member-Active-Port-Scan-Attack(P0), which determines, for example, that packet P0 may be the first packet of a new (potential) port scan attack because it is not a member of an existing (potential) port scan attack (e.g., because the subfunction Member(key,P0) returned false). Thus, a new element / port scan attack instance is created from P0 (by calling Insert(key,P0,new element())) and inserted into a local set data structure that manages (potential) port scan attacks. This new element may be populated with initial values ​​that may be derived from the initial packet and the environment, for example, a key may be set to 22.22.22.22 (or 22.22.22.00 / 24) or otherwise generated based on 22.22.22.22 (or 22.22.22.00 / 24), a value for a parameter of the current number of packets constituting the attack may be initialized to 1, a value for a parameter of the number of different ports scanned by the attack may be initialized to 1, a value for a parameter of the number of different destination IP addresses scanned by the attack may be initialized to 1, a value for a parameter of the current rate may be set to 0, a start time of the attack may be set to 3:00 a.m. local time (i.e., 03:00:00:00), etc. It may also be useful to generate a (unique) attack identifier that may be associated with this new element / new (potential) attack instance.

[0099] In the first iteration of step 3-4, during the Member-Active-Port-Scan-Attack() function invocation, the value of the parameter of the number of different ports scanned by the attack and the value of the parameter of the current speed are compared to their associated thresholds. In this example, since neither parameter value exceeds the corresponding threshold, it is determined that the attack is not active, and therefore the Member-Active-Port-Scan-Attack() function returns false. Also in the first iteration of step 3-4, the function Normal-Business-Hours(03:00:00:00) is called, which returns false. Thus, the logic "(Member-Active-Port-Scan-Attack(P0) AND NOT(Normal-Business-Hours(current-time())))" returns no / false, and therefore the Disposition-and-Directives string is set to "Allow, Log, Quick".

[0100] Thus, in step 3-5F, TIG 120a logic permits / forwards packet P0 towards destination 11.11.11.11, logs the packet, and terminates policy lookup (due to the "quick" command). Once processing of packet P0 is complete, logic program control returns to step 3-2 to process the next packet arriving at an interface of TIG 120a.

[0101] In the second iteration of step 3-2, in this example, TIG 120a receives a second inbound packet ("P1") containing a TCP SYN on one of its Internet interfaces 0.01 seconds after P0 arrives at 3:00:00:01 a.m. local time (i.e., P1 arrives at 03:00:00:01). This packet P1 may have been sent by host 22.22.22.22, which may be infected with malware to perform a port scanning attack. Thus, the source IP address of P1 is 22.22.22.22, the destination IP address is 11.11.11.11, which may be a public IP address associated with network 102, and the destination port is 23 (a well-known port for Telnet services). TIG 120a searches the policy for a rule that matches P1 and finds a rule with match criteria 22.22.22.22 and a "protect" disposition (e.g., the same rule that matched packet P0 above in the first iteration of step 3-2). The "protect" disposition signals TIG 120a to use the threat context information and associated logic to calculate dispositions and / or directives that may be applied to the packet.

[0102] In the second iteration of step 3-3, the TIG 120a logic determines that P1 is inbound (with respect to the network 102) and contains the TCP SYN flag, indicating that P1 may be a component of a port scan attack. Thus, the logic calls the procedure Compute-Disposition-and-Directives(P1, new string disposition_and_directives). The procedure calls the Boolean function Member-Active-Port-Scan-Attack(P1), which determines that packet P1 may be a member of an existing (potential) port scan attack (e.g., because the subfunction Member(key, P1) returned true). Thus, a side effect of the Member() function is to insert / integrate packet P1 into the element representing the (potential) port scan attack. For example, the value of the parameter of the current number of packets constituting an attack may be incremented by 1 to 2 (representing a total of two received packets associated with a potential attack), the value of the parameter of the number of different ports scanned by the attack may be incremented by 1 to 2 (representing a total of two different ports associated with a potential attack), the value of the parameter of the number of different destination IP addresses scanned by the attack may remain set to 1 (representing a total of one destination IP address associated with the attack), the value of the parameter of the current rate may be calculated to 100 (in units of packets per second since P1 arrived 0.01 seconds after P0) (representing the packet rate associated with the attack), etc.

[0103] In the second iteration of step 3-4, during the Member-Active-Port-Scan-Attack() function invocation, the value of the parameter number of different ports scanned by the attack and the value of the parameter current speed are compared to their associated thresholds. Since both parameter values ​​match or exceed the corresponding thresholds, the attack is determined to be active, and therefore the Member-Active-Port-Scan-Attack() function returns true. Also in the second iteration of step 3-4, the function Normal-Business-Hours(03:00:00:01) is called, which returns false. Thus, the logic "(Member-Active-Port-Scan-Attack(P1) AND NOT(Normal-Business-Hours(current-time())))" returns yes / true, and therefore the Disposition-and-Directives string is set to "Block, Log, Quick".

[0104] Thus, in step 3-5T, the TIG 120a logic blocks / drops packet P1, thereby protecting network 102 from an (active) port scan attack, logs the packet, and terminates policy lookup (due to the "quick" command). Once processing of packet P1 is complete, logic program control returns to step 3-2 to process the next packet arriving at an interface of TIG 120a. The processing of Figure 3 may be repeated for each subsequent arriving packet.

[0105] It should be noted that the above method for efficiently determining the association of a packet's threat context with a port scan attack may be easily adapted to other types of attacks having other (e.g., similar) characteristics (e.g., multiple different flows), e.g., port sweep attacks, certain DDoS attacks, etc. Also, the method is not limited to packets matching packet filtering rules having matching criteria corresponding to a threat indicator associated with a CTI or corresponding to a particular time frame (e.g., by comparing the packet observation time in transit with a known time frame such as business hours). The matching packet filtering rules may be used on any inbound packet containing a TCP SYN flag, e.g., an unsolicited TCP connection attempt, to detect packets that may be part of a port scan attack. Conversely, such a detector may be adapted to track outbound TCP SYN packets that may originate from, e.g., an internal host that is the source of a port scan attack or similar attack. Such hosts may be compromised by malware for port scan (or similar) attack applications, and thus, once detected, the relevant network administrator may flush and remove the malware from the host.

[0106] There may be multiple different attack detectors operating simultaneously against different types of attacks. Additionally, while in the exemplary illustration above, the CTI-derived packet filtering rules may be used to identify packets that may be associated with threats and may be components of these attacks, these attack detectors may also be used to detect potential and / or active attacks being carried out by packets that may not have been identified by the CTI-derived packet filtering rules.

[0107] The threat context aware packet filtering of the present disclosure may also be used to mitigate some important operational problems that often arise in practice. As mentioned above, it may be desirable to be able to perform CTI noise rejection at in-transit packet observation time. Examples of CTI noise include:

[0108] A CTI listing an IP address that hosts multiple domains, where one or more of the domains may be desirable (e.g., deemed not a threat) and one or more others of the domains may be undesirable (e.g., associated with a potential or actual threat). For example, a content delivery network (CDN) provider often hosts many domains (e.g., hundreds or thousands) on a single IP address X. If any (small) portion of the domains are determined to be a threat by any CTI provider, the CTI provider may include not only the domains in its published CTI, but also the single IP address X that hosts the threat domains (as well as legitimate domains). Thus, each domain hosted on IP address X, including non-threatening / legitimate domains, is associated with a threat. SPMS 150 may include IP address X as a match criterion in a packet filtering rule, and then include the rule in a policy that is distributed to TIGs 120, such as TIG 120a, that protect network 102. If the disposition of the rule is "block," many legitimate business communications between network 102 and the CDN may be blocked, i.e., there may be many false positives. If the rule's disposition is "Allow" and one of the directives is "Log" (as a warning of a possible threat), many false alerts may be generated. In either case, IP address X may be considered "CTI noise".

[0109] CTIs that list popular domains, such as social media platforms. As another example, often the most popular domains on the Internet (e.g., as measured by the speed of DNS resolution requests) are listed in the CTIP's CTI feed. This can occur because, for example, popular social media platforms are often used as attack vectors by malicious actors to result in URI threat indicators. Similar to the CDN example above, CTIPs may publish not only URIs but also associated domain names as CTI, which results in many false positives and false alarms. Such domain names may be considered "CTI noise." A domain may be considered to be sufficiently popular to constitute CTI noise, for example, if the domain is included in a list of popular domains and / or if the domain is determined to have a popularity score above a threshold popularity score.

[0110] CTI associated with a particular location, a particular network, and / or a particular administrative context. As yet another example of CTI noise that may be localized or otherwise specific to a particular network, i.e., location and / or administrative context may be factored into network protection decisions. An enterprise may begin protecting its network with one or more TIGs 120 and associated CTI-based policies, only to quickly discover that the CTI and associated packet filtering rules contain threat indicators that identify some or sometimes all of the enterprise's hosts as threats. If the disposition of the packet filtering rules is "block," the affected enterprise hosts may not be able to conduct legitimate communications with Internet hosts. This particular CTI noise problem is referred to herein as an "autoimmune problem," analogous to biological autoimmune problems, such as when a biological organism's immune system does not recognize itself and may therefore attack itself, and may occur in practice according to the following exemplary scenario: Often, public / WAN IP addresses of the enterprise network 102 may be assigned to the network perimeter interfacing the enterprise network with the Internet 110 and may be identified by the CTI provider as threat indicators, for example, because internal enterprise hosts may be infected with malware or otherwise controlled / compromised by malicious actors and may be engaging in malicious activity on the Internet. Such activity may be detected by the CTIP 140, which may identify the IP addresses and / or associated subnet address ranges of the compromised hosts as threat indicators, which may be published to a CTI feed, which may be ingested by the SPMS 150, which may convert the indicators into packet filtering rules with a “block” disposition, which may be included in policies distributed to the TIGs 120, including TIG 120a, that protect the network 102.Legitimate communications between internal corporate hosts connected to network 102 and Internet hosts may then be blocked, which an enterprise may consider to be highly undesirable behavior.

[0111] In the above examples of CTI noise and autoimmunity, it is often difficult, impractical, or impossible to filter out / filter the CTI noise in the CTIP 140 and / or SPMS 150 due to, for example, the volume, dynamics, lack of automation, visibility and / or access associated with the generation / creation of CTI and CTI noise, contextual differences unique to each subscriber, etc. Additionally, CTI noise may be contextual; for example, one subscriber (e.g., an enterprise associated with a particular TIG 120 and / or a particular network, such as TIG 120a and / or network 102) may view a set of indicators as noise (and not necessarily a threat), while another subscriber (e.g., another enterprise associated with a different particular TIG 120 and / or a particular network, such as TIG 120b and / or network 104) may view the same set of indicators as a threat, or, for example, an enterprise may view a set of indicators as a threat at one time, but later (e.g., when TIG 120 observes a packet in transit associated with that set of indicators) view the same set of indicators as noise (and not necessarily a threat).

[0112] The techniques and methods of the present disclosure may be used to solve the CTI noise problem and the autoimmune sub-problem. A general approach may identify CTI noise as threat context information determined in TIG 120 in response to a packet in transit and include the threat context information in subsequent calculations of TIG 120 dispositions and directives. Each TIG 120 may be configured with local public / WAN IP addresses of associated network boundaries as threat context information, and other indicators that are considered CTI noise by administrators of each TIG 120. For example, TIG 120a may be configured with one or more local public / WAN IP addresses of network boundaries of network 102, and TIG 120b may be configured with one or more local public / WAN IP addresses of network boundaries of network 104. For example, local public / WAN IP addresses and / or other indicators identified as CTI noise (e.g., IP addresses, domain names, URIs, and / or others) may be inserted into an efficient set data structure, such as a Bloom filter, which may be efficiently tested for element membership by the TIG 120 logic (in response to observing an in-flight packet that satisfies the rule) as the logic processes the rules and matching packets. The associated policy threat indicator rule for any indicator identified as CTI noise may be configured with a "protect" disposition instead of a "block" disposition (or with some other disposition other than "allow" and "block", or with no disposition at all, such as a null disposition, or with or without a disposition, and may be associated with information, such as a flag, that is interpreted as an instruction for TIG 120 to use threat context information at in-flight packet observation time to determine a disposition). Then, for example, when such a rule (e.g., a "protect" rule) matches a packet, the matching threat indicators may be tested for membership in the efficient set data structure, and the membership test results (i.e., true / member or false / not member) may be used as input to the computation of dispositions and directives.For example, a true result may cause the TIG 120 logic to compute a disposition of "allow" and "log" and "continue" commands, while a false result may cause the TIG 120 logic to compute a disposition of "block" and "log" and "quick" commands.

[0113] In a further example, the TIG120 logic may be configured to take CTI noise and / or autoimmunity into account (e.g., always), regardless of the disposition of the applicable rule. For example, the TIG120 logic may be configured to always take CTI noise and / or autoimmunity into account when a "block" rule matches a packet, and may calculate and apply dispositions and directives accordingly. In practice, the semantics of the "block" disposition of a rule may be modified to, for example, "block this packet unless the matching indicator is within a set of indicators associated with CTI noise and / or autoimmunity." Such a TIG120 logic configuration may be useful, for example, when the associated SPMS does not provide for the construction of rules with a "protect" disposition, but protection from CTI noise and / or autoimmunity is required. More generally, in some scenarios, it may be useful to have TIG120 logic that takes into account any threat context information, not just CTI noise, when applying any rule, such as rules with a "block" and / or "allow" disposition. For example, the TIG 120 logic may use threat context information to calculate dispositions and directives without relying on the "protect" disposition of the rule. While various examples described herein use the "protect" disposition as a precursor to using threat context information and associated logic, the TIG 120 may rely on any threat context information (including, but not limited to, CTI noise indicators) when applying any rule having any disposition, such as, but not limited to, a "block" or "allow" disposition. Thus, any examples described herein that use threat context information when applying a "protect" disposition rule may be similarly implemented when applying other dispositions of the rule, which may be substituted in these examples to recalculate or otherwise determine the final disposition and / or directive of the rule.

[0114] 1, when a packet originating from a host connected to network 102 matches a "protect" rule (or any other rule as desired, as mentioned immediately above) applied by TIG 120a protecting network 102, the associated TIG 120a logic may test whether the matching indicator corresponds to (local) CTI noise and / or autoimmune threat context information (consisting of local public / WAN IP addresses and possibly other CTI noise indicators) included in the set data structure filter and determine that the calculated disposition may be "allow" and the policy processing instruction may be "continue". On the other hand, for a packet including communication with a host connected to a different network 104 having a local public / WAN IP address not within the CTI and filtered by TIG 120b associated with network 104, the associated TIG 120b logic may determine that the calculated disposition is "block" and the policy processing instruction is "quick". Thus, network 102 and TIG 120a may allow legitimate communications between internal hosts and Internet hosts, while network 104 and TIG 120b may block malicious communications with threat hosts connected to network 102.

[0115] FIG. 4 illustrates a flow chart of an example process for solving the autoimmune problem and the more general CTI noise problem, i.e., performing CTI noise filtering at in-transit packet observation / filtering time using threat context information and related methods of the present disclosure. FIG. 4 tracks the processing of a TIG 120, such as TIG 120a protecting a (private) enterprise network 102. FIG. 4 also illustrates an example process for an SPMS 150 that may provide a set of policies and / or packet filtering rules to TIG 120a. Note that FIG. 4 may similarly be applied to TIG 120b protecting network 104, and TIG 120c protecting network 108.

[0116] In step 4-1, public / WAN IP addresses of the (private) enterprise network 102 assigned to the network boundary with the Internet 110 (or other public network) are identified and configured as threat context information to the local TIG 120a protecting the network 102 at the boundary. For example, the public / WAN IP addresses, as well as other CTI noise indicators (which may be, for example, IP addresses, domain names, URIs, and / or others), may be inserted into a dataset named CTI-NOISE, which may be an efficient set data structure such as a Bloom filter managed by the TIG 120a. Public / WAN IP addresses are typically assigned to enterprises by their Internet Service Providers (ISPs). Often, ISPs assign IP addresses to subscribers as subnet addresses. A subnet address is a block of contiguous IP addresses. For example, the (IPv4) subnet address 11.11.11.00 / 24 ​​may be assigned at the boundary of the network 102, representing a block of 256 contiguous IP addresses ranging from 11.11.11.00 to 11.11.11.255. Often, corporate networks are protected by a network firewall (not shown) located at the boundary that provides multiple functions, including network address translation (NAT). The NAT function may be embodied in a NAT gateway device, or a NAT interface device, that translates between the private IP address space of the private network 102 and the public IP addresses assigned at the network boundary. In this way, an internal host may be assigned a private IP address but communicate with an Internet host because the NAT (temporarily) assigns the internal host in Internet communication a public IP address that may be necessary for packets of the communication to be routed through the Internet.However, from the perspective of CTI and related network protection systems, the NAT function may exacerbate the autoimmune problem because, for example, a single compromised internal host may perform multiple (threat) communications that may be translated (by NAT) to multiple different public IP addresses. CTIP 140 may detect these multiple communications from multiple different IP addresses, determine that the associated subnet address prefix, e.g., 11.11.11.00 / 24, is a threat indicator, and may then publish the indicator / subnet address prefix to a CTI feed. In practice, this means that any Internet communication by any internal host may be recognized as a threat by subscribers of policies that include rules generated from the CTI feed, including TIG 120a protecting network 102.

[0117] In step 4-2, SPMS 150 may receive a CTI feed from CTIP 140 that includes public / WAN IP addresses of network 102 and identify them as threat indicators. The CTI also identifies IP address 66.66.66.66 (for example) associated with a host connected to network 132.

[0118] In step 4-3, the rule and policy generation component of the SPMS 150 creates one or more packet filtering rules with a disposition of “protect” and with matching criteria corresponding to the public / WAN IP addresses of the network 102 (i.e., 11.11.11.00-11.11.11.255), which are threat indicators. For example, rule R0 “protect 11.11.11.00 / 24” may be created. This rule R0 and other rules may be included in a policy distributed to a subscriber, including TIG 120a, that protects the network 102. Alternatively, if TIG 120 includes logic to automatically check a local set data structure for a CTI noise indicator when a “block” rule is matched, the SPMS may create a packet filtering rule with a disposition of “block” and with matching criteria corresponding to the public / WAN IP addresses of the network 102. Also, rule R1 "Block Log Quick 66.66.66.66" is created which follows rule R0 in the policy, i.e., a search of the policy may encounter rule R0 before encountering rule R1.

[0119] In step 4-4, TIG 120a (protecting network 102) receives the policy distributed in step 4-3 by SPMS 150. TIG 120a enforces the policy on packets in transit traversing the boundary between network 102 and the Internet 110.

[0120] In step 4-5, TIG 120a receives packet P1 originated by (internal) host A connected to network 102 with public IP address 11.11.11.01 (e.g., assigned by a NAT located at the border of network 102 with Internet 110), which is destined for host B connected to network 104 with public IP address, e.g., 44.44.44.21. Thus, the IP header of packet P1 contains source IP address value 11.11.11.01 and destination IP address value 44.44.44.21. The IP address 44.44.44.21 of (destination) host B is not within the CTI, and thus host B is not considered a threat and does not correspond to the match criteria of any packet filtering rule in the policy being enforced by TIG 120a.

[0121] In steps 4-6, TIG 120a applies the current policy to packet P1 to determine whether the rule matches packet P1. In this example, TIG 120a applies the current policy to packet P1, which matches rule R0 ("Protect 11.11.11.00 / 24") because P1's source IP address value is 11.11.11.01. The "Protect" disposition signals TIG 120a to calculate a disposition and / or directive for P1 using at least the threat context information, including the CTI noise indicator contained in a CTI-NOISE indicator set data structure (e.g., a Bloom filter). Thus, TIG 120a can determine (e.g., calculate) a disposition and / or one or more directives for P1 using the threat context information based on its determination that rule R0 is assigned the "Protect" disposition. TIG 120a tests the set data structure for membership of element 11.11.11.01, which is true (because 11.11.11.01 was inserted into the CTI-NOISE set data structure in step 4-1). A true result may cause TIG 120a logic to determine that (a) the disposition is "Allow" for packet P1, and (b) the instructions are "Log" and "Continue", so that, for example, an administrator may be alerted to the autoimmune instance and packet P1 may continue to be filtered by the rules in the policy. The "Continue" instruction causes TIG 120a to continue filtering P1 through the remainder of the policy. No other matching rules are found (possibly because host B's IP address 44.44.44.21 is not within the CTI), therefore the disposition of P1 is still "allow" and therefore TIG120a applies the calculated disposition and / or directive and thus in this example, promotes / allows P1 to forward towards its destination (i.e., IP address 44.44.21).If rule R0 was instead "allow 11.11.11.00 / 24" or "block 11.11.11.00 / 24," then the predetermined indicated "allow" or "block" disposition (respectively) may instead be applied to packet P1 without having to determine threat context information. Thus, TIG 120a may determine whether to determine threat context information for an observed in-flight packet based on the disposition indicated by the rule associated with (e.g., matching) that packet.

[0122] In step 4-7, TIG 120a receives packet P2 originated by host A with public IP address 11.11.11.01 connected to network 102, which is destined for host D with public IP address, for example, 66.66.66.66, connected to network 132. Thus, the IP header of packet P2 contains source IP address value 11.11.11.01 and destination IP address value 66.66.66.66. As described in step 4-1, the IP address 66.66.66.66 of (destination) host D is in the CTI. Thus, host D is considered to be a threat and corresponds to the match criteria of packet filtering rule R1 in the policy enforced by TIG 120a.

[0123] In step 4-8, TIG 120a applies the current policy to packet P2, which matches rule R0 ("Protect 11.11.11.00 / 24") because P2's source IP address value is 11.11.11.01. The "Protect" disposition signals TIG 120a to compute a disposition and directive for P2 from the threat context information, including the CTI noise indicator contained in the CTI-NOISE indicator set data structure. TIG 120a tests the set data structure for membership of element 11.11.11.01, which is true (because 11.11.11.01 was inserted into the set data structure CTI-NOISE in step 4-1). A true result may cause the TIG 120a logic to determine that (a) the disposition is "allow" for packet P2, and (b) the commands are "log" and "continue," so that, for example, an administrator may be alerted to the autoimmune instance and packet P2 may continue to be filtered by the rules in the policy. The "continue" command causes the TIG 120a to continue filtering P2 through the remainder of the policy. P2 matches rule R1, which is "Block 66.66.66.66 Log Quick." Thus, the policy lookup stops because of the "quick" command, packet P2 is logged because of the "log" command, and packet P2 is dropped because of the "block" disposition. Thus, the network 102 may be protected from certain external threats.

[0124] With respect to the threat context information used to determine packet dispositions and directives, the methods described above in connection with Figures 3 and 4 may be characterized as "local" because the threat context information in these examples is derived from both the current in-flight packet being processed by the TIG 120 and local environmental information determined (e.g., available and / or calculated) by the TIG 120 after the in-flight packet is received by the TIG 120. However, network protection may be further improved if global threat situation and awareness information regarding threats / attacks that may be actively occurring or have recently occurred on other networks other than the network protected by the (local) TIG 120 may be used by the local TIG 120 to determine packet dispositions and directives. This global threat context information is particularly useful for attacks in which a malicious actor may be attacking multiple different networks as part of a single activity. Such global threat context information may be collected and disseminated by what is referred to herein as a global threat context system / service, comprised of, for example, one or more Global Threat Context Servers (GTCS) 170 (see Figure 1). TIG120 can not only provide / push its own global threat context information to GTCS170, but can also receive / pull other TIG's global threat context information from GTCS170.

[0125] For example, with reference to Figure 5 and step 5-1, consider a TIG 120, such as TIG 120a in step 3-4 of Figure 3, that may determine that a potential attack, e.g., a potential port scanning attack, has transitioned to an active attack. For example, a host having IP address 22.22.22.22 and associated with network 130 may be port scanning network 102 protected by TIG 120a. At some point during the attack, TIG 120a may determine that an active port scanning attack is occurring against network 102, e.g., as in step 3-5T of Figure 3 above.

[0126] In step 5-2, TIG 120, e.g., TIG 120a, may provide characteristic information about the attack to GTCS 170, which may include, for example, the key (e.g., the IP address 22.22.22.22 of the host from which the attack originates or the subnet address range 22.22.22.00 / 24), the type of attack (e.g., a port scan attack), the start time of the attack (e.g., 3:00 a.m.), the network being attacked (e.g., identified by the subnet address prefix 11.11.11.00 / 24 ​​of network 102), dispositions and / or directives applied to the packets, matching rules, and associated metadata, etc. TIG 120a may provide the information to GTCS 170, for example, periodically and / or in response to the occurrence of an event, such as a particular number of packets being observed matching a particular rule or a particular key, or in response to the detection of an attack.

[0127] In step 5-3, the GTCS 170 may receive information of active attacks from TIG 120a, store the attacks (e.g., in a database and associated services for managing information of active attacks), associate a (globally unique) identifier / ID with the attack, and publish the characteristic information and identifier to GTCS subscribers, such as TIG 120a, TIG 120b, other TIGs 120 in other networks, and / or other devices and systems (further examples of these subscribers are discussed below).

[0128] In step 5-4, a TIG 120 that joins the GTCS 170 and protects a different network than the TIG described in step 5-2, e.g., TIG 120b protecting network 104 with subnet address range 44.44.44.00 / 24, may receive (global) threat context information from the GTCS 170 about active port scanning attacks detected and provided (in step 5-2) by TIG 120a. TIG 120b may create a new active attack element from the information and insert the element into a (locally efficient) set data structure for managing / tracking potential port scanning attacks. This new element may be configured such that an Is-Active-Attack(key) function that may be associated with the set data structure returns true when the key parameter value corresponds to the key of the new element.

[0129] In step 5-5, host 22.22.22.22 begins to attack network 104 protected by TIG 120b by sending a first TCP SYN packet to the network's 104 public IP address and port.

[0130] In steps 5-6, TIG 120b, protecting network 104, receives a (e.g., first) TCP SYN packet sent by host 22.22.22.22. TIG 120b may apply a policy to the packet and may match the packet against a "protect" rule that may have as a match criterion a CTI-derived threat indicator corresponding to IP address 22.22.22.22 and / or may have a match criterion corresponding to inbound TCP SYN packets. In either case, in response to determining that the matching rule has a disposition of "protect," TIG 120b logic may invoke a procedure Compute-Disposition-and-Directives(key), where the "key" shown for this procedure is host address 22.22.22.22, or an associated subnet address prefix, e.g., 22.22.22.00 / 24. Due to the configuration performed in step 5-4 resulting from the global threat context information for IP address 22.22.22.22, the calculated disposition of this initial TCP SYN packet may be "block." Note that in the absence of the global threat context information and associated actions, the calculated disposition of this initial TCP SYN packet may have been "allow."

[0131] In step 5-7, similar to step 5-2, except another TIG protecting a different network, another TIG, such as TIG 120b, may provide (additional) characteristic information about the attack to GTCS 170, including, for example, a key (e.g., IP address 22.22.22.22 of the host from which the attack originates or subnet address prefix 22.22.22.00 / 24), a type of attack (e.g., a port scan attack), a start time of the attack (e.g., just after 3:00 a.m.), a network being attacked (e.g., identified by subnet address prefix 44.44.44.00 / 24 ​​of network 104), an attack identifier (which may have been previously provided by GTCS 170), etc. TIG 120b may provide information to GTCS 170, for example, periodically and / or in response to the occurrence of an event, such as a particular number of packets matching a particular rule or a particular key being observed, or in response to the detection of an attack.

[0132] In step 5-8, the GTCS 170 may receive information of active attacks from the TIG 120b, store and / or update the attacks in the above-mentioned database and related services for managing information of active attacks, and publish the (updated) feature information to the GTCS subscribers. The updated feature information is then available for use by any of the TIGs 120 in subsequent iterations of step 5-4 for subsequent observed packets by any of the TIGs 120.

[0133] In addition to the TIG 120 protecting the network, the GTCS 170 subscribers may include, for example, (global) network threat situational awareness systems, CTI providers, Internet Service Providers (ISPs), Managed Security Service Providers (MSSPs), Network Security as a Service (NSaaS) providers, etc. Additionally, the information collected and disseminated by the GTCS 170 may be associated with many different types of attacks (not necessarily limited to just the port scanning attack example above). Thus, the TIG logic (or other GTCS subscribers) may use the global threat context information associated with many different attacks to protect the network from these many different attacks.

[0134] As described above or elsewhere in this document, there are many potential sources and types of threat context information, and many possible combinations thereof, that may be used to determine dispositions and instructions for packets in transit. Furthermore, the characteristics of cyber threats and their corresponding threat contexts change / evolve over time. As the number of sources and combinations of threat context information increases and the characteristics of the threats and associated threat contexts change / evolve, it may be impractical for a human to design and manually program the associated TIG120 logic to efficiently use the threat context information. In such scenarios, machine learning may be used as an alternative and / or augmentation approach to create and evolve at least a portion of the TIG120 logic. For example, a machine learning artificial neural network (ANN) may be created having input nodes corresponding to multiple different sources / types of threat context information associated with packets in transit being filtered by TIG120, and having output nodes corresponding to dispositions and instructions that may be applied to the packets. The ANN may be constructed in such a manner that the resulting logic for computing dispositions and instructions may be highly efficient (e.g., may have a constant time computational complexity), e.g., as a bounded depth classifier. Moreover, the ANN may be easily adapted to new and / or different threat context information, as well as new and / or different dispositions and instructions, for example, by adding and / or modifying input and output nodes.

[0135] FIG. 6 illustrates an example ANN 600 that may embody the machine learning logic of the TIG 120 for computing dispositions and instructions for observed in-flight packets based at least in part on threat context information associated with the in-flight packets. As illustrated, the ANN 500 may have multiple layers of nodes. For example, the ANN 500 may have four layers of nodes, including a first layer of N input nodes 610-1 through 610-N that may correspond to multiple (N) threat context information (TCI) sources and / or types, and a fourth layer of M output nodes 640-1 through 640-M that may correspond to multiple dispositions and / or instructions. M may be greater than, equal to, or less than N. The second layer (620-1 through 620-X) and third layer (630-1 through 630-Y) of nodes correspond to embedded or "hidden" layers. Although two hidden layers are shown in this example, the ANN 500 may have any number of hidden layers. Those skilled in the art will understand the general principles of artificial neural networks. As shown in FIG. 6, the output of each node in a layer of the ANN 600 may be connected to the inputs of two or more nodes (e.g., each node) in a subsequent layer, i.e., each layer of nodes may be connected (e.g., fully connected) to a subsequent layer of nodes. Each connection between nodes may be weighted and directed such that the input to a node in a given layer (except possibly the first layer) may be a weighted sum of the outputs of two or more nodes (or all nodes) in the previous layer. Except possibly for nodes in the first layer, the output of each node may be the result of applying an activation function (which may be a nonlinear function), such as a sigmoid function or a rectified linear unit (ReLU) function, to the input of the node. The output of the node may be constrained to a value within a particular range of values, such as between 0 and 1 (inclusive). In the context of the present disclosure, input values ​​of first tier nodes may be efficiently computed and / or accessed, and computations performed by other tier nodes may be efficiently computed such that dispositions and / or directives for currently in-flight packets may be efficiently determined by ANN 600 and efficiently applied to the in-flight packets by TIG 120 logic.

[0136] In FIG. 6, each (input) node of the first tier corresponds to a source and / or type of threat context information (TCI) that may be associated with a packet being filtered / observed by the TIG 120. For example, TCI-1 may include a value generated by a function associated with the observation time of the packet being forwarded (e.g., generating a value based on the observation time of the packet being forwarded), TCI-2 may include a value generated by a function indicating whether the packet being forwarded is a component of an active attack, and TC-3 may include a value generated by a function associated with threat context information associated with a packet filtering rule that matches the packet being forwarded (e.g., generating a value based on the threat context information). The inputs to the nodes of the first tier, i.e., values ​​of TCI-1 through TCI-N, may be constrained to be binary values, e.g., either 1 (true) or 0 (false), or may be constrained in another manner to be within the range [0,1]. For example, with reference to the examples of threat context information and additional information above, some example threat context information and / or additional information that may be input to the first tier of nodes or may be used as a basis for generating input to the first tier of nodes may include any one or more of the following:

[0137] Normal-Business-Hours(current-time()): One or more inputs (e.g., one or more binary valued inputs) that may be the output of a Boolean function that may efficiently determine whether the current time / observation time of a packet in transit occurs during a certain predefined time frame (e.g., normal business hours of the organization that manages the associated TIG 120 and protected network). Note that a similar exemplary function is described above in connection with FIG. 3.

[0138] Is-Active-Attack(): one or more inputs (e.g., one or more binary-valued inputs) that may be the output of a Boolean function that may efficiently determine whether a packet being forwarded is part of an active attack on the network. A similar exemplary function is described above in connection with FIG. 3. There may be other similar binary-valued input functions and input nodes associated with attack types, such as a node for a port scan attack, a node for a port sweep attack, a node for a leak attack, one or more nodes for one or more DDoS attack types, etc.

[0139] Threat-Indicator-Type-X: One or more inputs (e.g., one or more binary valued inputs) that indicate the type of threat indicator to be used as a matching criterion for a packet filtering rule that matches the packet being forwarded. For example, threat indicator types may include IP address, subnet address range, 5-tuple, domain name, URI, etc., and the associated input nodes may be labeled Threat-Indicator-Type-IP, Threat-Indicator-Type-Subnet-Address-Range, Threat-Indicator-Type-5-tuple, Threat-Indicator-Type-Domain-Name, Threat-Indicator-Type-URI, etc.

[0140] Domain Popularity: One or more inputs (e.g., one or more [0,1] valued inputs) indicating the popularity of the domain name (if any) contained in the packet being forwarded. Domain popularity data can be highly dynamic, readily available from multiple services / providers, and can be stored locally in efficient data structures for fast access. In general, domains with low popularity values ​​are often associated with more threat risk than domains with high popularity values.

[0141] Domain Name and URI Features: One or more inputs, e.g., (one or more binary-valued and / or [0,1]-valued inputs), associated with features, e.g., syntactic or lexical features, of the domain name and / or URI (if present) contained in the packet in transit that may be associated with a threat risk. Syntactic or lexical features may include, e.g., string length, number of labels, number of numeric characters, number of URL-encoded characters, correlation with words from human-spoken languages, information entropy measures, top-level domains (TLDs), etc. Direction (inbound or outbound): One or more inputs (which may be binary values) indicating whether the packet being forwarded and / or the associated flow of packets being forwarded may be inbound or outbound, Connection State: One or more inputs (e.g., one or more binary value inputs) that indicate whether a TCP connection is established for the flow associated with the packet being forwarded; Secondary Analysis Alert: Any result / output of a threat / attack analysis of a flow associated with a packet, e.g., one or more inputs (e.g., one or more binary value inputs) indicating whether the associated flow matched one or more signatures applied by an IDS / IPS and / or NBA system, etc. The analysis results may also include associated threat / attack information, such as attack type, which may also correspond to an input node. Global Threat Context Information: One or more inputs (e.g., one or more binary valued inputs) indicating an association of the packet being forwarded with global threat context information, such as active attacks on networks other than the local protected network, the type of attack, etc. Exemplary global threat context information is described above in connection with FIG. 5, and / or the like.

[0142] There are additional inputs to ANN 600 that may not be considered threat context information, but are included as inputs to improve the performance of ANN 600, for example, by reducing false positives and / or false negatives and / or uncertain outputs / results. Such information may include, for example, CTI-Provider-X: one or more inputs (e.g., one or more binary valued inputs) indicating whether a particular CTI provider has provided a CTI threat indicator that may be used as a matching criterion for a packet filtering rule that matches the packet being forwarded. The CTI provider information may be included in the metadata of the matching rule and may be efficiently extracted / accessed when the rule matches the packet being forwarded. For each CTI provider, there may be one such input node to ANN 600, and each such input node may be appropriately labeled, for example, as CTI-Provider-1, CTI-Provider-2, ...CTI-Provider-J. Similarly, there may be input nodes for the name / identifier of the CTI feed, the attack type associated with the CTI feed, the recommended disposition associated with the feed (e.g., as provided by the CTI provider), the trustworthiness associated with the feed, etc., and any of such information may be included in the metadata of the match rule.

[0143] The output nodes 640-1 through 640-M may correspond to computed dispositions and / or directives that may be applied to packets in transit. For example, one output node may correspond to a "block" disposition, another may correspond to a "allow" disposition, another may correspond to a "log" directive, another may correspond to a "capture" directive, another may correspond to a "spoof-tcp-rst" directive, another may correspond to a "quick" directive for policy processing, etc. The output values ​​may be constrained and / or trained to be within a particular range, such as the range [0,1]. Thus, the output value of a given output node 640 may be viewed as a probability or likelihood that the corresponding disposition or directive is a good choice for protecting the associated network.

[0144] An ANN such as the ANN 600 of FIG. 6 may use supervised learning methods and backpropagation for training / learning algorithms. The training data set of the ANN 600 may consist of many examples of {{input value}, {output value}} pairs, where the set {input value} may correspond to threat context information input values ​​associated with packets of the training examples (e.g., first layer node input values ​​TCI-1 through TCI-N) and the {output value} may correspond to desired dispositions and instructions for the packets of the training examples. During training, the {input value} may be input to the first layer of the ANN 600, and the actual output values ​​(e.g., outputs 640-1 through 640-M) may be compared to the desired {output value}. The difference between the desired {output value} and the actual output value may be used by the backpropagation algorithm to modify the weights of the connections between the nodes so that the difference may be smaller the next time the {input value} is input. Training and associated backpropagation may continue until convergence is achieved, i.e., until the cumulative differences between the desired {output values} and the actual output values ​​over the training data set are acceptably small, and therefore changes to the weights during backpropagation may be acceptably small.

[0145] Upon completion of training, backpropagation / learning may be turned off and the resulting ANN 600 may be referred to as a classifier or model. The ANN 600 classifier logic may be integrated with other portions of the logic of the TIG 120, such as application logic, to compute dispositions and / or directives for packets in transit that match packet filtering rules (derived from the CTI). For example, the ANN 600 classifier logic may be integrated with steps 3-3 and 3-4 of FIG. 3 and steps 2-7 of FIG. 2. In practice, the ANN 600 classifier logic may be combined with other logic, such as human-designed, manually programmed logic, for example, to implement default behaviors and / or to handle false positives and false negatives generated by the ANN 600 classifier, and / or to handle uncertain results generated by the ANN 600 classifier, and / or to handle deficiencies in the training data and / or associated models, etc. For example, it may be desirable for the value of the "block" disposition output node to be 1 or close to 1, and vice versa, when the value of the "allow" disposition output node is 0 or close to 0. However, there may be cases where both values ​​are close to 1, or close to 0, or otherwise the values ​​are indeterminate with respect to the choice of "Block" or "Allow" disposition, in which case manually programmed logic may be used to determine the disposition. In another example, the "Allow" disposition output node may be 1 or close to 1 (and the "Block" node's value close to 0), but the "spoof-tcp-rst" command output node's value may also be close to 1, in which case manually programmed logic may intervene to allow / promote the associated packet, but not send a TCP RST packet to the source of the packet. Otherwise, sending a TCP RST packet to the source may tear down the associated TCP connection, which may be a highly undesirable action.

[0146] It should be noted that in addition to artificial neural networks (ANNs), other machine learning algorithms and methodologies may be used to design the decision logic, e.g., evolutionary algorithms, genetic algorithms, genetic programming, etc. The choice of ANNs in FIG. 6 is exemplary and is not meant to exclude other machine learning approaches.

[0147] Any of the elements, such as any of the TIG 120 and any of the elements 130, 132, 138, 140, 150, 160, and 170, described herein or shown in any of the drawings, may be implemented partially or fully using one or more computing devices, such as the computing device 700 shown in FIG. 7. The computing device 700 may be, for example, a general-purpose computing device having general-purpose hardware configured to perform one or more specific functions using specific software and / or firmware, or may be, for example, a special-purpose computing device having special-purpose hardware (and / or special-purpose software and / or special-purpose firmware) customized for a specific function. Any of the hardware elements of the computing device 700, and / or the computing device 700 itself, may be emulated with a virtual version of the computing device 700. The computing device 700 may include one or more processors 701 that may execute computer-readable instructions of a computer program to perform any of the functions described herein or other operations. The instructions, along with other data, may be stored in storage 702, which may include memory, such as, for example, read-only memory (ROM) and / or random access memory (RAM), a hard drive, a magnetic or optical disk, a universal serial bus (USB) drive, and / or any other type of computer-readable medium. Data may be organized in any desired manner, such as, for example, stored as instructions in storage 702 and accessible via database software executed by the one or more processors 701. Computing device 700 may also include a user interface 704 for interfacing with one or more input devices 705, such as a keyboard, mouse, voice input, and one or more output devices 706, such as a display, speakers, printer, etc.The computing device 700 may also include a network interface 703 for interfacing with one or more external devices that may be part of a network external to the computing device 700. While FIG. 7 shows an exemplary hardware configuration, one or more of the elements of the computing device 700 may be implemented as software or as a combination of hardware and software. Modifications may be made to add, remove, combine, divide, etc., components of the computing device 700. In addition, the elements shown in FIG. 7 may be implemented using basic computing devices and components configured to perform operations as described herein. The processor 701 and / or storage 702 may also or alternatively be implemented via one or more integrated circuits (ICs). An IC may be, for example, a microprocessor that accesses programming instructions or other data stored in a ROM and / or hardwired within the IC. For example, an IC may include an application specific integrated circuit (ASIC) having gates and / or other logic dedicated to the calculations and other operations described herein. An IC may perform some operations based on execution of programming instructions read from a ROM or RAM, along with other operations hardwired into the gates or other logic.

[0148] As described above, the computing device 700 may be embodied as a packet filtering appliance, such as, for example, a TIG (e.g., any of the TIGs 120). FIG. 8 illustrates an example block diagram of such a packet filtering appliance 800, which may be located at a boundary 802 of a network, such as the network 102 (or, for example, the networks 104 or 108). Thus, the packet filtering appliance 800 is an example implementation of the computing device 700. The packet filtering appliance 800 may include one or more processors 804 (which may be the same as the processor 701), memory 806 (which may be the same as the storage 702), network interfaces 808 and / or 810 (which may be the same as the network interface 703), a packet filter 812 (which may be performed by the processor 701), and a management interface 814 (which may be performed by the processor 701, the user interface 704, the input device 705, and / or the output device 706). The processor 804, memory 806, network interfaces 808 and / or 810, packet filter 812, and / or management interface 814 may be interconnected via a data bus 816 (which may be an arrow interconnecting any of the various elements of FIGS. 7 and 8). The network interface 810 may connect the packet filtering appliance 800 to a first network, such as network 102 (or 104 or 108). Similarly, the network interface 808 may connect the packet filtering appliance 800 to a second network, such as network 110. The memory 806 may include one or more program modules that, when executed by the processor 804, may configure the packet filtering appliance 800 to perform one or more of the various functions described herein.The memory 806 may also be used to store rules, databases, logs, and / or any other information used and generated by the packet filtering appliance 800.

[0149] The packet filtering appliance 800 may be configured to receive policies (such as filtering rules described herein) from one or more security policy management servers (e.g., SPMS 150 shown in FIG. 1). For example, the packet filtering appliance 800 may receive policies 818 from the security policy management servers via management interface 814 (e.g., via out-of-band signaling) or via network interface 808 (e.g., via in-band signaling). The packet filtering appliance 800 may include one or more packet filters or packet discriminators and / or logic for implementing the one or more packet filters or packet discriminators. For example, the packet filtering appliance 800 may include a packet filter 812 that may be configured to inspect information associated with packets received by the packet filtering appliance 800 (e.g., from network 110) and route such packets to one or more of operators 820, 822, and / or 824 (which may be implemented as hardware and / or software executed by processor 804) based on the inspected information. For example, the packet filter 812 may examine information associated with a packet received by the packet filtering appliance 800 (e.g., a packet received from the network 110 via the network interface 808) and, based on the examined information, route the packet to one or more of the operators 820, 822, or 824. These operators may implement, for example, dispositions and directives associated with a packet filtering rule that matches the packet.

[0150] The policy 818 may include one or more filtering rules, and the configuration of the packet filter 812 may be based on one or more of the rules included in the policy 818. For example, the policy 818 may include one or more rules specifying that packets having specified information should be sent to an operator 820, packets having different specified information should be sent to an operator 822, and all other packets should be sent to an operator 824. The operators 820, 822, and / or 824 may be configured to perform one or more functions on packets received from the packet filter 812. For example, one or more of the operators 820, 822, and / or 824 may be configured to send packets received from the packet filter 812 to the network 102, which may implement a "allow" disposition, may drop packets received from the packet filter 812, or may implement a "block" disposition. One or more of operators 820, 822, and / or 824 may be configured to drop packets by sending them to a local "infinite sink" (e.g., the / dev / null device file on UNIX / LINUX systems). In some embodiments, one or more of operators 820, 822, and / or 824 may be configured to apply directives to the packets, such as "log," "capture," and other directives described herein.

[0151] One or more of operators 820, 822, and / or 824 may also be configured to implement a "protect" disposition as described herein based on one or more filtering rules set forth in policy 818. For example, if one or more packets are determined by operator 822 to match or otherwise satisfy one or more rules that may be configured for a "protect" disposition, operator 822 may send information indicative of this (e.g., indicating which rules have been satisfied and / or indicating logging parameters that should be used) to processor 804. Processor 804 may then begin executing logic to calculate dispositions and directives that may be applied to one or more packets using various threat context information as described herein. Processor 804 may include and / or be supplemented by ANN 600 discussed above with respect to FIG. 6, along with necessary physical and / or logical inputs to the input layer of node 610 and physical and / or logical outputs from the output layer of node 640.

[0152] Packet filtering appliance 800 may obtain threat context information from (or generate threat context information based on information retrieved from) various sources, some of which may be local (sources internal to packet filtering appliance 800) and others of which may be remote (sources external to packet filtering appliance 800). For example, processor 804 may include a clock that may be used to maintain a current time and to determine packet observation times in transit. With reference to other examples of threat context information above, appliance location, appliance ID, administrator and associated security policy preferences, network type, active threat types, analysis results of multi-packet multi-flow threats / attacks, CTI provider and associated information, threat indicator type and fidelity, threat indicator lifespan, flow origination, flow direction, flow state, connection state, global threat context, domain name, URI, URL, domain name popularity, domain name registration status, data transfer protocol method, protocol risk, contextual CTI noise, etc. may be determined by processor 804 based on processing of information stored in memory 806, based on calculations by processor 804, based on information received from network 110 via network interface 808, based on information received from network interface 102 via network interface 810, and / or based on information received via management interface 814, for the in-flight packet and in response to receipt of the in-flight packet, respectively.

[0153] In addition to the other information described above, memory 806 may store various information used by the packet filtering appliance 800, such as CTI noise information, global threat context information received from GTCS 170 via network interface 810, domain name popularity information, and / or any other information received from SOC 160, SPMS 150, and / or CTIP 140.

[0154] The functions and steps described herein may be embodied in computer usable data or computer executable instructions, such as one or more program modules, executed by one or more computers or other devices (e.g., computing device 700, such as packet filtering appliance 800) to perform one or more functions described herein. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types when executed by one or more processors of a computer or other data processing device. The computer executable instructions may be stored on a computer readable medium (e.g., storage 702, such as memory 806), such as a magnetic disk, optical disk, removable storage medium, solid state memory, random access memory (RAM), ready-only memory (ROM), flash memory, etc. As will be appreciated, the functionality of the program modules may be combined or distributed as desired. In addition, the functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. Particular data structures may be used to more effectively implement one or more aspects of the present disclosure, and such data structures are contemplated to be within the scope of the computer-executable instructions and computer-usable data described herein.

[0155] Although not required, those skilled in the art will appreciate that various aspects described herein may be embodied as a method, system, apparatus, or one or more computer-readable mediums storing computer-executable instructions. Thus, aspects may take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, or an embodiment combining software, hardware, and firmware aspects in any combination.

[0156] As described herein, the various methods and operations may operate across one or more computing devices and networks. Functionality may be distributed in any manner, or may be located on a single computing device (e.g., a server, a client computer, etc.).

[0157] Aspects of the present disclosure have been described in terms of exemplary embodiments thereof. Numerous other embodiments, modifications, and variations within the scope and spirit of the appended claims will occur to those skilled in the art from review of this disclosure. For example, those skilled in the art will understand that the steps illustrated in the exemplary figures may be performed in other than the order recited, and that one or more illustrated steps may be optional. Any and all features of the following claims may be combined or rearranged in any manner possible. For another example, those skilled in the art will understand that references to the IPv4 protocol in the description of this specification and the exemplary figures may be replaced with references to other protocols, such as the Internet Protocol version 6 (IPv6) protocol.< / metadata> < / metadata> < / metadata> < / metadata>

Claims

1. 1. A method comprising: receiving, by a packet filtering appliance, a plurality of packet filtering rules determined based on a plurality of threat indicators determined based on cyber intelligence reports from a plurality of cyber threat intelligence (CTI) providers; receiving, from a first network, a packet in transit destined for a second network; determining an observation time of the in-flight packet based on determining that the in-flight packet matches a first packet filtering rule of the plurality of packet filtering rules; determining a disposition based on the observation time; and and applying the disposition to the in-flight packet.

2. determining a command based on the observation time of the in-flight packet; The method of claim 1 , further comprising: applying the instructions.

3. 2. The method of claim 1, wherein the in-flight packet is one of a plurality of in-flight packets received by the packet filtering appliance at a particular packet rate, and wherein the determining of the disposition is performed within a time frame that is less than or equal to the inverse of the particular packet rate.

4. The method of claim 1 , wherein the determining the disposition is performed statelessly.

5. 2. The method of claim 1, further comprising receiving, by the packet filtering appliance, a second in-flight packet from the first network destined for the second network, the second in-flight packet being received by the packet filtering appliance immediately after the in-flight packet, and wherein the determining of the disposition is completed prior to receiving the second in-flight packet.

6. 2. The method of claim 1, wherein the determining the observation time of the in-flight packet is further based on a determination that the first packet filtering rule does not indicate a predetermined disposition to be applied to a matching packet.

7. determining that a multi-packet, multi-flow attack is active at the time the in-flight packet is received by the packet filtering appliance; determining whether the in-flight packet is a member of the multi-packet, multi-flow attack based on determining that the in-flight packet matches the first packet filtering rule; The method of claim 1 , wherein the determining the disposition is further based on whether the in-flight packet is determined to be a member of the multi-packet, multi-flow attack.

8. 2. The method of claim 1, wherein the second network is a network protected by the packet filtering appliance, and the determining the disposition is further based on determining whether the in-flight packet is associated with an attack originating in one or more networks other than the second network.

9. 1. A method comprising: receiving, by a packet filtering appliance, a plurality of packet filtering rules determined based on a plurality of threat indicators determined based on cyber intelligence reports from a plurality of cyber threat intelligence providers; receiving, from a first network, a packet in transit destined for a second network; determining threat context information based on an observation time of the in-flight packet based on determining that the in-flight packet matches a first packet filtering rule of the plurality of packet filtering rules, the first packet filtering rule indicating a predetermined disposition to be applied to a matching packet; determining a disposition based on the threat context information; and and applying the disposition to the in-flight packet.

10. determining a command based on the threat context information; and The method of claim 9 , further comprising: applying said instructions.

11. The determining the threat context information comprises: determining that a multi-packet, multi-flow attack is active at the time the in-flight packet is received by the packet filtering appliance; determining whether the in-flight packet is a member of the multi-packet, multi-flow attack based on determining that the in-flight packet matches the first packet filtering rule; 10. The method of claim 9, wherein the determining the disposition based on the threat context information is based on whether the in-flight packet is determined to be a member of the multi-packet multi-flow attack.

12. 1. A method comprising: receiving a plurality of packet filtering rules, each indicating a criterion and an action to be taken, by a packet filtering appliance configured to process packets in transit crossing a boundary between a first network and a second network; the packet filtering rules are determined based on a plurality of threat indicators determined based on cyber threat intelligence reports from a plurality of cyber threat intelligence providers; receiving a first packet filtering rule of the plurality of packet filtering rules indicating a first criterion and an action is determined after an in-flight packet matching the first packet filtering rule is received; receiving, by the packet filtering appliance, a first in-flight packet from the first network and at a first time, the first in-flight packet being destined for the second network; based on determining that the first in-flight packet matches the first criterion of the first packet filtering rule, the packet filtering appliance: determining first threat context information associated with the receiving of the first in-flight packet; determining a first action based on the first threat context information; and applying the first action to the first in-flight packet; receiving, by the packet filtering appliance, a second in-transit packet from the first network and at a second time destined for the second network; based on determining that the second in-flight packet matches the first criterion of the first packet filtering rule, the packet filtering appliance: determining second threat context information associated with the receiving of the second in-flight packet, the second threat context information having at least one value different from the first threat context information; determining a second action based on the second threat context information and independent of the determining of the first action; and applying the second action to the second in-flight packet.

13. The method of claim 12 , wherein the first action comprises a first disposition and the second action comprises a second disposition that is different from the first disposition.

14. 13. The method of claim 12, wherein one of the first action or the second action is to allow the first in-transit packets to proceed to the second network, and the other of the first action or the second action is to block the second in-transit packets from proceeding to the second network.

15. The method of claim 12 , wherein the first action includes a first instruction and the second action includes a second instruction that is different from the first instruction.

16. 13. The method of claim 12, wherein applying the first action to the first in-transit packet is performed prior to the receiving of the second in-transit packet.

17. 13. The method of claim 12, wherein the determining the first threat context information comprises determining the first threat context information based on an observation time of the first in-transit packet.

18. 13. The method of claim 12, wherein the determining the first threat context information comprises determining the first threat context information based on whether the first in-flight packet is a member of an attack that is active at the time it is received by the packet filtering appliance.

19. 1. A packet filtering appliance configured to process packets in transit traversing a boundary between a first network and a second network, the packet filtering appliance comprising: one or more processors; one or more non-transitory computer-readable media that store instructions; A packet filtering appliance, wherein the instructions, when executed by the one or more processors, configure the packet filtering appliance to perform the method of any one of claims 1 to 18.

20. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause a packet filtering appliance configured to process packets in transit crossing a boundary between a first network and a second network to perform the method of any one of claims 1 to 18.

Images