Method and system for preventing attacks associated with the domain name system

Abstract

Some denial of service cyber attacks on the Internet's Domain Name System (DNS) operate through false, false, or unregistered domain name DNS requests to resolve domain names that are not registered in the DNS. Other cyber attacks steal sensitive data by encoding data in false domain names or domain names that are otherwise not registered in the DNS, which are transmitted over the network in false DNS requests. DNS gatekeepers can filter packets in transit containing DNS requests and can effectively determine whether the requested domain name is registered in the DNS. When the domain name is not registered in the DNS, the DNS gatekeeper can take one of several protective actions. The DNS gatekeeper discards requests that are determined to be illegitimate, thereby preventing the attack.

Description

Technical Field

[0001] Aspects described herein generally relate to computer hardware and software and network security. Specifically, one or more aspects of the present disclosure generally relate to computer hardware and software for efficiently filtering in-transit packets according to packet filtering rules.

[0002] background

[0003] As the information age continues to develop, network security is becoming increasingly important. Network threats / attacks can take many forms (e.g., unauthorized requests or data transfers, viruses, malware, large amounts of traffic designed to overwhelm resources, etc.).

[0004] In order to deal with these types of threats and attacks, various network defense methods and systems have been developed and deployed. One network defense system uses a packet gateway, such as a Threat Intelligence Gateway (TIG), to protect a network. A packet gateway may include an (inline) network packet filtering device with the ability to apply a set of packet filtering rules to TCP / IP (Transmission Control Protocol / Internet Protocol) packets in transit. A TIG may act as a gateway or interface between a network (e.g., a private network) that has been secured or protected by Network Threat Intelligence (CTI) and a network (e.g., the Internet) that has not been similarly secured / protected.

[0005] Packet filtering rules can be based on a threat indicator database provided by CTI sources including CTI provider organizations. A set of packet filtering rules can represent a security policy enforced by a TIG for protecting / defending a network. Threat indicators can be used as packet matching criteria for identifying risky packets. References to standards herein can refer to a single standard and / or multiple standards. CTI indicators or threat indicators can include Internet network addresses of resources (e.g., IP addresses, IP address ranges, L3 / L4 ports and protocols, domain names, uniform resource identifiers (URIs), etc.), which may be controlled or operated by threat actors or may be otherwise associated with malicious activities. CTI indicators / threat indicators can also include identifiers of certificates and related certificate authorities used to protect certain TCP / IP communications (e.g., X.509 certificates used by the Transport Layer Security (TLS) protocol to protect Hypertext Transfer Protocol (HTTP) intermediary sessions). The CTI provider includes threat metadata associated with each threat indicator, including, for example, threat / attack type, threat name, threat risk score, threat subject (attribution), etc. TIG and associated applications can use threat metadata to further enhance protection against threats and further improve network security posture.

[0006] The network security policy manager can create a network security policy as a set of CTI derived packet filtering rules by receiving threat indicators and associated threat metadata from one or more CTI providers and generating packet filtering rules based on the indicators and metadata. Each packet filtering rule includes at least: (a) criteria for matching packet data, (b) an action or combination of actions to be applied to the packet if the criteria is matched, and (c) threat metadata. The matching criteria may include one or more pairs consisting of a packet field name (e.g., an L3 source IP address field) and a field value. The field value may be a threat indicator in the form of a network address (e.g., an IP address, a domain name, a URI, etc.) or an identifier of a certificate or certificate authority. The action or combination of actions may include blocking or dropping packets, allowing or forwarding packets, logging packets, capturing packets, redirecting or rerouting packets, and modifying packets to protect the network. In the context of TIG and CTI derived policies, rule actions may be referred to as packet transformation functions (PTFs), which can transform packets in a manner that protects the network. For example, a PTF may transform a TCP SYN packet that initiates a connection establishment with a threat endpoint into a corresponding TCP RST packet, thereby stopping the connection establishment process. The threat metadata may be used to calculate a threat risk score, select a threat analysis method to apply to a packet and communications associated with the packet, or select or calculate an action to apply to the packet based on the threat risk.

[0007] One or more TIGs may be inserted inline, for example, in an Internet access link of one or more networks. A policy manager of one or more TIGs may be configured to: (a) receive CTI (threat indicators and associated metadata) from one or more CTI providers; (b) generate one or more policies consisting of packet filtering rules derived from the threat indicators and metadata; (c) download one or more policies or transmit one or more policies to one or more (subscribing) TIGs, which may be associated with the policy manager via policy subscriptions. As packets (in either direction) traverse the Internet access link, the TIGs may enforce policies by applying packet filtering rules to each packet in transit.

[0008] The effectiveness of protecting a network using TIG and associated CTI generally depends on the scope and quality of CTI indicators and metadata applied by TIG, as well as the performance of TIG. At the current time of disclosure, there are a total of millions of threat indicators (and associated threat metadata) available from multiple CTI sources, in the form of IP addresses, 5-tuples, domain names, URIs, etc., as well as certificate identifiers, certificate authority identifiers, etc. These large CTI databases can be converted into a similar number of packet filtering rules. Since the threat indicators actually observed in the communication traffic of a given network are not necessarily known, TIG can apply all available CTI or associated data packet filtering rules at any time to ensure effective protection. TIG policies can consist of millions, tens of millions, or hundreds of millions of rules. Therefore, TIG must be able to apply millions of packet filtering rules to each packet in transmission without adversely affecting network performance. 10G and 40 / 100G network links can be used at the TIG insertion point, which means that the packet transmission rate can be millions or tens of millions of packets per second. TIG must filter each packet in transmission through millions of rules to enforce the policy without adversely affecting network performance. For such large policies and high packet rates, even if the policies are stored in main memory and even if a fast (e.g., sublinear) policy search algorithm is used, TIG performance may be insufficient and may result in unacceptable performance degradation. For example, high latency may result in packet loss due to buffer overflows. Latency incurred during packet filtering should be low, and packets should not be dropped (e.g., due to buffer overflows).

[0009] TIG can be designed to achieve the necessary performance through some combination of the following: (a) high-performance CPUs that can be architected or configured for network packet processing; (b) fast / efficient algorithms and associated data structures for searching rules for matches between rule indicators and field values ​​associated with packets currently in transit; and (c) storing policies (which may include millions of packet filtering rules) in high-speed local memory (e.g., on-board SDRAM, commonly referred to as main memory), so that the CPU can quickly access the main memory via a high-speed, high-bandwidth data bus.

[0010] However, recent network trends make TIG difficult or practically impossible to perform at the levels required to maintain effective network protection. The growth rate of the number of CTIs (measured by the number of threat indicators and associated metadata), and therefore the growth rate of the associated policies (measured in bytes, which is a function of the number of packet filtering rules and the byte size of those rules), is higher than the rate of improvement of processor speeds, processor costs, processor power requirements, memory speeds, memory densities, memory costs, etc. As a result, it is no longer practical for TIG to handle increases in policy size by increasing the size of memory. This problem is further compounded by ever-increasing network link speeds and ever-increasing network traffic. Therefore, there is a need to improve the efficiency of TIG technology so that large increases in the size of CTI-derived policies require only relatively small increases in computing resources, which is necessary to maintain TIG performance at acceptable levels (e.g., relatively low latency and minimal to no packet loss) while providing effective network protection.

[0011] Many attacks can be related to the Internet's Domain Name System (DNS). DNS is sometimes referred to as the "phone book" of the Internet, and DNS is a database distributed across the Internet that maps human-readable domain names to machine-readable IP addresses. Many popular Internet applications, including the World Wide Web (www) and e-commerce applications, rely heavily on DNS to implement the routing of services across the Internet's routers and interconnected link infrastructure, such as web browser requests and web server responses. Therefore, successful attacks on DNS that result in poor DNS performance may cause these Internet applications and the Internet itself to stop operating effectively, or otherwise severely impair their operation. Conversely, DNS requests can be used as an attack vector for certain other types of attacks, including distributed denial of service (DDoS) attacks on DNS, which may cause damage and loss to network assets. Therefore, there is a need for efficient DNS protection that can (a) prevent attacks on DNS and (b) prevent DNS from being used as an attack vector.

[0012] Overview

[0013] A brief summary is given below to provide a basic understanding of some aspects of the present disclosure. It is neither intended to identify the key or important elements of the present disclosure nor to delineate the scope of the present disclosure. The following summary only gives some concepts of the present disclosure in a simplified form as a preface to the following description.

[0014] Various aspects of the present disclosure relate to efficient packet filtering for network threat intelligence (CTI) applications. Packet filtering methods can be time and space efficient. In fact, only a small fraction of the packets filtered by any packet gateway will match any CTI-derived packet filtering rules in the security policy executed by the gateway. Therefore, a time-efficient method of determining whether a packet matches any rule (or does not match) before searching the rules of the security policy can eliminate a large number of policy searches. If the method is faster than searching through the policy, the expected search time may be significantly reduced when a sufficient amount of representative packet traffic is averaged. In addition, a security policy can include multiple rules that are identical except for the associated threat indicator values. These rules have the same actions and threat metadata. Therefore, space (e.g., memory resource) efficiency can be achieved by associating multiple different indicators with a single rule (e.g., the same action and threat metadata).

[0015] Aspects of the present disclosure also relate to efficiently preventing some attacks associated with the DNS. The attack vector is a DNS query request message containing a domain name that may not be registered in the DNS. Hereinafter, the phrase "DNS query request message" may be abbreviated as "DNS request", etc. The terms "DNS query request message" and "DNS request" may refer to a message, a TCP / IP packet containing the message, or both. In addition, a "DNS query response message" may be similarly abbreviated or referred to as a "DNS response", etc.

[0016] Aspects of the present invention provide a method for processing a packet including a Domain Name System (DNS) query request. The method may include: receiving a plurality of packets in a computing device such as a host or a proxy (e.g., an endpoint or an intermediate proxy or a network device); determining whether each of the plurality of packets contains a DNS query request based on packet header information; and testing a probabilistic data structure to determine whether the domain name of the DNS query request is represented in the probabilistic data structure. The probabilistic data structure, which may be a Bloom filter, may be configured with a data set representing all domain names known to be registered in the DNS. The method may determine whether the DNS query request indicates a legitimate DNS query request based on determining that the domain name of the DNS query request is not represented in the probabilistic data structure, and may discard the DNS query request based on determining that a packet in the plurality of packets does not indicate a legitimate DNS query request.

[0017] When it is determined that a packet includes a DNS query request, and if it is determined that the domain name of the DNS query request is not represented in the probabilistic data structure, the method can determine whether the DNS query request indicates a legitimate DNS query request. DNS query requests may include legitimate DNS requests, and may include some illegal requests. Illegal requests may include erroneous requests, such as requests that belong to legitimate attempts to resolve domain name requests, and may include false requests, such as requests submitted in attempts to deny services to others and / or leak information from compromised systems. Data about DNS query requests determined to be illegal can be transmitted to a management server. Packets determined to include legitimate requests can be transmitted to their intended destinations.

[0018] The computing device may monitor network traffic for responses to the transmitted DNS query requests. Based on determining that the responses to the DNS query requests indicate that the DNS query requests have been successfully completed, the computing device may insert the domain name of the monitored DNS query requests into the probabilistic data structure. The computing device may transmit the domain name of the successfully completed DNS query requests to the management server.

[0019] In some embodiments, in order to determine whether a DNS query request indicates a legitimate DNS query request, a computing device may determine the rate of invalid DNS query requests received. The computing device may also determine the rate of change of the number of invalid DNS query requests received. If such a rate meets a threshold, the computing device may determine that the packet being processed does not indicate a legitimate DNS query request. The computing device may also determine whether the DNS query request indicates a legitimate DNS query request based on the country code associated with the DNS query request or the timing of the update received for the probability data structure. The computing device may also determine whether the DNS query request indicates a legitimate DNS query request based on the syntactic features of the domain name in the request. The computing device may determine that a packet associated with an unregistered domain may be associated with a lag in the registration of the relevant domain name. The computing device may determine that a packet associated with an unregistered domain may be associated with a legitimate data exchange service. The computing device may also determine whether the DNS query request indicates a legitimate DNS query request based on a determination of the probability that the request is legitimate, which may be based on one or more of the aforementioned factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present disclosure is particularly pointed out in the appended claims. Features of the present disclosure will become more apparent by reviewing the disclosure as a whole, including the drawings provided herewith.

[0022] Some of the features herein are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like reference numerals refer to similar elements and in which:

[0023] Figure 1 An illustrative environment of an efficient Threat Intelligence Gateway (TIG) and associated policy server for protecting at least one of a plurality of different networks is depicted;

[0024] Figure 2 Describes the functional and system components of an efficient TIG with a memory hierarchy;

[0025] Figure 3 is a flow chart describing the general operational concepts of configuring a TIG for efficient operation and then efficiently filtering packets accordingly;

[0026] Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8 , Fig. 9 , Fig.10 , Fig.11 , Fig.12 and Fig.13 depicts an exemplary sequence of events illustrating a method for configuring and operating a high-efficiency TIG according to one or more illustrative aspects described herein;

[0027] Fig.11 Further describing the Figure 1 An exemplary sequence of events occurring in the depicted environment where TIG is configured to effectively protect a network from attacks on the Internet DNS;

[0028] Fig.14 ——It can be used with Figure 1 Same or Similar - Describes an exemplary network environment of a DNS attack prevention system for protecting both the DNS and portions of a network that uses the DNS from threats and attacks;

[0029] Fig.15 Depicting exemplary components of a DNS gatekeeper (DNS-G / K) protection device;

[0030] Fig.16 A flow chart depicting an exemplary operational concept of DNS-G / K;

[0031] Fig.17 Another flow chart depicting an exemplary operational concept of DNS-G / K;

[0032] Fig.18 A flowchart depicting the concept of operation of the DNS Filter Manager;

[0033] Fig.19 An exemplary sequence of events illustrating a method for preventing certain attacks on the DNS is depicted;

[0034] Fig. 20 Describes a sequence of events that illustrates a method for preventing certain DNS tunneling attacks;

[0035] Fig.21 depicts an exemplary computer system architecture that can be used in accordance with one or more exemplary aspects described herein; and

[0036] Fig. 22 An illustrative network system architecture is depicted that can be used in accordance with one or more illustrative aspects described herein.

[0037] Detailed Description

[0038] In the following description of various illustrative embodiments, reference is made to the accompanying drawings that form a part of this specification, and in the accompanying drawings, various embodiments in which various aspects of the disclosure may be practiced are shown by way of illustration. It should be understood that other embodiments may be utilized and structural and functional modifications may be made without departing from the scope of the disclosure. In addition, reference is made to specific applications, protocols, and embodiments in which various aspects of the disclosure may be practiced. It should be understood that other applications, protocols, and embodiments may be utilized and structural and functional modifications may be made without departing from the scope of the disclosure.

[0039] Various connections between elements are discussed in the following description. These connections are general and, unless otherwise indicated, may be direct or indirect, wired or wireless, physical or logical, virtual or software defined. Similarly, elements (including hosts and devices) may be physical or virtual. In this regard, this specification is not intended to be limiting.

[0040] A Bloom filter can be a time-efficient and space-efficient probabilistic data structure for determining membership of an element in a set. A Bloom filter is parameterized by (1) the number of elements in the set, N; (2) the storage size or number of storage device / memory bits, M, required to store the N elements; and (3) a false positive rate, P. The false positive rate, P, is the probability that a membership test for an element X in a set falsely returns a match value of True (e.g., a membership test determines that X is a member of the set when, in fact, X is not a member of the set). The value of M depends on N and the false positive rate, P. For a fixed value of N (the number of elements in the set), the storage size, M, is inversely proportional to the false positive rate. That is, as the false positive rate, P, decreases, the storage size, M, increases. The false positive rate can be selected for a Bloom filter in the context of a CTI application and associated CTI-derived policies.

[0041] For example, for a set of 500 million domain names and a false positive rate P of 0.01 or 10 -2The storage size M of a Bloom filter may be about 600MB. In comparison, the average size of a domain name registered in the Internet DNS is about 20 bytes. Therefore, a simple uncompressed list of 500 million domain names requires about 10GB of storage space, which is about (10G / 600M) = 16.66 times the storage size M. If the false positive rate P is reduced by a factor of 1000 to 0.00001 or 10 -5 , then the storage size M increases to about 1.5GB, a factor of only 2.5 (=1.5G / 600M). This is space efficient. In general, the storage size M scales with the logarithm of the false alarm rate P. The set membership test algorithm is also time efficient, with the theoretical time complexity function also on the order of the logarithm of the false alarm rate P. In practice, the implementation of the set membership algorithm can be very fast and in almost constant time, even if the false alarm rate P varies by several orders of magnitude.

[0042] In order to achieve space efficiency and time efficiency, probabilistic data structures such as Bloom filters can be used to avoid unnecessary packet filtering rule searches or to avoid duplicate storage, for example, duplicate storage of common rule actions. For example, 500 million domain names can be stored in a Bloom filter of approximately 600MB in size (when the false alarm rate P=0.01). In practice, if each rule averages 100 to 1000 bytes (for example, 20 bytes for domain name indicators, 30 bytes for specifying actions and related options, and 50 to 950 bytes for metadata), a policy consisting of 500 million packet filtering rules (with domain name indicators) may require, for example, 50GB to 500GB of storage / memory. That is, compared with a packet filtering rule-based policy without a Bloom filter, a Bloom filter may require less storage / memory, which is reduced by 2 to 3 orders of magnitude. The use of Bloom filters can eliminate redundant storage associated with storing actions and metadata separately for each rule by grouping multiple threat indicators associated with the same operator or action (e.g., a blocking action) and the same metadata, instead storing the actions and metadata only once for a Bloom filter containing multiple threat indicators. That is, instead of storing each rule and the associated rule action, the rule action can be associated with a particular Bloom filter and stored a single time as a rule action to be executed based on a test indication match of the Bloom filter. As a result, the memory / space requirements of a policy integrating Bloom filter technology can be relatively much smaller, perhaps by orders of magnitude, than the memory / space requirements of a group filtering rule policy that does not integrate Bloom filter technology.

[0043] For CTI applications, the false negative rate of a Bloom filter is always zero (regardless of the non-zero false positive rate P), which maintains the same level of policy enforcement or security as without a Bloom filter. That is, if the test to determine whether element X is a member of set I returns negative or false, then X is definitely not a member of set I. Therefore, while a TIG using Bloom filter technology to achieve time efficiency may unnecessarily search a policy / filter a packet in the case of a probability of a false positive rate P, the TIG will never skip a policy search or fail to filter a packet when there is a matching rule in the policy.

[0044] Although the term Bloom filter may be used throughout the specification, the selection of filters or data structures is exemplary and is not intended to be limiting or restrictive in any way. Any data structure with sufficient time and space efficiency for testing set membership and maintaining security can be used. For example, a Cuckoo filter is a probabilistic data structure with similar time and space efficiency to a Bloom filter. A Cuckoo filter also has the ability to effectively delete or remove elements from a set, while a standard Bloom filter does not have the ability to remove elements from a set (although Bloom filter variants that support deletion capabilities have been developed). The ability to effectively insert, update, or delete elements from a set may prove useful in some applications, implementations, and / or embodiments. Security policy updates may include updates that add or delete rules, or updates that change rule actions associated with specific threat indicators. For example, an update may change a rule action from a monitoring action to a discard action. The data structure may support a variety of functions, such as an Insert() function for adding elements to a set, a Boolean Member() function for testing the set membership of an element, and a Delete() function for removing elements from a set. The following description also assumes that the data structure is probabilistic and therefore associated with a (non-zero) false alarm rate P, but again, this choice is exemplary and not intended to be limiting.

[0045] Bloom filter technology can improve the efficiency of CTI application / TIG functions in a variety of ways. Eight exemplary methods are described below (in addition to the methods already provided above). For simplicity, the first six exemplary methods involve indicators in the form of network addresses (e.g., IP, domain name, URL, etc.), the seventh method involves indicators in the form of certificate identifiers, and the eighth method is general with respect to indicator types.

[0046] In the first example, Bloom filter technology can be used to improve performance. For each rule in the filter rule set to be searched, the associated index can be extracted and the index can be inserted into the policy Bloom filter. The obtained policy Bloom filter can be stored in the main memory of the packet filtering device. Before searching the packet filtering rule set to find the match of the network address of the packet in the current transmission, the packet filtering device can determine whether the network address is a member of the policy Bloom filter. If the result of the determination is TRUE (true) or indicates a match, the packet filtering device can search the corresponding rule set to find a matching rule and can apply the rule action to the packet. If the packet filtering device determines that the result is FALSE (false) or does not indicate a match, the device can skip unnecessary searches for the packet filtering rules (for example, because no matching rules will be found). By skipping unnecessary searches, the average policy search time per packet can be reduced and TIG performance or throughput can be improved. This probability data structure or Bloom filter can be referred to as a time efficiency probability data structure or Bloom filter, or a strategy probability data structure or Bloom filter.

[0047] In a second example, a policy may include a subset of packet filtering rules that are identical except for their matching criteria or criteria associated with threat indicators. A rule may have the same action, the same threat metadata, and the same one or more matching criteria (which may, for example, be associated with one or more threat indicators). The space (e.g., memory usage) efficiency of policy storage may be achieved by associating multiple different indicators with a single rule (e.g., the same action and / or threat metadata). For example, a TIG or policy management server may classify or divide multiple rules (or a portion of a security policy) of a security policy into subsets based on the type of threat indicator associated with the rule and / or based on the action and threat metadata associated with the rule. A subset associated with a single type of packet matching criteria or threat indicator may avoid unnecessary searches for rules associated with the subset. The search for a subset probability data structure or a Bloom filter may be skipped when the packet does not include the type of packet matching criteria associated with the subset. A subset associated with a single rule action may avoid unnecessary duplication of rule actions and unnecessary storage of separate rules, which may save memory space. For each such subset, the associated subset Bloom filter can be configured to contain all indicators of the associated rules, and the subset Bloom filter can be further associated with the common rule actions and threat metadata of the associated rules. The common rule actions and threat metadata of the subset can be stored once for the probabilistic data structure, rather than being stored and associated with each rule for each rule of the subset. Then, for each such subset, the associated subset Bloom filter can replace the subset of rules in the policy, thereby reducing the space requirements for storing the policy. Therefore, the policy can be represented as a collection of these probabilistic data structures or Bloom filters. During the policy search, the packet filtering device tests whether the network address of the packet is a member of any subset probabilistic data structure or Bloom filter. If the packet filtering device determines that there is a match through the subset Bloom filter test, the action and threat metadata associated with the subset Bloom filter can be applied to the packet. Time efficiency can also be achieved if the time to test elements such as network addresses for membership in the subset Bloom filter is less than the time required to search for subsets of rules in other ways. In some applications, it may be useful to include a packet filtering rule that is applied when a Bloom filter membership test returns FALSE or determines that there is no match between a packet and a set. That is, a subset Bloom filter may have two associated packet filtering rules, including a first rule that is applied when a membership test returns TRUE to indicate a match and a second rule that is applied when a membership test returns FALSE to indicate no match. Such a subset Bloom filter may be referred to as a space-efficient Bloom filter.

[0048] In a third example, since TIG policies may become too large to be stored in available main memory, it may become impractical to fully increase the main memory capacity due to factors such as commodity costs, layout and density constraints, and power or thermal limitations. Auxiliary memory that may be slower (e.g., measured in read access time) but correspondingly larger, denser, and cheaper (e.g., per byte) than the main main memory may be added to the TIG. Such auxiliary memory may require some other RAM-like properties, such as byte addressability, to support fast search algorithms. The capacity of the auxiliary memory can be sized to store policies that are too large to fit into the main main memory. To ensure that TIG performance is adequate, the ratio of the auxiliary memory speed to the main main memory speed may be equal to or greater than the (expected) fraction of packets that will match the CTI-derived packet filtering rules. A time-efficient Bloom filter that stores all indicators for the rules in the policies stored in the auxiliary memory may be stored in the main main memory. Before searching the policies stored in the auxiliary memory, the packet filtering device may perform a fast initial search of the Bloom filter. If the initial search of the Bloom filter indicates a match, the packet filtering device may perform a policy search and packet filtering via the auxiliary memory. If the initial search of the Bloom filter does not indicate a match, the (relatively slow) search of the packet filtering rules in auxiliary memory may be skipped.

[0049] In a fourth example, it may be advantageous to store certain portions of the Bloom filter and packet filtering policy in a high-speed memory (e.g., a primary main memory) and store other portions / remainders of the policy in a low-speed memory (e.g., an auxiliary memory). For example, it may be advantageous to store a first packet filtering rule set with domain name and URI indicators in an auxiliary memory and a second packet filtering rule set with IP address and IP address range indicators in a primary main memory, wherein a time-efficient Bloom filter corresponds to two rule sets stored in the primary main memory. It may be advantageous to prioritize certain types of rules (e.g., using a higher speed memory) depending on the type of traffic associated with a particular protected network. For example, using a primary memory to store packet filtering rules with indicator types that are expected to be searched more frequently (such as IP addresses and IP address ranges) may be beneficial relative to storing rules with indicator types that are expected to be searched less frequently (such as domain names and URLs). In a TCP / IP network, each packet contains an IP address in the L3 / IP header, while a small portion of the packets contain domain names and an even smaller portion contain URLs. Therefore, if the primary memory is not sufficient to store all filter rules in one policy, it may be beneficial to store rules with IP address indicators in the primary memory, and rules with domain name and URL indicators in the secondary memory, for example. For example, it may be beneficial to use the primary memory to store rules associated with popular protocols (such as HTTP / HTTPS (web) on port 80 / 443) and higher frequency matching CTI rule attributes (such as rules associated with lower fidelity threat indicators and rules associated with a large number of indicators (by type)), such as filter rules for IP address and IP address range indicators that may be searched more frequently than filter rules for domain name and URL indicators.

[0050] In a fifth example, it may be advantageous to integrate the above methods in any combination and variations of the methods to achieve further efficiencies. For example, a time-efficient Bloom filter may be associated with a subset of CTI-derived packet filtering rules, and the subset may have some portions of the rules represented by a space-efficient Bloom filter. In practice, the combinations and variations that achieve greater efficiencies may depend on many factors, including factors such as CTI characteristics, performance requirements, cost, and operating environment.

[0051] In a sixth example, a Bloom filter or combination of filters as described above can be used to protect the Domain Name System (DNS) from certain network attacks. DNS is a service that converts human-readable domain names into machine-readable IP addresses. DNS can be viewed as a large, globally distributed memory / store of {domain name, IP address} pairs and an associated processing system for converting human-readable domain names into machine-readable IP addresses. Many popular applications, such as the World Wide Web (WWW), rely on DNS for efficient operation. Some network attacks against DNS use fake DNS query requests as an attack vector. A fake DNS query request is a request to resolve a domain name that is not registered in the DNS.

[0052] There are at least two types of attacks on DNS, which can use false DNS query requests and can be mitigated by the processing methods described in this article. First, a distributed denial of service (DDoS) attack on DNS works by having many malware-infected devices issue false DNS query requests for domain names that are not registered in DNS. Such domain names may be generated in part by a domain generation algorithm (DGA), which can generate random alphanumeric strings that are unlikely to match domain names registered in DNS. DNS may search in vain for matches that will never be found in globally distributed databases, which may require significantly more work / computing resources than searching for domain names registered in DNS. If a sufficient number of these false requests are issued at the same time, DNS may spend a lot of computing resources to try to provide services for these false requests, resulting in insufficient computing resources to provide services for legitimate DNS requests, and may cause denial of service. Many networked applications (such as web) may use DNS, so an effective DDoS attack on DNS may act as an indirect attack on legitimate Internet applications that use DNS. If applications (such as web browsers) are denied DNS service, these applications do not know how to address packets so that packets can be routed to target domains (such as web servers) through the Internet.

[0053] Bloom filter technology and related efficiency methods can be used to mitigate such attacks. There are approximately 500 million domain names registered in the DNS. A (space-efficient) Bloom filter with a storage size M of approximately 600MB and a false alarm rate P=0.01 can be configured with N=500 million registered domain names. Copies of the Bloom filter can be distributed to, for example, (inline) packet filtering devices, such as TIG, which are configured to check whether the domain name in any DNS query request is registered in the DNS. If the Bloom filter test indicates that the DNS filter indicates a match (e.g., True), the packet filtering device can forward the associated packet to the destination of the packet (e.g., a DNS server). If the Bloom filter test indicates that the DNS filter indicates a mismatch (e.g., False), the packet filtering device can discard the packet (and can, for example, generate and forward a corresponding packet containing a DNS query response with an appropriate return code). By discarding the packet, the search for the DNS is skipped. By positioning these packet filtering devices configured with Bloom filters at strategic points, for example, at Internet access links, especially Internet access links of networks operated by DNS infrastructure providers, many false DNS query requests can be dropped / never served. As a result, many searches for DNS are skipped, which can sufficiently mitigate DDoS attacks so that legitimate DNS query requests can be served at an acceptable rate.

[0054] The second type of DNS attack is an exfiltration attack, which can exploit DNS by encoding stolen sensitive information as (fake) domain names in DNS query requests. These fake requests are sent to collection servers, which may impersonate DNS servers, or may be legitimate DNS servers controlled by malicious parties, which may extract and log the domain names in the requests. By extracting and logging the stolen information that appears in the DNS query requests, sensitive information is leaked or stolen. This type of network attack can be called a DNS tunneling attack.

[0055] To prevent such attacks, note that the domain names in such tunnel attack requests are unlikely to map to domain names registered in the DNS. Therefore, to prevent such attacks, a packet filtering device configured with a (space-efficient) Bloom filter (referred to as a "DNS Bloom filter") contains all domain names registered in the DNS, which can be deployed, for example, at an Internet access link of a protected enterprise network. Each DNS query request can be filtered by a packet filtering device configured with a DNS Bloom filter. The packet filtering device can determine whether the domain name in any DNS query request is registered in the DNS based on a test of the DNS Bloom filter. If the packet filtering device determines that the DNS Bloom filter indicates a match (e.g., the membership test returns a True result), the packet filtering device can forward the associated packet to its destination (e.g., a DNS server). If the packet filtering device determines that the DNS Bloom filter indicates a mismatch (e.g., a False result), the packet filtering device can discard the associated packet, which can prevent DNS tunnel attacks.

[0056] For the above-mentioned DNS attack mitigation applications, it may not be necessary to use CTI provided by the CTI provider. Instead, these DNS attack mitigation applications can become a source of CTI data. When the affiliation test indicates a mismatch (e.g., a false result), it is determined that the tested domain name is not registered in the DNS. Therefore, the domain name can be identified as a threat indicator. Therefore, the DNS attack mitigation application can be regarded as a source of CTI, rather than a consumer of CTI.

[0057] In a seventh example, Bloom filter technology can be used to improve performance and security by applying threat intelligence derived from certificates used to protect certain network communications. For example, the Hypertext Transfer Protocol Secure (HTTPS) protocol used for secure web / HTTP communications can use the TLS protocol to securely tunnel HTTP. During the TLS tunnel setup handshake, X.509 certificates can be exchanged between two network endpoints. These certificates may have been issued by a certificate authority (CA) organization. However, the certificates and CAs may be associated with attacks or otherwise associated with threat risks.

[0058] Thus, CTI can be provided and determined for certificates and CA data. For example, CTI can be associated with a revoked certificate. A CA may decide to revoke a certificate that the CA has issued because the CA may believe that the certificate has been compromised in some way. For example, a private key used during certificate generation may be determined to be stolen. A CA may publish a certificate revocation list (CRL) containing serial numbers of revoked certificates. Relevant threat indicators may include a source and certificate data pair that uniquely identifies a certificate (e.g., {CA-Identifier, Serial Number}, where "CA-Identifier" corresponds to the "Issuer Name" value of an X.509 certificate issued by the CA). The CA may provide a server that is used to download the CRL upon request or may be queried about the revocation status of the certificate. For example, a TLS endpoint application (e.g., a web browser) and an intermediary (e.g., an SSL / TLS proxy) may use a CRL and a CRL server during the communication session tunnel setup to determine whether a certificate associated with a communication session has been revoked. When it is determined that a certificate associated with a communication session has been revoked, the communication session tunnel setup may be terminated, alerted, monitored, or otherwise processed in order to mitigate the risks associated with using a certificate that may have been compromised.

[0059] In the case where endpoints do not typically check certificate revocation status, the resulting TLS tunnel may be easily decrypted by malicious entities. To mitigate this threat, a CTI provider can collect CRL data issued by multiple public certificate authorities, can convert CRLs into threat indicators (e.g., {CA identifier, serial number} pairs), and can provide the resulting CTI for use by subscribers. These subscribers may include TIG and / or a security policy manager that provides services to TIG. Since certificate-based CTI can contain millions of indicators (there are approximately 20 million revoked certificates), TIG may use time-efficient and space-efficient Bloom filters and associated packet filtering rules for this CTI when configuring packet filtering policies for protecting the network. When a packet containing a certificate is transmitted through TIG, TIG can extract the issuer name data (which can correspond to the CA identifier) ​​from the certificate and extract the serial number from the certificate. TIG can filter the packet through an appropriate Bloom filter based on the {CA identifier, serial number} pair to determine the revocation status of the certificate. If the certificate has been revoked, TIG can apply the associated packet filtering rules to the associated packet determined to include the revoked certificate.

[0060] In the eighth example, for example, during a network attack event, one or more rules may need to be inserted into an actively executed policy. The packet filter rules can be inserted into a policy currently executed (inline) by a packet filter device (e.g., TIG, network firewall, router access control list (ACL), etc.) on network packet traffic. For performance and integrity reasons, the packet filter rules of the policy can be arranged, sorted, indexed and / or otherwise optimized in a manner that inserts new packet filter rules while maintaining optimality, performance and integrity, which may be an expensive operation considering time and computational complexity, and may result in the allocation of additional memory and / or the memory space reorganization of data (rules) stored in the memory. For example, a set of packet filter rules can be sorted and indexed in a manner that supports (sublinear) binary search. Inserting a new packet filter rule into a set can result in the allocation of memory for storing new rules and the re-indexing of the set to support binary search. Therefore, it may be impractical to insert new rules into the active policy without causing a (temporary) stop of the packet filtering service, which may result in loss of packet transmission service, and / or possible packet loss due to buffer overflow, and / or loss of applications or functionality derived from the packet filtering service (e.g., network security).

[0061] Data structure filtering techniques can be used to address dynamic rule insertion issues of new memory allocation and reorganization that can result in packet transmission service disruption and / or loss of security. During policy creation or based on changes in security policy, at least one empty space-efficient Bloom filter can be provided for rule insertion during active policy execution by a packet filtering device. The size (M) of such a dynamically populated Bloom filter can be set to include up to a certain number (N) of elements (e.g., threat indicators) while keeping the false positive rate (P) below a specified value. For example, a dynamically populated, initially empty Bloom filter associated with a "Block" action can be provided and can allow the insertion of up to 100,000 indicators (of any type) with a false positive rate of 10 -12 . When a new metric is (effectively) inserted into a Bloom filter, no additional memory may need to be allocated to store the new metric, nor may the metrics in the set need to be reorganized. Thus, adding a new rule to a policy does not result in any new memory allocation or any spatial reorganization of the data in memory, and so there is no loss of service or security. A side effect of metric insertion is an increase in the false positive rate of the Bloom filter. However, when fewer than 100,000 metrics are inserted into such a dynamically populated Bloom filter, the false positive rate will be less than the desired maximum of 10 -12 When inserting more than 100,000 indicators, the false positive rate may exceed the maximum false positive rate of 10-12 , but this may not result in a loss of service or security. When the size of the set exceeds a threshold (e.g., the time complexity of the search may vary with the logarithm of the false alarm rate P), the insertion operation may not be considered effective and will not result in any new memory allocation or any spatial reorganization of data in memory. Therefore, the packet filtering service, and thus the packet transmission service, will not be adversely affected by inserting rules into the active policy.

[0062] Another approach to addressing the dynamic rule insertion problem is to allow new threat indicators and rules to be inserted into actively enforced policies without providing dynamically populated, initially empty Bloom filters. New rules can be added or inserted into active policies by inserting the associated indicators into appropriate space-efficient Bloom filters. As additional indicators are added, the false positive rate of the associated filter will increase. Such a Bloom filter can have a threshold limit on the additional indicators, and the limit can be associated with a threshold false positive rate that is considered to be tolerable for the associated application. The threshold false positive rate can be set by a system or policy administrator.

[0063] The first method of inserting rules during active policy execution uses a dynamically fillable, initially empty Bloom filter, which may be more effective when the number of rules expected to be inserted is relatively large compared to the number of rules in the policy (at policy creation time), and when false positive rates must be strictly maintained or have relatively low tolerances. The second method of inserting rules during active policy execution may be more effective when the number of rules expected to be inserted is relatively small compared to the number of rules in the policy (at policy creation time), and when false positive rates have relatively high tolerances. As mentioned above, standard Bloom filters do not support deleting elements from a set. Therefore, for applications that may require rule deletion, a (probabilistic) data structure that has similar properties to a standard Bloom filter but also supports valid element deletion should be used to represent the policy, such as some Bloom filter variants, cuckoo filters, etc.

[0064] Bloom filters can also be used to ensure privacy for certain CTI applications. For example, consider a first organization, Org X, which collects threat intelligence / threat indicators about threat subjects in the form of network addresses. The first organization, Org X, may want to distribute a collection of these indicators to a second organization, Org Y, so that Org Y can search for / detect these indicators in network communication traffic. However, in some cases, such indicators may be considered highly sensitive (e.g., confidential or sensitive personal information); therefore, Org X may not want to publish / distribute the indicators in a form that may compromise the privacy of the indicators. Org X may not want Org Y to be able to identify the indicators in the collection, for example, in case Org Y leaks them to malicious subjects or allows them to be stolen by malicious subjects. However, Org X wants Org Y to be able to detect when these indicators appear in network communication traffic.

[0065] Bloom filter technology can be used to achieve these multiple goals. Org X can insert (Insert()) indicators into sensitive data bloom filter B, and can use an indicator encoding algorithm to populate sensitive data bloom filter B. The Insert() function encodes a set of indicators in such a way that it is computationally intractable to invert the encoded indicators and thereby determine the indicators in the set, even if the encoding algorithm used by the Insert() function is known. Therefore, Org X can distribute sensitive data bloom filter B to Org Y without compromising the privacy of the indicators contained in B. Furthermore, if sensitive data bloom filter B or the bit vector of the data structure is stolen by a malicious subject, the malicious subject is also unable to determine the indicators in the set. At the same time, Org Y can use sensitive data bloom filter B to detect the presence of indicators in network communication traffic by extracting network addresses in the traffic and then testing the network addresses for membership in B.

[0066] Furthermore, a first organization, Org X, may wish to control which external entities are able to perform membership tests on sensitive data Bloom filter B. For example, in some cases, Org X may wish that a second organization, Org Y, be able to perform membership tests on sensitive data Bloom filter B, but Org X may not wish that a third organization, Org Z, associated with a certain set of threat indicators be able to test the membership of these indicators in sensitive data Bloom filter B if Org Z has obtained a copy of the bit vectors contained in sensitive data Bloom filter B. In other words, Org X may not wish for Org Z to know that Org X has collected threat intelligence associated with Org Z.

[0067] In order to perform a membership test Member() on a sensitive data bloom filter B, the Member() function must use the same encoding algorithm A as the algorithm used by the Insert() function to populate the sensitive data bloom filter B with indicators. The encoding algorithm A is constructed from a specific arithmetic combination of certain selected hash functions and simulated hash functions. Thus, Org X may secretly and / or securely share encoding algorithm A with Org Y, and may distribute only the bit vectors contained in B, rather than B itself, to Org Y. Org Y may perform a membership test on the sensitive data bloom filter B. Org X may not share encoding algorithm A with Org Z, such that even if Org Z obtains a copy of the bit vectors contained in B, Org Z cannot perform a membership test on the sensitive data bloom filter B.

[0068] The subject matter described herein can be implemented using any combination of hardware, software, or firmware. For example, the subject matter described herein can be implemented using a non-transitory machine-accessible and readable medium having machine-executable instructions for controlling the machine to perform each step stored thereon. Exemplary machine-readable media suitable for implementing the subject matter described herein include disk memory devices, chip memory devices, programmable logic devices, application-specific integrated circuits, and downloadable electrical signals. In addition, the machine-readable medium implementing the subject matter described herein can be implemented using a single device or computing platform, or can be distributed across multiple devices or computing platforms. Other details and features will be described in the following sections.

[0069] Turning to the subject of DNS, for many networked applications (such as applications of HTTP intermediaries of the World Wide Web (www) etc.), DNS is not a strict requirement for effective operation on the Internet or other networks. As used herein, DNS can be public or private, so that, for example, the DNS discussed herein can be associated with the Internet or an intranet. However, in practice, the DNS of the Internet may be necessary because many Internet networked applications usually rely heavily on DNS to be able to quickly and effectively resolve domain names to IP addresses. Therefore, if DNS is not readily available to provide services for these networked applications, the networked applications may become unavailable. Since the global economy and society have relied heavily on these networked applications, when some Internet applications cannot operate effectively, disastrous consequences may result.

[0070] If a malicious actor is able to successfully attack the DNS itself, the malicious actor could cause great damage to the global economy and society. The fact that such attacks have already occurred has publicly exposed the fact that the DNS does not have adequate protection measures to prevent such attacks. One type of direct attack on the DNS may use a large botnet, such as a large number of infected Internet of Things (IoT) devices, to send many DNS query requests to resolve domain names that may not be registered in the DNS. Each such request causes the DNS to search its globally distributed database before determining that the domain name is not in the DNS and therefore cannot be resolved to an IP address. When a sufficient number of these malicious unresolvable requests are sent at a high enough rate, the DNS may spend most of its resources to serve these malicious requests, while insufficiently serving legitimate requests or refusing to provide sufficient service for legitimate requests, which may cause the associated legitimate network applications to become unavailable. This type of attack is an application-level DDoS attack against the DNS.

[0071] Unregistered domain names may also be used to perform some leakage attacks to steal sensitive data via the Internet. For example, a malicious subject may have infiltrated a private enterprise network and installed malware that will find and collect sensitive information of potential value, such as a credit card information database. The malicious subject may wish to retrieve / transmit and collect such sensitive information via the Internet without being detected by the enterprise network management. That is, the malicious subject may attempt to leak information. One method of leaking sensitive information, such as credit card information (e.g., card1234123412341234) is to encode the sensitive information in a false domain name (e.g., card1234123412341234.www.example.com), insert the false domain name in the DNS query request message, and send the false DNS query request message to an external host posing as a DNS server, which is owned / operated or otherwise controlled by the malicious subject. Since enterprises often configure network firewalls to allow most or all outbound DNS query requests to any external destination, this type of attack may usually be successful. It is unlikely that a false domain name will match a domain name registered in the DNS. Therefore, it is possible to identify DNS attacks using unregistered domain names as attack vectors.

[0072] It is important to note that DNS requests with unregistered domain names are sent to the DNS for some legitimate, non-malicious reasons. For example, because the DNS was built and implemented as a "lookup" service for certain types of information (i.e., translating domain names into IP addresses), some companies and their products use the DNS's lookup service for their own purposes, where unregistered domain names are used to transmit and exchange information. Such companies typically register a domain in the DNS, such as "leverage-DNS.net", and then configure and operate a DNS authoritative name server for "leverage-DNS.net". The DNS will therefore route DNS requests for resolving domain names ending with "leverage-DNS.net" to that authoritative name server. For example, hosted application instance X can issue a DNS request containing the (unregistered) domain name "lookup.5h6di89wp77.leverage-DNS.net". The DNS will route this DNS request to the name server "leverage-DNS.net". The name server may execute logic that, for example, extracts the label "5h6di89wp77" from the domain name, searches the local database for a record indexed by the keyword "5h6di89wp77", and then returns the record in a DNS response corresponding to the DNS request. This DNS response may be routed to application instance X, thereby completing the data exchange.

[0073] Practical implementations of the present disclosure should take into account such non-malicious uses of DNS requests with unregistered domain names. For example, a name database of an authoritative name server used for legitimate data exchange may be maintained and accessed by an application that wants to determine whether a particular unregistered domain name may be part of a legitimate data exchange operation. Practical implementations should also take into account situations where such associated systems may be exploited in some way by malicious entities.

[0074] In the following description, a general method for protecting DNS from attacks using unregistered domain names included in DNS requests as attack paths is introduced. The system may include DNS attack prevention logic that may be deployed in a device inserted at a key location in the network or configured to be run by one or more devices in the network. For example, the DNS attack prevention logic may be installed in a (transparent or non-transparent) computing device (e.g., device) in a network path, which may be located at or near the initiator of the DNS query request, or may be installed near the DNS name server that resolves the DNS request, or may be installed on an endpoint, or may be installed on a proxy that can terminate a DNS session. For example, protocols that use encrypted DNS sessions (such as DoT (RFC 7858) and DoH (RFC 8484)) may require the installation of DNS attack prevention logic in a man-in-the-middle of a proxy device. Each of these devices and associated DNS attack prevention logic may include a DNS gatekeeper function (hereinafter referred to as "DNS-G / K") that may control access to the DNS and its resources. DNS-G / K may execute DNS attack prevention logic and associated algorithms. The DNS attack prevention logic may detect a DNS query request message in transit (contained in a packet in transit), and the DNS attack prevention logic may determine a domain name contained in a qualified name or QNAME field.

[0075] The DNS attack prevention logic can determine whether the domain name is registered in the DNS based on the analyzed message without querying the DNS. If the DNS attack prevention logic determines a True value, indicating that the domain name is registered in the DNS, the DNS attack prevention logic can forward the packet containing the DNS query request message to its destination. That is, the DNS attack prevention logic can forward the query request message to a DNS resource, such as an authoritative name server. Otherwise, when the DNS attack prevention logic determines a False value, indicating that the domain name is not registered in the DNS, the DNS attack prevention logic can perform a DNS protection action. For example, the DNS attack prevention logic can discard packets containing erroneous DNS query requests (such as misspelled domain names) or false DNS query requests (which may be part of a leak attempt or a DDoS attack) to avoid wasting DNS resources or avoiding the transmission of stolen data. The DNS attack prevention logic may generate a corresponding DNS query response with a response code such as "NXDOMAIN" and send the response back to the source of the erroneous or false DNS query request, or may log and capture the erroneous or false DNS query request and may: (i) report the source IP address of the erroneous or false DNS query request so that network management agencies can check whether the source host is infected with malware and, if necessary, remove the malware or otherwise prevent the host from attacking the DNS; and / or (ii) report the unregistered domain name so that management agencies / analysts can check for potential attacks using the domain name as a pathway and / or potential attacks using the domain name as training data for threat intelligence and / or machine learning systems, and / or as input to global DNS / Internet protection systems, etc. Based on a variety of factors described below, for network protection purposes, not generating a DNS response or preventing a DNS response from being sent to the source of the request may be more protective.

[0076] The test for the existence and / or registration of a domain name in the DNS discussed above can minimize or eliminate the use of DNS resources to handle erroneous requests, and can also be advantageously as close to instantaneous as possible relative to packet transmission time and network performance metrics. That is, the delay added by the test and the entire DNS-G / K function should be relatively low, otherwise it may not be implemented, and should be efficient in terms of computing time, space / memory, and processing resources. Recalling the above, for example, a probabilistic data structure called a Bloom filter can meet these criteria. A Bloom filter (a) stores elements of a set in space efficiently, (b) inserts elements into a set in time efficiently, and (c) determines in time efficiently whether an element is a member of a set. A Bloom filter is parameterized by a false alarm rate P, which can include the probability that a set membership test for the element returns True when the element is not actually a member of the set. For a Bloom filter, the false alarm rate is zero. That is, if the set membership test of an element returns False, it can be determined that the element is not a member of the set.

[0077] Bloom filters can be used to test the existence / registration of a domain name in DNS to efficiently determine whether a given domain name is a member of the set of all domain names currently available for registration in DNS. For example, when the false positive rate P = 10 -6 When , a Bloom filter of about 1.8 GB in size is sufficient to store 500 million Bloom filter-encoded domain names. This Bloom filter containing 500 million elements can instantly determine whether a given domain name is a member of the set, even when the false positive rate P is very low (e.g., 10 -6 or one in a million). The space and time requirements scale with the logarithm of P. Therefore, let the false alarm rate P = 10 -6 Reduce by a factor of one million to P = 10 -12 Increasing the false positive rate to 1 in 1 trillion will only result in a 2x increase in space and time requirements. For comparison, at the current time of publication, the DNS contains about 500 million registered domain names. For comparison, considering that the average size of a domain name in the DNS is about 20 bytes, the memory requirement for storing 500 million unencoded domain names is about 10 GB; and the above P = 10 -6 The Bloom filter requires about 1.8GB of memory.

[0078] For the remainder of this specification, a Bloom filter (hereinafter abbreviated as "B / F") data structure will be used in the description of the present disclosure, but this choice is exemplary and is not meant to be limiting or restrictive in any way. Also as described above, any data structure with sufficient and similar time and space efficiency can be used to store a set of elements, insert elements into a set, and test the membership of an element in a set. For example, a cuckoo filter has similar time and space efficiency to a Bloom filter, and also has the ability to effectively delete or remove elements from a set - a capability that a standard Bloom filter does not have. Some Bloom filter variants that support deletion capabilities have been developed. The ability to effectively delete elements from a set can prove useful in some applications, implementations, and / or embodiments. The data structure can support an Insert() function for adding elements to a set, a Boolean Member() function for testing the set membership of an element, and a Delete() function for deleting elements from a set. The following description also assumes that the data structure is probabilistic and is therefore associated with a non-zero false alarm rate P, but again, this choice is exemplary and is not intended to be limiting in any way.

[0079] A list of all domain names registered in the DNS can be created by aggregating lists and sublists of registered domain names, often referred to as zone lists or zone files. These lists can be obtained from multiple DNS registry operator organizations or related representatives, from authoritative name servers that discover or publish their zone files, and from domain list aggregator services. However, some registrars for country code zones typically do not publish their zone files to any third party. Therefore, some registered country code domains may need to be discovered and published by other means, such as web crawling. Such domains may be discovered by third-party service providers. Regardless of the collection method used, it may not be possible to obtain a 100% complete list of registered country code domains. This uncertainty may be taken into account in the DNS protection logic of the DNS gatekeeper DNS-G / K.

[0080] A Bloom filter (also and / or alternatively referred to as a DNS-REG or Bloom filter DNS-REG) may be created from this list. The Bloom filter DNS-REG may be created by a centralized service or server, which may generate the Bloom filter DNS-REG and distribute copies of the Bloom filter DNS-REG and associated DNS protection logic to subscribers, which may include one or more DNS gatekeepers (DNS-G / K). The centralized service may be associated with a management server for a subscription service that manages the Bloom filter DNS-REGs. The management server may store multiple Bloom filter DNS-REGs and may be configured to assign a specific Bloom filter DNS-REG to each device registered for the service. The management service may select a specific Bloom filter DNS-REG based on the processing capabilities of the subscribing device.

[0081] The Bloom filter DNS-REG may contain all domain names registered in the DNS and may be associated with a logic function. The logic function may include a Boolean function Insert-Into-DNS-REG() that accepts a domain name as input, may insert the domain name into the set DNS-REG, and may return a True value if the insertion operation is successful, or may return a False value if the operation is unsuccessful. The logic function may also include a Boolean test function Member-of-DNS-REG() that accepts a domain name as input, and may return a True value if the input domain name is an element or member of the Bloom filter DNS-REG, or may return a False value if the input domain name is not an element of the set. Another logic function may be a delete element function Delete-from-DNS-REG(), which may delete a specified element from the set DNS-REG. The delete element function may be useful for certain applications, including updating the Bloom filter DNS-REG when a domain name is deregistered from the DNS.

[0082] The DNS gatekeeper DNS-G / K may include logic for ingesting / receiving packets in transit, for buffering and queuing packets, for forwarding or discarding packets containing DNS requests, for extracting domain names (e.g., QNAME field values) from DNS requests, for managing local copies of DNS-REG and related subscriptions to centralized services, for calling functions Insert-Into-DNS-REG() and Member-of-DNS-REG() and Delete-From-DNS-REG(), for logging and reporting Member-of-DNS-REG() calls that return False, for deciding whether to generate a corresponding DNS response and forward it to the source or initiator of an erroneous or false DNS request, etc. The DNS-G / K may be included or embedded in a host operating system, such as Linux, that manages DNS requests on behalf of an application instance. For example, the DNS-G / K logic may be included in a web browser from which a DNS request is initiated / originating / issuing, or embedded in a DNS stack (such as BIND), which is a known DNS server software that can be used in both a DNS authoritative name server and a DNS recursive resolver. DNS-G / K may be implemented as a micro-service that manages the Bloom filter DNS-REG and filters DNS requests through the Bloom filter DNS-REG.

[0083] These DNS gatekeepers (DNS-G / K) may be embodied in devices and host computers (including endpoint elements) that may be widely distributed across the Internet and may be inserted into the network path of DNS requests, including path terminations at source and destination endpoints. These devices and hosts may include DNS proxies, DNS servers, DNS resolvers, DNS name servers, network firewalls, network proxies, network gateways, network devices, routers, switches, endpoints, etc. DNS-G / K may also be deployed in Internet-connected networks that are operated by DNS registrar companies, DNS infrastructure and hosting service providers, and other organizations that may operate DNS registration services or DNS name servers for multiple DNS registrar companies or other organizations that may cause many DNS requests to enter and leave their networks.

[0084] Since the DNS may continuously register new domain names, the Bloom filter DNS-REG may be updated regularly (e.g., hourly, daily, or weekly updates). The Bloom filter DNS-REG may be updated by a centralized service or management server and distributed to DNS-G / K subscribers via a push or pull mechanism. Alternatively or simultaneously, the centralized service or management server may collect newly registered domain names from a registrar organization or its representative, from a third-party domain list aggregator organization, from an authoritative name server that finds its zone file, and / or from a DNS agency such as ICANN. The newly collected domain names may be provided to DNS-G / K subscribers in a feed, or may be provided to DNS-G / K subscribers as part of an update package. When a DNS-G / K subscriber receives such a feed, the subscriber may insert the domain name into their local copy of the Bloom filter DNS-REG by calling the Insert-Into-DNS-REG() function or generating an updated DNS-REG. Similarly, a centralized service or management server may collect newly deregistered domain names and may provide a feed containing the latest / recently deregistered domain names to DNS-G / K subscribers. When a DNS-G / K subscriber receives such a feed, the DNS-G / K subscriber may delete the domain name from its local copy of the Bloom filter DNS-REG by calling the Delete-From-DNS-REG() function.

[0085] The DNS gatekeeper DNS-G / K may include logic for handling false positives and may include logic for handling hysteresis issues associated with the Bloom filter DNS-REG, which will be described in more detail below. Such logic may be used to replace logic for forwarding packets containing DNS query request messages to their destinations or logic for performing the above-mentioned DNS protection actions.

[0086] Regarding false positives, as described above, a Bloom filter is a probabilistic data structure that can be parameterized by a false positive rate P. The false positive probability P is associated with a membership test that determines whether an element is contained in a Bloom filter, and is the probability that the membership test may erroneously return a True value. The Bloom filter can be an instance of a Bloom filter DNS-REG. The logic can determine whether the domain name of the packet being tested is registered in the DNS. The system can perform a membership test in the Bloom filter DNS-REG for a given domain name, where the false positive probability is P. The membership test may erroneously return a True value when the domain name is not actually contained in the Bloom filter DNS-REG. Logic may be executed to forward a packet containing a DNS query request message to its intended destination, which may forward an erroneous or false DNS request to its destination. That is, potential attacks associated with erroneous or false DNS requests may not be prevented or mitigated.

[0087] In the case where erroneous or false DNS requests are part of a DDoS attack on the DNS, the adverse impact of a false positive test result may be small. This is because, assuming a reasonable value for the false positive rate P, it is likely that most false DNS requests will be detected. When most false DNS requests are detected, those false requests that are detected will not result in false positives and can therefore be blocked / dropped or otherwise handled as in the logic that performs DNS protection actions. Therefore, false positives are unlikely to result in a successful DDoS attack because a large number of false requests must be allowed for a DDoS attack to be successful. If the DNS requests are part of a DNS tunneling attack, such as as part of a data exfiltration operation, the adverse impact of a false positive may be larger, depending on the sensitivity of the leaked information and / or the dependency of the leaked information on other information that is blocked / dropped in the logic that performs DNS protection actions. For example, if the false positive contains a stolen root password, the adverse impact may be large. Conversely, if the false positive contains a small portion of a larger file, the exfiltrated information may be of little value because its value depends on the context of the rest of the file, and because the rest of the file with associated information is contained in a separate DNS request, the rest of the file may not have been successfully exfiltrated, which is determined to be a true positive and may be blocked / discarded by the logic that performs DNS protection actions.

[0088] The false alarm rate P can be selected so that false alarms are unlikely to occur in practice. For example, the false alarm rate can be selected so that the false alarm rate P = 10 -12 or smaller. The trade-off is that the smaller the false alarm rate P, the more memory is required to store the Bloom filter DNS-REG. Memory storage imposes a practical limit on the false alarm rate P. However, different DNS-G / K subscribers may have different requirements and constraints on the false alarm rate P, and different requirements and constraints on the memory / storage device of the Bloom filter DNS-REG. Therefore, the centralized service or management server that provides the Bloom filter DNS-REG can also provide different versions of the Bloom filter DNS-REG, which can be parameterized by different false alarm rates P, and thus different memory sizes of the Bloom filter DNS-REG. The centralized service or management server can allow each subscriber to select which version of the Bloom filter DNS-REG the subscriber wants to access or deliver, and the selection criteria is a combination of the false alarm rate P and the associated memory size.

[0089] A centralized service that may be provided by a management server may select or assign a specific Bloom filter DNS-REG based on the processing power of the computing device. The management server may be configured to retrieve memory capacity information from a third-party device, such as from a product data sheet from a manufacturer's website or a sales website. The management server may determine a specific Bloom filter DNS-REG to assign to a computing device based on the memory size of the computing device, a false positive rate P limit of a security policy associated with the computing device, or a latency limit of the computing device.

[0090] The management server may also receive information about DNS request processing from multiple different DNS-G / K subscribers. This information may include domain names of monitored DNS query requests that are not found in a particular local Bloom filter DNS-REG, but are confirmed to include valid domain names when the request is allowed to proceed and is confirmed as valid by the DNS-G / K based on a response from the DNS. The information may also include record data related to DNS requests that are determined to be false by testing a particular local Bloom filter DNS-REG or by monitoring DNS responses to DNS query requests that are allowed to proceed to their intended destination. The information may include at least the source, intended destination, and other packet information of such DNS query requests.

[0091] The DNS protection logic of DNS-G / K may also consider the value of the false alarm rate P together with other parameter values. When deciding how to handle a DNS request, other parameter values ​​may include a DNS request rate value (described below). For example, the processing of the True instance of Member-of-DNS-REG() may be different, depending on the value of the false alarm rate P associated with the local Bloom filter DNS-REG. For example, compared to the second DNS-G / K of the local DNS-REG with a lower P value, the first DNS-G / K of the local Bloom filter DNS-REG with a relatively high false alarm rate P is more likely to conclude that the True result of the membership test may be a false alarm. Therefore, the first DNS-G / K may handle the associated DNS request differently than the second DNS-G / K. For example, the first DNS-G / K may decide to discard the DNS request anyway due to the relatively high possibility of false alarms and in combination with other parameter values ​​(such as the DNS request rate value and other determined attack status values), while the second DNS-G / K may forward the request to the DNS.

[0092] Regarding the lag mentioned above, delays may occur when an instance of the DNS-REG does not contain the same set of registered domain names as the actual DNS. For example, domain name XYZ may have been recently registered in the DNS, but domain name XYZ may not have yet been distributed by a centralized service or management server as an update to the distributed system that implements the Bloom filter DNS-REG. That is, at a particular point in time, DNS-G / K may not have downloaded the latest available version of the Bloom filter DNS-REG. This type of delay can be referred to as synchronization lag. Another example is a type of delay that may occur with respect to country code domains. As mentioned above, since the registrar of the country code zone does not typically reveal its zone files to any third party, non-deterministic methods such as web crawling can be used to compile a list of country code domains, with the result that a 100% complete list of registered country code domains is not available for a particular device or system. This type of delay can be referred to as registration delay or country code delay.

[0093] Based on these types of lags, for a given execution instance of the Member-of-DNS-REG() test, it may be determined whether domain name XYZ of the packet being tested is in the DNS-REG, returning False, indicating a determination that domain name XYZ is not registered in the DNS. Based on this determination, logic for performing DNS protection actions may be executed, and the logic may be configured to drop or block the DNS request being tested. When domain name XYZ is registered after the last update of the DNS-REG, this test does not reflect the validity of the recently registered domain name XYZ. Dropping valid DNS requests due to delays in updating the DNS-REG may be referred to as "lag-drop".

[0094] The adverse effects of delayed discarding can depend on the intention of the subject that issued the associated DNS request. For example, if a legitimate or non-malicious subject issues a valid DNS request but is discarded by DNS-G / K due to hysteresis, the adverse effects of delayed discarding can be considered significant because legitimate business transactions may not be completed. This example shows that in the logic that performs the above-mentioned DNS protection actions, it may not be desirable to discard or block DNS requests. On the contrary, some DNS requests for resolving recently registered domain names may be associated with malicious activities, in which case the effects of delayed discarding may be considered harmless, desirable, or beneficial. For such requests associated with malicious activities, it is desirable for the logic to perform DNS protection actions to discard or block DNS requests. As described above, in some embodiments, it is impractical to determine whether the False return value of Member-of-DNS-REG() is due to hysteresis or because the input domain name is a false name that was never intended to be registered in DNS. In practice, a False return value is more likely to be due to a false DNS name than a hysteresis problem; therefore, in the logic that performs DNS protection actions, discarding or blocking DNS requests may be more desirable than forwarding or allowing DNS requests.

[0095] An alternative to dropping or blocking in logic that performs DNS protection actions may be to forward or allow the DNS request, but to log the DNS request and monitor subsequent network traffic for the corresponding DNS response returned by the DNS. When it has been determined that the associated domain name is not registered in the DNS, the corresponding DNS response may include a response code NXDOMAIN. In addition, when there is no corresponding DNS response, the logic may determine that the request is an erroneous or false DNS request associated with an attack. The network security application may then analyze the logged DNS requests, for example, to determine whether the DNS request may be associated with a DNS tunneling attack or a leakage attack. The logic may determine the probability that the request is associated with a legitimate request, and based on determining that the probability that the request is legitimate exceeds a threshold, the logic may allow the request determined to be legitimate to proceed to the DNS for resolution.

[0096] When the corresponding DNS response contains the IP address for resolving the domain name, or the logic determines that the domain name has been registered in the DNS, it can be determined that the False return value of Member-of-DNS-REG() is associated with hysteresis. For example, if the DNS request contains a domain name suffixed with a country code top-level domain (ccTLD), the logic can determine that the DNS request is associated with a hysteresis problem caused by collecting all country code domain problems. Therefore, the logic can determine that the DNS request is likely associated with communications that may be non-malicious. If the DNS request does not include a domain name with a country code TLD, the logic can determine that any hysteresis problem may be a synchronization hysteresis situation, as discussed above for a DNS request containing a recently registered domain name. Since the logic can determine that the DNS request may be associated with malicious activity, the logic can monitor the DNS request using a network security application to monitor subsequent activities and / or request responses associated with the DNS request. Monitoring can allow the logic to determine whether subsequent activities (such as HTTP(S) sessions associated with the IP address provided by the DNS response) and any associated subjects are malicious. Based on determining that the domain name in the request is valid, DNS-G / K can insert the registered domain name into the local copy of the Bloom filter DNS-REG, and can transmit the domain name determined to be a registered domain name to a centralized service or management server that distributes Bloom filter DNS-REG copies to subscribers.

[0097] When an erroneous or false DNS request that may be associated with a DDoS attack on the DNS is detected, the logic may perform a DNS protection action to drop or block the request to prevent or mitigate the attack. As described above, it may be impractical to determine whether an erroneous or false DNS request is associated with a DDoS attack or other type of attack. However, the logic may estimate whether an erroneous or false DNS request is associated with a DDoS attack by determining when an associated system discovers a high rate of false DNS requests, which is a characteristic of a DDoS attack. If invalid DNS requests are observed when the current rate of invalid DNS requests observed is higher than normal, the logic may determine that a high rate of invalid requests indicates that a DDoS attack is occurring. The rate of invalid DNS requests may be compared to a threshold. When the logic determines that a DDoS attack may be occurring, the logic may perform a DNS protection action logic to drop or block DNS requests. When the rate of invalid DNS requests may be comparable to a threshold, the logic may perform a DNS protection action logic to drop or block DNS requests.

[0098] Rate thresholds may be set for each device. The threshold rate for invalid DNS requests may be determined based on an initial rate of invalid DNS requests during a normal operating cycle (e.g., when no active DDoS attacks are occurring). For example, the logic may monitor the rate of invalid DNS requests over a period of time. Since network traffic may vary depending on the time of day and the day of the week, the rate of invalid DNS requests may be adjusted based on a particular time of day or day of the week. For example, for an enterprise network, the expected rate of erroneous DNS requests during lunch time on weekdays may be higher than the rate of erroneous DNS requests on weekend evenings. For a home network, the expected rate of invalid or false DNS requests during weekend evenings may be higher than the rate of invalid or false DNS requests during lunch time on weekdays. Each specific gatekeeper device may have different rates based on multiple factors, including the location and network associated with the device. For example, a device based on a home network may have significantly lower total traffic compared to an enterprise network. Each specific gatekeeper device may be configured to monitor changes in the rate over time to adjust the expected rate of invalid or false DNS requests.

[0099] The rate threshold may be set relative to the false alarm rate P of the Bloom filter DNS-REG. That is, relative to the false alarm rate P of the Bloom filter DNS-REG, the threshold may be set to allow more possibly false requests to be passed to the intended destination, so as to monitor the response to determine a possible registration lag with respect to the Bloom filter DNS-REG. When the false alarm rate is low, DNS-G / K may set a lower rate threshold, which may allow more possibly false DNS query requests to be forwarded to their intended destination. When the false alarm rate is high, DNS-G / K may set a higher rate threshold, which may allow fewer possibly false DNS query requests to be forwarded to their intended destination.

[0100] Generating the associated DNS responses may exacerbate the DDoS attack by filling the DNS caches in any intermediate DNS servers. When the logic determines that a DDoS attack may be occurring, the logic may drop or block the DNS request without generating the associated DNS response (e.g., a response where RCODE:3=NXDOMAIN). Alternatively, if the logic determines that the request is a false DNS request and the current rate of false DNS requests observed is not higher than normal, the logic may perform a DNS protection action to forward or allow the DNS request, and also log, monitor and / or network analyze the associated events to determine the likelihood of an attack, the severity of the attack, and potential remediation measures. In order to determine whether a DDoS attack is occurring, the logic may determine whether the current rate of erroneous DNS request queries meets a threshold.

[0101] One way to estimate the normal rate of false DNS requests is to measure the time interval between each observation of false DNS requests and then use an efficient algorithm to estimate the average interval (and thus the normal rate). The average interval between false DNS requests observed during a period when a DDoS attack is not occurring may be greater than the average interval between false DNS requests observed during a DDoS attack. The logic can monitor and continuously estimate the average time between observations, or conversely can monitor and continuously estimate the observed rate. These average intervals can be estimated based on an exponentially weighted moving average (EWMA) method or based on an exponential smoothing method. Exponential smoothing is time and space efficient because each update to the estimate after each observation requires only constant time calculations (typically two multiplications and one addition), and typically only three values ​​need to be stored at any time.

[0102] An estimate of the average rate of false DNS requests can be determined to establish a baseline. When an estimate of the average rate of false DNS requests has been determined, a change in the estimated average rate of false DNS requests can be effectively determined. The change can be estimated using, for example, an exponential average calculation applied to an estimated sequence of the average rate of false DNS requests. In the absence of a DDoS attack, the calculated change in the estimated average rate of false DNS requests should be close to zero. When the system determines that there is no DDoS attack, the logic can perform a DNS protection action to forward or allow false DNS requests, while performing a recording, monitoring, or analysis function on the false DNS requests and the corresponding DNS responses (if any). When a DDoS attack is launched, the calculated change in the estimated average rate should be non-zero and positive. When it is determined that a DDoS attack is launched, the logic that performs the DNS protection action can discard or block false DNS requests. In addition, if the calculated change in the estimated average rate of false DNS requests becomes relatively large and positive (increasing), the logic that performs the DNS protection action can be modified to discard false DNS requests without generating a corresponding DNS response with a NXDOMAIN response code.

[0103] This alternative protective action may be implemented because devices that may be generating false DNS requests (including such requests during a DDoS attack) generally do not care about receiving DNS responses. Because the devices generating the requests do not care about the responses, computing resources consumed by the DNS-G / K to generate DNS responses may be wasted, and the effectiveness of a denial of service attack on the DNS-G / K may be increased. When DNS responses with NXDOMAIN response codes may be generated during such a DDoS attack, the DNS cache in the intermediate DNS server may fill up with NXDOMAIN responses to false DNS requests. In this case, DNS responses to legitimate DNS requests (which may include resolved IP addresses) may not be cached on the intermediate DNS server, which may result in increased DNS resource consumption used to serve legitimate DNS requests, which may further exacerbate the DDoS attack. This situation can be considered a denial of service attack on the DNS cache, and by extension, an attack on the DNS.

[0104] When the DDoS attack subsides, the logic can detect a change in the estimated average rate of false DNS requests. The change in the estimated average rate of false DNS requests can be determined to become negative when the attack begins to subside. Based on determining that the attack has begun to subside, when false DNS requests are detected and dropped, the logic can begin to perform DNS protection actions to generate DNS responses with NXDOMAIN response codes.

[0105] In the case of determining that no attack is currently occurring, during the initial stage of measuring the false DNS request rate, and / or when DNS-G / K is positioned so that it only observes a relatively small amount of false DNS requests (e.g., at the edge of a well-protected private network), the sample size of false requests may be too small to calculate an accurate estimate. In these cases, as described in more detail above, estimating the rate associated with all DNS requests may also be useful. The system can determine the request rate associated with both false DNS requests and legitimate DNS requests, and can compare the request rate of false DNS requests to the request rate of all DNS requests. When the ratio of false requests to all requests changes significantly, the logic can determine that an attack is occurring. The ratio of false requests to all requests can be used to determine whether the system should generate and transmit a DNS response to a possible false request.

[0106] Figure 1A representative environment 100 is depicted, which may include packet filtering devices such as threat intelligence gateways (TIGs) 200 and 250 for protecting networks 112 and 114 by executing CTI-derived policies provided by a security policy manager (SPM) 120. TIGs 200 and 250 may be inserted inline in a network access link 106 connecting networks 112 and 114 with network 110. Network 110 may be the Internet, which provides interconnection services between networks 112, 114, 116, and 118, and may enable communications between IP addressable hosts 120 and 130 to 137 attached to the network. TIGs 200 and 250 may be physical or logical, and the intra-network and inter-network connections of the networks may be any combination of wired, wireless, physical, virtual, and / or logical / software defined.

[0107] The network 112 may be an exemplary enterprise network that uses CTI-derived policies and TIG 200 implementing aspects of the present disclosure to protect its networked corporate assets, such as very large policies that require the space, time, and resource efficiencies achieved through the present disclosure. The TIG 200 may subscribe to the SPM 120 to obtain CTI-derived policies that are provided by many CTI providers and span a range of indicators, indicator types, attack types, risk values, indicator fidelity, etc., resulting in policies consisting of tens or hundreds of millions of packet filtering rules that may be characterized by their associated threat indicators.

[0108] Network 114 may be an exemplary network operated by a DNS infrastructure and managed service provider organization having networked corporate assets including DNS servers and proxies (e.g., hosts 132 and 133) that handle a large volume of DNS traffic, typically representing organizations that value high-performance DNS services: such as DNS domain name registry operators, authoritative name server providers, large global enterprises, content delivery network providers, ISPs, etc. Network 114 is protected by TIG 250, which may subscribe to SPM 120 for CTI-derived policies and may be configured with packet filtering logic that may determine whether a domain name included in a packet is registered in DNS.

[0109] Network 116 may be a corporate enterprise network that is not protected / secured by TIG and related policies. Hosts 134 and 135 attached to network 116 may be infected with malware or otherwise controlled by a malicious subject.

[0110] Network 118 and associated networked assets may be controlled / operated by an organization of malicious subjects. Network 118 is not protected / secured by TIG and associated policies. The organization may use hosts 136 and 137 attached to network 118 to perform various attacks on remote network assets via network 110 (e.g., the Internet), such as networks 112, 114, and 116.

[0111] Figure 2 The system architecture of the TIG 200 supporting various aspects of the present disclosure is shown. The components may include a processor CPU 220 (which may execute logic for configuring and operating the TIG 200), network interfaces 212 and 214, a management interface 240, and a hierarchical structure of memory modules TIER-0MEM 230, TIER-1MEM 231, ... TIER-N+1MEM 239. The components may be connected to a bus BUS 210, which may be used to transmit information between the components of the TIG 200. The bus 210 may provide a data communication channel between the components of the TIG 200. The bus 210 may vary in form and performance, from a relatively fast and small bus (e.g., an on-chip silicon bus connecting processor logic and on-chip cache memory) to a relatively slow and global bus (e.g., the Internet). Other forms of buses may include, for example, an integrated / embedded data bus of a printed circuit board (PCB), parallel data lines connecting computers and peripherals, serial optical cables connecting ports / interfaces of network switches and routers, L2 / L3 switching networks, L3 routing networks, and the like. The bus 210 may be any combination of silicon, wired, wireless, physical, logical, virtual, software defined, etc. For example, Figure 2 As shown, a portion of bus 210 may be identified by network link 106 connecting network interfaces / ports 212 and 214 to networks 112 and 110 .

[0112] Memory modules 230 to 239 may form a hierarchical structure layered / parameterized by speed (read / write access time), density (bytes per unit volume in 3D space), capacity (measured in bytes), and cost per byte. Due to spatial limitations associated with memory type and location, capacity (measured in bytes) may be highly correlated with density. (Capacity and density may be used interchangeably in this disclosure). Smaller enumerated memory modules (e.g., TIER-0MEM 230) may be associated with higher speed, lower capacity, and higher cost than higher enumerated memory modules (e.g., TIER-3MEM 233). Typically, as the enumeration number increases from 230 to 239, speed and cost decrease, while capacity increases. Parameters of adjacent layers may vary by one or more orders of magnitude. For example, TIER-0MEM 230 may be an on-chip SRAM cache memory; TIER-1MEM 231 may be an SDRAM "main memory" on a PCB; TIER-2MEM 232 may be an NVRAM on a PCB (e.g., a NAND FLASH SSD); TIER-3MEM 233 may be a magnetic mechanical / rotating disk; and so on, where TIER-N+1MEM 239 is identified as a globally distributed memory, such as used by the Internet DNS to store {domain name, IP address} pairs and related information.

[0113] Hierarchical memory can enable cost-effective implementation of the features described herein related to time and space efficiency. Policies or partitions of policies, and any associated Bloom filters, can be stored in the highest speed memory with sufficient space / available capacity. Policies can be stored after processing policy rules to achieve time efficiency and / or space efficiency measures, such as using Bloom filters to exploit rule matching rates / probabilities and rule redundancy. That is, rules or policies can be processed to minimize the size of the Bloom filter, minimize the false alarm rate of the Bloom filter, or eliminate redundant rules within the Bloom filter. If the performance measurement value of the TIG does not meet an acceptable threshold, the TIG can allocate one or more memory layers to the memory hierarchy configured by the TIG. TIG can also provide a hierarchical system of Bloom filters to reduce the number of packet filtering rule searches in the policy without compromising security.

[0114] Figure 3A flowchart describing representative operations for configuring and operating an efficient TIG is depicted. In step 3-1, a packet filtering device, such as a threat intelligence gateway TIG 200, may download a CTI derived policy from a security policy manager SPM 120. The CTI derived policy may include multiple rules that may be preprocessed to eliminate redundancy. The packet filtering device may process or apply the packet filtering rules in the order in which the packet filtering rules are arranged in the policy file (however, if the integrity of the policy is not affected, the rules may be reordered / rearranged). The packet filtering device may process or apply the packet filtering rules in the order in which the rules are spatially arranged. Some features described herein may include processes corresponding to rule ordering. Since TLS certificate messages and other packet data may be segmented across multiple L2 packets, references to "packets" in this description may refer to multiple packets containing a single certificate message or other packet data. For illustrative purposes to help describe the method, some exemplary orderings are described.

[0115] SPM or TIG can sort and group rules to meet certain requirements and / or support TIG functionality. For CTI-derived policies, rules can be sorted / grouped in a way that improves security, improves TIG performance (e.g., reduces policy search time), improves network analysis efficiency, etc. Rules can be sorted and sub-sorted, for example, by indicator type and associated fidelity, or by whether the rules have the same action and threat metadata (but with different indicators). Network address indicator types can be URI, domain name, and IP address, and the associated rules can be sorted in descending order of fidelity. For example, URI rules can precede domain name rules, and domain name rules can precede IP address rules. Similarly, for certificate-related indicator types, certificates have higher fidelity than certificate authorities, which can be taken into account when sorting associated packet filtering rules. Rules can also be sorted and sub-sorted by rule action or by rule (non-)dependencies. For example, rules with block / reject / drop actions can precede rules with non-blocking actions. Order-independent rules can be rearranged without changing the integrity of the policy. Various combinations of sorting and grouping criteria can be used. Rules may be ordered and grouped in a manner that supports / optimizes time and / or space efficiency.

[0116] In step 3-2, according to the methods described herein, the policy can be divided into subsets of rules, where each subset can allow time and / or space efficiencies. For example, if it is known before initiating a search whether the current network address does not match any of the indicators in the rules in the subset, then a subset of rules with indicators that have a relatively low rate of matching the network address of the filtered packet can be expected to achieve time efficiency by skipping the search for matching rules. If each rule in the subset has the same action and the same threat metadata (but with different indicators), another space efficiency can be achieved because the rule subset can be efficiently stored in memory as a set of (different) indicators associated with a single instance having the same action and threat metadata.

[0117] In step 3-3, for each subset, the Bloom filter can be configured to contain each indicator for each rule in the subset. Each Bloom filter can be associated with a subset of rules, and can also be associated with one or more indicator types contained in the filter, so as to allow further efficiency improvements. For example, a Bloom filter and its associated rule subset can contain only URI and domain name indicators; then the packet filtering device can search for policies for rules for IP address type indicators, and can skip the search for URI and domain name subset Bloom filters (and associated rule subsets). For a time-efficient Bloom filter, a pointer points to a location in a (hierarchical) memory where an associated subset of filtering rules can be stored. The time-efficient Bloom filter and its associated rule subset can be stored in the same layer of the memory hierarchy, or it can be the case that the Bloom filter can be stored in a different (e.g., faster) layer of the memory hierarchy than its associated subset, because the rule subset may be too large to be placed in the same layer as its Bloom filter. Therefore, the associated rule subset can be stored in a memory layer with a larger capacity (but slower speed). To achieve space efficiency, a Bloom filter may be associated with a single instance of actions and threat metadata associated with an associated subset of all (different) indicators and packet filter rules in the Bloom filter.

[0118] In some cases, a rule subset may not produce time or space efficiency, and / or may require that the subset be always searched, and / or the rule may not be derived from CTI, or may not be otherwise characterized as a threat indicator. In order to associate a subset Bloom filter with each subset in a policy, a general Bloom filter may be associated with such a rule subset that can always return a TRUE value for any set membership test of any network address / element input. The general Bloom filter may not require any memory / storage space to store the associated rule elements. If each rule in the associated subset includes only indicators of one or more indicator types, the general Bloom filter can be associated with these one or more indicator types. This can produce efficiency when the rules in the associated subset have only indicators of the "IP address (IP address)" type and the current network address being searched may be a URL. Therefore, even if the membership test for whether the URL is in the general Bloom filter can return a TRUE result, the subset search can be safely skipped. The general Bloom filter can be considered a special case of a time-efficient Bloom filter.

[0119] Furthermore, in some cases, it may be advantageous to associate a single time-efficient Bloom filter with multiple subsets or even all subsets (e.g., an entire policy). Testing membership of elements in a single set X may be more time-efficient than performing membership tests separately for multiple subset Bloom filters associated with each subset associated with a partition of the single set X. For example, for a policy that includes a subset J of filter rules associated with domain name indicators and a subset K of filter rules also associated with domain name indicators, where the domain name indicators in J are different from the domain name indicators in K. Two Bloom filters J' and K' may be generated that include the domain name indicators of J and the domain name indicators of K, respectively, and a third Bloom filter L' may be generated that serves as a combined subset filter that includes the domain name indicators of both J and K (each Bloom filter may have the same false alarm rate P). The time to test membership in a single combined subset Bloom filter L' may be less than the sum of the time to test membership in Bloom filter J' and the time to test membership in Bloom filter K'.

[0120] Steps 3-2 and 3-3 are described herein as being performed by TIG 200, but these steps may be performed by SPM 120, or portions of the steps may be divided between TIG 200 and SPM 120. If portions of the Bloom filters may be calculated by SPM 120, then the policy may be downloaded by TIG 200, and the policy file may also include any associated Bloom filters and associated information. Another example is if SPM 120 performs partitioning of the policy into subsets and specifies the type of efficiency or Bloom filter to be used for each subset (e.g., time efficiency, space efficiency, inefficiency (general purpose), etc.), then TIG 200 configures the Bloom filters.

[0121] In steps 3-4, subset Bloom filters and their associated policy subsets may be distributed by the packet filtering device or SPM 120 and may be stored in a memory hierarchy. Bloom filters and their associated subsets may be stored in the fastest memory module with sufficient available capacity. Time-efficient Bloom filters may be stored in a memory module that is as fast as or faster than the memory storing the rule subsets associated with the filter. In terms of memory capacity, Bloom filters may take precedence over subsets of policies. The packet filtering device may determine a memory module for storing a Bloom filter or a rule subset of a policy based on a determined remaining capacity of each memory of the packet filtering device. The packet filtering device may prioritize storing Bloom filters so that rule subsets may not replace Bloom filters for capacity reasons.

[0122] In step 3-5, the TIG 200 may receive at least one packet in transit (via a network interface / device port) to begin the packet filtering process. In step 3-6, a network address and a certificate-associated identifier (if any) that may correspond to a threat indicator of a rule in the policy are extracted from the packet. The network address may be a source IP address and / or a destination IP address in an L3 / IP layer packet header, a domain name in a DNS query request message in an L7 / application layer packet, a URL and a domain name in an HTTP GET request method in an L7 / application layer packet, etc. The certificate-associated identifier may include data extracted from fields contained in an X.509 certificate that may be contained in a TLS certificate message, such as an {issuer name, serial number} pair that uniquely identifies or characterizes an X.509 certificate, and identity data associated with a certificate authority that issued the certificate, which may include the issuer name of the certificate.

[0123] In step 3-7, for each network address and certificate-associated identifier (if any) extracted from the packet, the packet filtering device 200 may search for the network address or other identifier in each subset Bloom filter as appropriate or based on the determination of the associated threat indicator, and the data may be searched in an order corresponding to the order defined by the policy. The search may be performed until (T) the network address is found to be an element / member of the Bloom filter (the match result is true), or until (F) all Bloom filters have been searched without success (no match or match is false). Based on the no match result of the Bloom filter search, it can be determined that the network address or identifier is not a member of any subset of the Bloom filter set, and therefore, there is no filter rule in the policy that will match and no filter rule operator that must be applied. Based on the no match result (F), in step 3-8F, the packet filtering device 200 may stop the search process because it has been determined that there is no rule with a matching indicator in the policy, and the search process may be returned to step 3-5 to filter the next packet in transmission. Based on the matching result (T), in step 3-8T, the packet filtering device 200 can search for a packet filtering rule in a policy subset associated with the Bloom filter in which the indicator matches the network address or identifier. In step 3-9, the packet filtering device 200 can apply the action of the filtering rule to the packet, and can return the packet processing to step 3-5 to filter the packet in the next transmission.

[0124] If multiple network addresses or certificate-associated identifiers are extracted from the packet in step 3-6, the packet filtering device 200 may determine in step 3-8 the order in which each network address or identifier may be searched in the subset Bloom filter based on the fidelity of the metrics associated with each network address and identifier. For example, if a URL and an IP address are extracted from the packet, the packet filtering device 200 may search the URL in the Bloom filter before performing a similar search for the IP address, because (usually) the URL metric has a higher fidelity than the IP address metric. Similarly, if a certificate identifier and a certificate authority identifier are extracted from the packet, the packet filtering device 200 may search the certificate identifier in the subset Bloom filter before performing a similar search for the certificate authority identifier, because (usually) the certificate metric may have a higher fidelity than the certificate authority metric.

[0125] Figure 4An embodiment and associated sequence of events in which the TIG 200 is configured for efficient packet filtering when a policy is divided into two subsets, wherein the division of the packet filtering rules associated with the policy can be determined by the metric type and the likelihood that a packet will match a rule in a subset. A time-efficient Bloom filter can be associated with one subset, while a general Bloom filter can be associated with the other subset. Both subsets of rules and their associated Bloom filters can be stored in the same memory module. This illustration generally corresponds to Figure 3 Steps 3-1 to 3-4.

[0126] In step 4-1, SPM 120 may send a file to MGMT I / F 240, a management interface of TIG (via network 110, which may be, for example, the Internet), containing a policy to be executed by TIG 200 on packets in transit. The policy file may include a header with a policy descriptor, which may include information such as a partitioning scheme for partitioning the policy into two subsets, one or more types of efficiency to be applied to each subset, the size of the policy, the size of each subset, and the type of indicator in each subset. In step 4-2, MGMT I / F 240 may transmit the policy file to TIER-3MEM 233 (via BUS 210), which may be a low-speed memory such as a local mechanical / rotating disk (non-volatile, relatively large capacity, relatively cheap, relatively slow read / write access). In step 4-3, CPU 220 may retrieve the policy file header from TIER-3MEM 233. The policy file header may indicate how the policy is divided into rule subsets, which efficiencies may be applied, and how the subsets may be distributed across multiple memory modules (based on size information if necessary). When CPU 220 processes the policy header file information, CPU 220 may temporarily store some of the information in, for example, TIER-0MEM 230 (which may be identified by an on-chip cache memory), and TIER-1MEM 231 (which may be identified by a main memory on a PCB (not shown)).

[0127] In step 4-4, based on the policy file header information, the CPU 220 can retrieve a subset of the packet filtering rules and can generate an associated subset Bloom filter. For example, the CPU 220 can retrieve the first subset SS-1 from the disk 233, which consists of packet filtering rules with URL and domain name indicators. The policy header information can indicate that a policy wide time efficiency Bloom filter should be generated for the first subset SS-1. The CPU 220 can extract all indicators in the first subset, and can insert the extracted indicator data into the time efficiency or policy Bloom filter B / F-1 (which can be configured for the false alarm rate P). The CPU 220 can associate the Bloom filter B / F-1 with the first subset SS-1, and can associate the Bloom filter B / F-1 with the indicator type ("URL", "domain name") contained in the Bloom filter. In step 4-5, the CPU 220 may analyze the memory capacity and may determine that the main memory 231 has sufficient remaining available capacity to store the Bloom filter B / F-1 and its associated first subset SS-1, and the second subset SS-2. Therefore, the CPU 220 may store the time-efficient Bloom filter B / F-1 and the first subset SS-1 in the main memory 231.

[0128] In step 4-6, based on the policy file header information, the CPU 220 may retrieve the second subset SS-2 from the disk 233, which may be composed of packet filtering rules having indicators of type IP address and IP address range. Based on determining that the policy header information does not specify a specific subset efficiency for the second subset SS-2, the packet filtering device may generate a general Bloom filter B / F-2 of type {"IP address", "IP address range"} to be associated with SS-2. The general Bloom filter B / F-2 may be associated with multiple threat indicator types. Based on the analysis of the memory capacity, in step 4-7, the CPU 220 may store the general Bloom filter B / F-2 and the second subset SS-2 in the main memory 231. When it is determined that each subset of the policy has been processed or stored, the TIG 200 may determine that the configuration of packet filtering for the current policy has been completed.

[0129] Figure 5 Shows Figure 4 A representative use case of a TIG 200 configured in FIG. 1 and the associated efficient packet filtering process. This illustration generally corresponds to Figure 3For example, in step 5-1, a human user operating a web browser hosted by a desktop computer 130 attached to network 112 can initiate a web / HTTP session with a website named www.enterprise-xyz.com hosted by a server 134 attached to network 116. After resolving www.enterprise-xyz.com in the DNS (e.g., to an IP address such as 12.34.56.78), computer 130 can then access the web browser on port 80 / HTTP (not in Figure 5 A TCP connection is established between 130 and 134 on the computer 130 (shown in FIG. 1 ). Computer 130 may forward at least one packet to website 134, which includes an HTTP GET request for http: / / www.enterprise-xyz.com (which may default to the URL of the home page, such as http: / / www.enterprise-xyz.com / index.html).

[0130] In step 5-2, the network interface 212 of the TIG 200 can receive and ingest the HTTP get domain name packet in transit, and can transmit the received packet to the CPU 220 via the BUS 210 for filtering. The CPU 220 can extract data for packet filtering from the request / packet, such as (a) the domain name www.enterprise-xyz.com, and (b) the destination IP address 12.34.56.78. The CPU 220 can sort the extracted data based on the policy rules and can search the policy for each indicator in descending order of the indicator type fidelity defined by the policy. The CPU 220 can search the policy based on a higher fidelity indicator (such as the domain name www.enterprise-xyz.com) to obtain a matching rule. If it is determined that there is no match, the CPU 220 can search for a rule that matches a lower fidelity indicator (such as the IP address 12.34.56.78).

[0131] In step 5-3, the CPU 220 may test the indicator types associated with (and included in) the time-efficiency Bloom filter B / F-1 for the first subset SS-1 of packet filtering rules to determine whether they match the type (domain name) of www.enterprise-xyz.com. Since B / F-1 contains indicators of the "domain name" and "URL" types, the CPU 220 may determine whether www.enterprise-xyz.com is a member of B / F-1. When it is determined that the first subset SS-1 of the policy does not contain a packet filtering rule with the threat indicator www.enterprise-xyz.com, the B / F-1 membership test for www.enterprise-xyz.com returns FALSE or no match value, and the CPU 220 may skip searching the first subset SS-1 of packet filtering rules. For packets including IP addresses, CPU 220 may determine the types of indicators associated with (and contained in) Bloom filter B / F-1 associated with the first subset of packet filtering rules SS-1 to determine whether they match the type (IP address) of 12.34.56.78. Since B / F-1 contains indicators of type "domain name" and "URL", CPU 220 may skip the membership test for B / F-1 (and therefore skip the search of SS-1 rules).

[0132] In step 5-4, CPU 220 may then determine the metric type associated with B / F-2 (i.e., the general Bloom filter associated with the second subset SS-2 of packet filtering rules). Since the metric types associated with B / F-2 are "IP address" and "IP address range", CPU 220 may skip the membership test of B / F-2 for the domain name www.enterprise-xyz.com and may also skip the search for the rules associated with the second subset SS-2. For the packet including IP address 12.34.56.78, CPU 220 may determine the metric types associated with B / F-2 (i.e., the Bloom filter associated with the second subset SS-2 of packet filtering rules) to determine whether they match the type (IP address) of 12.34.56.78. Since there is a match in metric types, CPU 220 may determine whether the IP address associated with packet 12.34.56.78 is contained in B / F-2. When B / F-2 is a general bloom filter, the search may return a TRUE value. CPU 220 may search for a packet filtering rule with an indicator that matches 12.34.56.78 in the second subset of the rules of SS-2. When no match is found, no action or rule operator is applied to the packet, and the packet may be forwarded to its intended destination. Based on determining the mismatch, in step 5-5, CPU 220 may transmit the packet to network interface 214 via BUS 210. In step 5-6, network interface 214 may forward the packet that does not match the packet filtering rule to network 110, toward the target destination website www.enterprise-xyz.com (which is hosted by server 134 having IP address 12.34.56.78).

[0133] In steps 5-7, a human user operating an email application hosted by a desktop computer 131 attached to network 112 may click on a URL link http: / / www.cyber-threat-network.com / download / malware.exe embedded in the phishing email. Based on this action, computer 131 may initiate a web / HTTP session with a website hosted at the domain name www.cyber-threat-network.com by sending packets to server 136 attached to network 118. After resolving the domain name www.cyber-threat-network.com in the DNS to an IP address such as 87.65.43.21, computer 131 may establish a TCP connection between computers 131 and 136 on port 80 / HTTP, and computer 131 may forward packets containing an HTTP GET request for the URL http: / / www.cyber-threat-network.com / download / malware.exe to host 136. In steps 5-8, the network interface 212 of the TIG 200 may receive and ingest the packet in transit and may transmit the packet to the CPU 220 via the BUS 210 for filtering. The CPU 220 may extract (a) the URL http: / / www.cyber-threat-network.com / download / malware.exe and (b) the destination IP address 87.65.43.21 from the request / packet. The CPU 220 logic may be configured to search for indicators in the policy at reduced fidelity. Based on the fidelity determined for each type of threat indicator associated with the packet, the CPU 220 may first search the policy for a rule that matches the URL (e.g., http: / / www.cyber-threat-network.com / download / malware.exe) and then (if necessary) may search for a rule that matches 87.65.43.21.

[0134] In step 5-9, the CPU 220 may determine the indicator type associated with (and contained in) the Bloom filter B / F-1 for the first subset SS-1 of packet filtering rules to compare with the indicator type associated with the received request / packet to determine whether the B / F-1 indicator type matches the type URL (e.g., http: / / www.cyber-threat-network.com / download / malware.exe). When B / F-1 contains indicators of the "domain name" and "URL" types, the CPU 220 may determine whether the URL http: / / www.cyber-threat-network.com / download / malware.exe is a member of B / F-1. When the first subset SS-1 in the policy does contain a packet filtering rule with the threat indicator http: / / www.cyber-threat-network.com / download / malware.exe, the B / F-1 membership test returns a TRUE value. Based on the match determination of the B / F-1 membership test, in step 5-10, the CPU 220 may search the first subset SS-1 of the packet filtering rules to find a matching rule. The CPU 220 applies the action or operator of the rule, which may be to block or drop the packet. The CPU 220 may then stop its search for policies and may return a value, such as a TCP RST packet, to the originating computer to terminate the connection request (not in Figure 5 ).

[0135] Figure 6 An embodiment and associated sequence of events are shown in which a TIG is configured for efficient packet filtering when a policy may be partitioned into two subsets, where the partition may be determined based on the metric type of the rules in the subset and the likelihood that the packet will match the rules in the subset. A time-efficient or policy Bloom filter may be associated with the two subsets. The subsets together may be too large to fit into available main memory 231, although each subset itself may fit into main memory 231. A subset with a lower likelihood of matching a rule may be selected to be stored in a different memory module that has sufficient available capacity but may be slower than main memory 231. Despite the slower memory, TIG performance may be maintained because the time-efficient method of searching the Bloom filter may avoid unnecessary searches of each rule of the subset. This illustration generally corresponds to Figure 3 Steps 3-1 to 3-4. Steps 6-1, 6-2, 6-3 and Figure 4 Steps 4-1, 4-2, and 4-3 are similar, and the detailed description of receiving the policy file is not repeated.

[0136] In step 6-4, based on the policy file header information, the CPU 220 can retrieve the first subset SS-1 from the disk 233, which can be composed of packet filtering rules with URL and domain name indicators. The policy header information can indicate that a time-efficient Bloom filter should be constructed for the first subset SS-1, and the CPU 220 can extract all indicators in the first subset and insert these indicators into the Bloom filter B / F-1 (for example, it has been configured for the false alarm rate P). The CPU 220 can associate the Bloom filter B / F-1 with the first subset SS-1 and the indicator type ("URL", "domain name") of the rules contained in SS-1. In step 6-5, based on the policy header file information, the CPU 220 can determine that the main memory 231 has sufficient available capacity to store the subset SS-1, but does not have sufficient available capacity to store both the subsets SS-1 and SS-2 at the same time. Since there is a low probability that SS-1 will need to be searched, CPU 220 may determine that SS-1 will be stored in secondary memory module TIER-2MEM 232 (which has sufficient available capacity to store the first subset SS-1, but is slower than primary memory 231). Based on the capacity determination, in step 6-5A, CPU 220 may store the first subset SS-1 in memory module 232 and may associate the memory location of SS-1 with its associated Bloom filter B / F-1. In step 6-5B, CPU 220 may store Bloom filter B / F-1 in primary memory 231.

[0137] In step 6-6, based on the policy file header information, the CPU 220 may retrieve from the disk 233 a second subset SS-2, which may be composed of packet filtering rules having indicators of the type "IP address" and "IP address range". The policy header information may indicate that a time-efficiency Bloom filter should be constructed for SS-2. The CPU 220 may extract all indicators in SS-2, and may insert the extracted threat indicators into the Bloom filter B / F-2 (e.g., which has been configured for a false alarm rate P). The CPU 220 may also associate the Bloom filter B / F-2 with the second subset SS-2, and may associate the Bloom filter B / F with the type of indicators it contains ("IP address", "IP address range"). In step 6-7, the CPU 220 may determine that there is sufficient available memory in the main memory 231 to store the Bloom filter B / F-2 and the second subset SS-2. Based on the capacity determination, the CPU 220 may store the rules of B / F-2 and the subset SS-2 in the main memory 231. Based on determining that TIG 200 has completed configuration for efficient packet filtering for the received policy, TIG 200 may begin receiving packets and begin filtering the packets using the generated Bloom filters.

[0138] Figure 7 Shows Figure 6 A representative use case of a TIG 200 configured in FIG. 1 and the associated efficient packet filtering process. This illustration generally corresponds to Figure 3 Steps 3-5 to 3-9. Steps 7-1 and 7-2 are the same as Figure 5 Steps 5-7 and 5-8 are substantially the same, and detailed descriptions of receiving and ingesting packets for processing in steps 7-1 and 7-2 will not be repeated. In step 7-3, CPU 220 may determine the indicator type associated with (and contained in) Bloom filter B / F-1 for the first subset SS-1 of packet filtering rules to determine whether the indicator type matches the type ("URL") of http: / / www.cyber-threat-network.com / download / malware.exe. When B / F-1 contains indicators of the "domain name" and "URL" types, CPU 220 may determine whether the URL http: / / www.cyber-threat-network.com / download / malware.exe of the received packet is a member of B / F-1. When the first subset SS-1 in the policy contains a packet filtering rule with the threat indicator http: / / www.cyber-threat-network.com / download / malware.exe, the B / F-1 membership test returns a TRUE value. In step 7-4, CPU 220 may obtain the memory location of SS-1 in TIER-2MEM 232 from B / F-1, and may search the rules of subset SS-1 to find a matching packet filtering rule (e.g., a rule with the index http: / / www.cyber-threat-network.com / download / malware.exe). CPU 220 may then apply the action or operator specified for the rule, which may be a BLOCK or drop packet. CPU 220 may then stop further or parallel searches for policies, and may return a message or packet to the originating computer indicating that the communication session has been blocked or terminated (this may include, for example, sending a timeout message or a TCP RST signal / packet to the originating computer to terminate the TCP connection).

[0139] Figure 8 An embodiment and associated sequence of events in which TIG is configured for efficient packet filtering when policies are divided into four subsets are shown. Three of the four subsets are represented by space-efficient Bloom filters and associated information. The fourth subset is associated with a time-efficient Bloom filter.

[0140] The first subset of packet filtering rules SS-1 may be represented by a space efficient probabilistic data structure or Bloom filter B / F-1 discussed below, and associated information, such as associated threat indicator types and associated rule actions. The indicators for the rules are elements of the set of all domain names registered in the Internet DNS. These indicators are contained in the DNS Bloom filter B / F-1. The memory module associated with SS-1 may be the Internet DNS itself, which may be considered a globally distributed database of {domain name, IP address} pairs. The memory module may be Figure 2 and Figure 8 TIER-N+1MEM 239 identification. The DNS probabilistic data structure B / F-1 can be associated with two rule actions. As a first rule, when the membership test returns a TRUE value (e.g., the current domain name is registered in the DNS), the action or operator of the associated rule is a conditional operation. If the current packet contains a DNS query request (to resolve the domain name to an IP address), the rule action is to apply a packet conversion function that converts the packet containing the DNS query request into a packet containing a DNS query response (e.g., to resolve the domain name to an IP address). The packet conversion can be completed by forwarding the current packet containing the DNS query request to the Internet DNS (which will generate a corresponding DNS query response (packet)). If the current packet determines the ELSE condition (e.g., the packet does not contain a DNS query request), the action can be empty, which can signal the packet filtering logic to continue searching the rest of the policy including additional Bloom filters and rule subsets. If the current packet is not a DNS query request, it may be appropriate to skip the membership test associated with B / F-1 (and the subsequent application of the B / F-1 packet filtering rule). When the membership test returns FALSE (e.g., the domain name is not registered in the DNS), the action of the associated rule can also be a conditional action. When the membership test returns FALSE and if the current packet does contain a DNS query request (to resolve the domain name to an IP address), the rule action can be to discard the packet and (optionally) apply a packet conversion function that generates a packet containing a corresponding DNS query response with RCODE=3 (NXDOMAIN or other indication of a non-existent Internet or Intranet domain name) and send the packet to the source host to terminate the request. When the membership test returns FALSE and the result is an ELSE response, indicating that the current packet does not contain a DNS query request, the action can be a no-op, which can signal the packet filtering logic to continue searching the rest of the policy, including additional rule subsets and their associated filters.

[0141] The second and third subsets (SS-2 and SS-3, respectively) are also represented by space-efficient Bloom filters B / F-2 and B / F-3, respectively (and associated information). For purposes of example, assume that both B / F-2 and B / F-3 are derived from a single CTI provider's domain name list (and thus any corresponding rules can have the same threat metadata), but each corresponding packet filtering rule has only a single action: either a BLOCK action (e.g., an action to drop a packet) or a MONITOR action (e.g., an action to allow / forward a packet, but also to log and capture the packet for possible subsequent threat analysis). SS-2 may contain all rules with a BLOCK action, and accordingly the associated B / F-2 may contain or represent all metrics for rules with a BLOCK action. SS-3 may contain all rules with a MONITOR action, and accordingly B / F-3 may contain all metrics for rules with a MONITOR action. Thus, any determination of a TRUE result for the B / F-2 test may result in the application of a BLOCK action, and the CTI provider and associated threat metadata and packet and environment data may be stored as a log of threat events (for subsequent use, for example, in a network analysis application). Similarly, any determination of a TRUE result for the B / F-3 test may result in the application of a MONITOR action, and the CTI provider and associated threat metadata and packet and environment data may be stored as a log of threat events. False rules for both B / F-2 and B / F-3 may result in the application of a null operation or action, which may signal the packet filtering logic to continue searching the policy to find a matching rule. The false alarm rate P2 associated with B / F-2 may be limited to be (much) less than / lower than the false alarm rate P3 associated with B / F-3. Since the rule subset SS-2 consists only of BLOCK rules, a false alarm / TRUE membership test result for B / F-2 means that the current packet may be blocked / discarded when it should not be blocked / discarded, which is highly undesirable assuming that the current packet is part of a legitimate business communication of the protected network. Since the rule subset SS-3 consists only of MONITOR rules, false positive / TRUE membership test results will not lead to the discarding / blocking of legitimate business traffic. In practice, the false positive rate P2 can be, for example, 1 / 10 12 That is, for example, it is expected that only one in a trillion legitimate HTTP GET requests may be mistakenly discarded, an event that may never occur in practice; while a typical value for the false positive rate P3 might be, for example, 1 / 10 3 (one thousandth) or 1 / 10 6(parts per million). As discussed above, the system cost of a lower false alarm rate P for a Bloom filter is an increase in memory space requirements. However, the memory space requirements can vary as the logarithm of the false alarm rate P. Thus, for a Bloom filter with 1 / 10 12 A Bloom filter with a false positive rate of N elements only needs to be twice as fast as one with a false positive rate of 1 / 10 6 The memory space of a Bloom filter with the same set of N elements for each packet filtering device may be reduced, even if the false alarm rate differs by a factor of one million. These decisions may be made based on capacity thresholds associated with each memory provided in the packet filtering device, and as an example of selecting a false alarm rate for a space-efficient Bloom filter. The fourth subset SS-4 may include packet filtering rules with a metric of the "IP address" type. The metric may be included in a time-efficient Bloom filter B / F-4 calculated by the policy manager SPM 120. The fourth subset SS-4 may not be represented using a space-efficient Bloom filter.

[0142] The false alarm rate can be selected as follows. False alarms are undesirable, so a relatively low false alarm rate is desirable. One of the benefits of the disclosed system and method is that the efficiency of memory / storage space usage is improved; however, because the size of the Bloom filter (e.g., memory / space requirements) increases with the decrease in the false alarm rate, it is undesirable for the false alarm rate to be unnecessarily low (on the contrary). Therefore, to select the false alarm rate, the system should consider the adverse effects of false alarms. For a time-efficient Bloom filter, the adverse effect of a false alarm is that the associated subset of the packet filtering rules is unnecessarily searched, so there is a (small) time / delay cost; however, since no matching rules for the false alarm will be found, no action will be (erroneously) applied to the current packet and the adverse effect is only a response delay. However, for a space-efficient Bloom filter, the adverse effect of a false alarm is that the action of the matching rule will be incorrectly applied to the current packet. If the action is to block / discard (legal) packets, or otherwise modify / convert packets, the adverse effect of a false alarm can be considered to be significantly adverse. If the action is to allow the (legal) packet but log and capture a copy of it for subsequent network analysis, then the adverse effects are less severe. Taking all of these factors into account, the method for selecting the false positive rate will be chosen in such a way that (1) the false positive rate of a time-efficient Bloom filter can be higher than the false positive rate of a space-efficient Bloom filter; (2) the false positive rate of a space-efficient or subset Bloom filter can be determined in part by the associated action: (a) the false positive rate should be relatively low when the action causes a packet currently in transit to be dropped / blocked, and should also be so low that in practice a false positive is likely to never occur (e.g., 1 / 10 12, or one in a trillion); (b) when the action does not cause the currently transmitted packet to be dropped / blocked, the false alarm rate can be relatively high, but also relatively lower than the false alarm rate of the time-efficient Bloom filter. Finally, the false alarm rate of a general Bloom filter can be undefined, so it may not be necessary to select a rate for a general Bloom filter.

[0143] Figure 8 The examples in usually correspond to Figure 3 In step 8-0, the {domain name, IP address} pair may be persistently stored in the globally distributed Internet DNS, which is represented by the memory module TIER-N+1MEM 239. Steps 8-1, 8-2, 8-3 and Figure 4 Steps 4-1, 4-2, and 4-3 are basically similar, and a detailed description of these steps will not be repeated here. In step 8-4, based on the policy file header information, the CPU 220 may retrieve the first subset SS-1 from the disk 233. As discussed above, SS-1 is represented by a space-efficient Bloom filter B / F-1 and associated information, which may include rule actions and metadata for processing logic. Based on the last update time of B / F-1, B / F-1 may include all domain names (currently) registered in the Internet DNS. In step 8-5, when it is determined that there is sufficient available capacity in the main memory 231, the CPU 220 may store B / F-1 and associated information in the TIER-1MEM 231.

[0144] In step 8-6, based on the policy file header information, the CPU 220 may retrieve two rule subsets SS-2 and SS-3 from the disk 233. As discussed above, SS-2 and SS-3 may be represented by space-efficient Bloom filters B / F-2 and B / F-3 and associated information. In step 8-7, when it is determined that there is sufficient available capacity in the main memory 231, the CPU 220 may store B / F-2 and B / F-3 and associated information in the TIER-1MEM 231, which may include rules for SS-2 and SS-3. In step 8-8, based on the policy file header information, the CPU 220 may retrieve the subset SS-4 from the disk 233 and may calculate the time-efficient Bloom filter B / F-4, which contains all metrics (of the "IP address" type) associated with the packet filtering rules in SS-4. In step 8-9, upon determining that there is sufficient available capacity in main memory 231, CPU 220 may store B / F-4 and SS-4 in TIER-1MEM 231. TIG 200 may then be configured for efficient packet filtering for the current policy and may begin processing packets.

[0145] Fig. 9 Shows Figure 8 A representative use case of a TIG 200 configured in FIG. 1 and the associated efficient packet filtering process. This illustration generally corresponds to Figure 3 Steps 3-5 to 3-9.

[0146] For example, in step 9-1, a human user operating an email application hosted by a desktop computer 131 attached to network 112 may click on a URL link http: / / www.cyber-threat-network.com / download / malware.exe in a phishing email. Computer 131 may initiate a web / HTTP session with a website named www.cyber-threat-network.com, which is hosted by server 136 attached to network 118. However, before computer 131 can connect to the website, desktop computer 131 may need to resolve the domain name www.cyber-threat-network.com to an IP address. Therefore, computer 131 may generate and forward a packet containing a DNS query request to an Internet DNS service (to resolve www.cyber-threat-network.com to its IP address). The destination IP address field value (e.g., 18.27.36.45) of the packet, which may be included in the L3 / IP header, may be the IP address of a DNS server that computer 131 has been configured to contact for the DNS service. The destination port of the packet in the L4 / TCP header may be 53, which is the port of the DNS service. In step 9-2, the network interface 212 of TIG 200 can receive and ingest the packet in transit and can transmit the packet to CPU 220 via BUS 210 for filtering. CPU 220 can extract (a) the domain name www.cyber-threat-network.com, and (b) the destination IP address 18.27.36.45 from the DNS query request. The packet filtering logic can be configured to filter the domain name indicator before filtering the IP address indicator. Therefore, the domain name www.cyber-threat-network.com can be filtered by the policy first.

[0147] In step 9-3, CPU 220 may check the metric type associated with (and contained in) a (space-efficient) Bloom filter B / F-1 that contains all domain names currently registered in the Internet DNS to determine whether the Bloom filter metric type matches the type ("domain name") of www.cyber-threat-network.com. Since B / F-1 contains an metric of type "domain name," CPU 220 checks whether the domain name www.cyber-threat-network.com is a member of B / F-1. When www.cyber-threat-network.com is registered in the Internet DNS, the membership test may return a TRUE result. In step 9-4, CPU 220 may convert the packet containing the DNS query request into a packet containing a corresponding DNS query response by sending the packet containing the request to the Internet DNS represented by TIER-N+1MEM239 (via network interface 214). CPU 220 may stop searching for policies. In step 9 - 5 , the Internet DNS may generate a packet containing a corresponding DNS query response, which may include the resolved IP address (eg, 87.65.43.21) of the domain name www.cyber-threat-network.com, and may send the packet to host 131 .

[0148] In step 9-6, after www.cyber-threat-network.com is resolved in the DNS, and a TCP connection is established between computer 131 and host 136 on port 80 / HTTP (effectively filtering packets by TIG 200 policy without a match), computer 131 can forward a packet containing an HTTP GET request for http: / / www.cyber-threat-network.com / download / malware.exe to 136. In step 9-7, network interface 212 of TIG 200 can receive and ingest the packet in transit and can pass the received packet to CPU 220 via BUS 210 for filtering. CPU 220 can extract from the packet (a) URL http: / / www.cyber-threat-network.com / download / malware.exe, (b) domain name www.cyber-threat-network.com, and (c) destination IP address 87.65.43.21. The CPU 220 logic may be configured to search the policy for indicators in order of decreasing fidelity, thus, the CPU 220 may first search the policy for a rule matching the URL http: / / www.cyber-threat-network.com / download / malware.exe, then search the domain name www.cyber-threat-network.com, then search the IP address 87.65.43.21.

[0149] In step 9-8, the CPU 220 may determine the indicator types associated with (and contained in) the Bloom filter B / F-1 for the first subset SS-1 of the packet filtering rules to determine whether they match the indicator type (i.e., "URL") of the URL http: / / www.cyber-threat-network.com / download / malware.exe. When B / F-1 contains an indicator of the type "domain name" (rather than "URL"), the CPU 220 may skip the membership test for B / F-1 (and thereby skip the search for SS-1). Similarly, the indicator type may be checked for B / F-2, B / F-3, and B / F-4 in the absence of a type match. Therefore, when there is no rule with an indicator matching http: / / www.cyber-threat-network.com / download / malware.exe, the policy may not be searched. In step 9-9, the CPU may search the policy for a rule whose indicator matches the domain name www.cyber-threat-network.com. The membership test on B / F-1 may return a TRUE value (e.g., www.cyber-threat-network.com is registered in DNS). When the current packet is not a DNS query request, no rule action will be applied based on the match indication of the membership test of B / F-1, and the policy search can continue. When (space efficiency) the membership test in B / F-2 returns TRUE (e.g., there is a rule with an indicator matching http: / / www.cyber-threat-network.com / download / malware.exe), and the associated rule action is BLOCK, the packet can be discarded, which can prevent the malware download from occurring.

[0150] Fig.10 Shows Figure 8 2 and the associated efficient packet filtering process. In this configuration, TIG 200 can detect and process false DNS query requests containing domain names that are not registered in the DNS. This example generally corresponds to Figure 3Steps 3-5 through 3-9 of . Host 135 attached to (unsecured) network 116 may have been hijacked to act as a collection server for sensitive data that may be leaked / stolen using a DNS tunneling approach that uses fake DNS query requests to leak information. However, the IP address or any domain names associated with host 135 may not be known to the CTI provider, so TIG 200 may not have been configured with CTI-derived packet filtering rules for detecting communications between host 135 and hosts attached to network 112. Host 130 attached to (secured) network 112 may have been compromised by malware that collects sensitive corporate information, such as credit card numbers managed by host 130. The malware may attempt to leak this information to host 135 using DNS tunneling.

[0151] In step 10-1, malware executing on host 130 may send a packet addressed to host 135 (having an IP address, e.g., 45.36.27.18) and destination port 53 (DNS), which may contain a DNS query request with a (fake) domain name "visa5234334164939009.www.legit-domain.com", which actually contains a credit card number. In step 10-2, network interface 212 of TIG 200 may receive and ingest the packet in transit and may pass the packet to CPU 220 via BUS 210 for filtering. CPU 220 may extract from the DNS query request (a) the domain name visa5234334164939009.www.legit-domain.com, and (b) the destination IP address 45.36.27.18. The packet filtering logic may have been configured to filter the domain name indicator before filtering the IP address indicator. Therefore, visa5234334164939009.www.legit-domain.com can be filtered by policy first.

[0152] In step 10-3, CPU 220 checks the types of indicators associated with (and contained in) a (space-efficient) DNS Bloom filter B / F-1 that contains all domain names currently registered in the Internet DNS to determine whether they match the type ("domain name") of visa5234334164939009.www.legit-domain.com. Since B / F-1 contains indicators of type "domain name," CPU 220 may check whether the domain name visa5234334164939009.www.legit-domain.com is a member of B / F-1. Since the fictitious domain name visa5234334164939009.www.legit-domain.com has not been registered in the Internet DNS, the membership test returns a FALSE value. In step 10-4, CPU 220 may apply the operator or action associated with the FALSE result of test B / F-1 for the DNS query request. The false result actions may be (a) discarding the current packet containing the DNS query request for visa5234334164939009.www.legit-domain.com, thereby preventing leakage / DNS tunneling attacks; (b) applying a packet conversion function that generates a packet containing a corresponding DNS query response with RCODE=3 (NXDOMAIN); and (c) the DNS query response may be sent to host 131. Based on a FALSE result for the B / F-1 test, CPU 220 may stop searching for policies.

[0153] Fig.11 A representative use case of an efficient TIG is shown, which can protect the Internet DNS from DDoS attack types that use fake DNS requests as an attack vector. The context of the use case is given by Figure 1 TIGs 200 and 250 protecting networks 112 and 114, respectively, may be configured as described above. Figure 8 are configured with the same strategy in the same way. To review, Figure 8The policy configured in includes a subset SS-1 represented by a space-efficient DNS probabilistic data structure (such as a Bloom filter B / F-1) and information about all domain names currently registered in the Internet DNS. One type of attack on the Internet DNS is a DDoS attack, in which many malware-infected endpoints distributed around the world may send many fake DNS query requests to resolve domain names that are not registered in the DNS. Malware may use a domain generation algorithm (DGA) to randomly generate domain names, such as q0e8vqq0y8n.www.legit-site.net, which is unlikely to match a registered domain name. The DNS may search its distributed database to try to resolve these (unresolvable) requests. If the number of these fake requests is large enough, the DNS will spend a (large) majority of its resources trying to satisfy the fake requests while underserving legitimate requests, thereby refusing to serve legitimate requests. Since many popular applications (such as the web) rely on DNS to operate, a successful DDoS attack on the DNS can lead to a successful attack on applications that rely on the DNS.

[0154] DNS infrastructure provider networks, such as network 114, are designed and configured to receive many DNS query requests to provide services through, for example, DNS name servers 132 and 133; therefore, during a DDoS attack on DNS using false DNS requests as an attack vector, such networks may receive many such false DNS requests. TIG 250 may be configured to receive many false DNS requests. Figure 8 114 and certain portions of the Internet DNS from these false requests while continuing to serve legitimate DNS requests, thereby mitigating DDoS attacks. Figure 8 TIG 200 configured as in and protecting network 112 may prevent false DNS requests that may be issued, for example, by malware-infected host 131 attached to network 112 from being transmitted to the Internet (in Figure 1 The network 110 is used to identify and attack the DNS.

[0155] Host 131, which may be attached to (TIG-protected) network 112, and hosts 134 and 135, which may be attached to (unprotected) network 116, may be infected with malware that may issue false DNS requests when commanded, for example, by host 136, which may be attached to (operated by a malicious actor) network 118. Further, Figure 1The DNS infrastructure (not shown) of the networking environment can be configured such that DNS query requests originating from hosts attached to networks 112 and 116 are serviced by DNS name servers 132 and 133 attached to network 114. For example, servers 132 and 133 can be configured as authoritative name servers for domains example-132.net and example-133.net, in which case DNS query requests for domain names (including example-132.net and example-133.net) can be routed to servers 132 and 133.

[0156] In step 11-1, host 134 may generate and send a false DNS query request to DNS server 132 to resolve the domain name ppk6h0ec.www.example-132.net. In step 11-2, host 135 may generate and send a false DNS query request to DNS server 132 to resolve the domain name 61kwrari.www.example-132.net, which may be sent simultaneously with the request from host 134. In step 11-3, host 130 may generate and send a legitimate DNS query request to DNS server 132 to resolve the domain name www.example-132.net, which may be sent simultaneously with the requests from other hosts. In step 11-4, host 131 may generate and send a false DNS query request to DNS server 133 for resolving the domain name hyqlyn2c.www.example-133.net, which may be sent simultaneously with the requests from other hosts.

[0157] In step 11-5, TIG 200 (a) may receive and ingest a legitimate DNS query request from 130, (b) may test B / F-1 to determine that the domain name www.example-132.net is registered in the DNS, and (c) forward the DNS query request to DNS server 132. In step 11-6, TIG 200 (a) may receive and ingest a false DNS query request from 131, (b) may test the DNS probabilistic data structure B / F-1 to determine that the false domain name hyqlyn2c.www.example-133.net is not registered in the DNS, (c) may discard the DNS query request (packet), (d) may decide to generate / spoof a corresponding DNS query response with RCODE=3 (NXDOMAIN), and (e) may send the DNS query response back to the source host 131. In step 11-7, TIG 250 (a) may receive and ingest the false DNS query request from 134, (b) may test B / F-1 to determine that the false domain name ppk6h0ec.www.example-132.net is not registered in the DNS, (c) may discard the DNS query request (packet), (d) may decide to spoof / generate a corresponding DNS query response with RCODE=3 (NXDOMAIN), and (e) may send the DNS query response back to the source host 134. In step 11-8, TIG 250 (a) may receive and ingest the false DNS query request from 135, (b) may test B / F-1 to determine that the false domain name 61kwrari.www.example-132.net is not registered in the DNS, (c) may discard the DNS query request (packet), (d) may decide to generate a corresponding DNS query response with RCODE=3 (NXDOMAIN), and (e) may send the DNS query response back to the source host 135.

[0158] In step 11-9, TIG 250 (a) may receive and ingest a legitimate DNS query request from 130, (b) may test B / F-1 to determine that the domain name www.example-132.com is registered in the DNS, and (c) may forward the DNS query request to DNS server 132. In step 11-10, DNS server 132 (a) may receive a DNS query request from 130 to resolve www.example-132.com, (b) may service the request by searching the DNS for the IP address 12.34.56.78 assigned to www.example-132.com, (c) may generate a corresponding DNS query response, and (d) may send the DNS query response back to source host 130. Based on the test of domain name B / F-1, TIGs 200 and 250 may prevent false DNS query requests from consuming / wasting any DNS resources attempting to resolve unregistered domain names. Denial of service attacks may be remedied / prevented. As a result, DNS may be fully utilized to service legitimate DNS query requests.

[0159] Fig.12 An embodiment and associated sequence of events are shown in which a TIG can be configured for efficient packet filtering when the policy is divided into three subsets, where the division is determined by the type of metric and the type of packet transformation function of the rule. A time-efficient Bloom filter can be associated with the entire set or with the union of all three subsets. Each subset can be associated with (and represented by) a space-efficient Bloom filter. All Bloom filters can have a false alarm rate value that has been selected based on the type of Bloom filter (e.g., time efficiency or space efficiency) and based on the type of packet transformation function associated with the rules in the subset (e.g., BLOCK or MONITOR). The subsets and associated Bloom filters can be stored in the same memory module. This illustration generally corresponds to Figure 3 Steps 3-1 to 3-4.

[0160] In step 12-1, SPM 120 may send a file to the management interface MGMT I / F 240 of the TIG (via network 110, which may be identified, for example, by the Internet), the file containing the next policy to be executed on the packets in transit by TIG 200. The policy file may include a header with a policy descriptor, which may include information such as a partitioning scheme of the packet filtering rule set into subsets, one or more types of efficiency to be applied to each subset, the size of the policy, the size of each subset, and the type of indicator in each subset.

[0161] In step 12-2, MGMT I / F 240 may transmit the policy file to TIER-3MEM 233 (via BUS 210), which may be identified by a local mechanical / rotating disk (non-volatile, relatively large capacity, relatively cheap, relatively slow read / write access). In step 12-3, CPU 220 may retrieve a policy file header from disk 233. The policy file header may indicate how the policy is divided into rule subsets, which efficiencies may be applied, and how the subsets may be distributed across multiple memory modules (based on size information if necessary). While CPU 220 processes the policy header file information, the CPU may temporarily store portions of the information in other portions of the memory, such as TIER-0MEM 230, which may be identified by an on-chip cache memory, and TIER-1MEM 231, which may be identified by a main memory on the PCB. These temporary storage events are not shown.

[0162] The policy descriptor data may specify a plurality of information. The policy descriptor data may specify that SS-1 is a subset of group filter rules, where each rule may include (a) a threat indicator of the certificate identifier type, which is a {CA identifier, serial number} pair; (b) a BLOCK grouping operator or transformation function; and (c) threat metadata, such as "<threat type=revoked certificate>". SS-1 may be encoded as a threat indicator using a false positive rate P=10 -12 The policy descriptor data may specify that SS-2 is a subset of grouped filtering rules, where each rule may include (a) a threat indicator of a domain name type; (b) a BLOCK grouping operator or transformation function; and (c) threat metadata, such as "<threat type=certificate phishing site>". SS-2 may be encoded as using a false positive rate P=10 -12 The policy descriptor data may specify that SS-3 is a subset of grouped filter rules, where each rule may include (a) a threat indicator of a domain name type; (b) a MONITOR grouping operator or transformation function; and (c) threat metadata, such as "<threat type=malware CDN>". SS-3 may be encoded as using a false positive rate P=10 -6 The policy descriptor data may specify a time-efficient Bloom filter B / F-ALL, which is associated with and contains all threat indicators in SS-1, SS-2, and SS-3, and has a false positive rate P=10 -3 .

[0163] In step 12-4, based on the policy file header information, CPU 220 may retrieve the first subset SS-1 from disk 233. CPU 220 may extract all indicators (certificate identifier types) in SS-1 and may simultaneously insert the indicators into a space-efficient Bloom filter B / F-1 (false positive rate P=10 -12 ) and time-efficient Bloom filter B / F-ALL (false positive rate P = 10 -3 ) both. CPU 220 may associate Bloom filter B / F-1 with the group conversion function BLOCK, the threat metadata "<threat type=revoked certificate>", and the indicator type certificate identifier. CPU 220 may associate the indicator type certificate identifier with B / F-ALL. In step 12-5, CPU 220 may determine that main memory 231 has sufficient remaining available capacity to store Bloom filter B / F-1 (which represents SS-1). Based on the capacity determination, CPU 220 may store space-efficient Bloom filter B / F-1 in main memory 231.

[0164] In step 12-6, based on the policy file header information, the CPU 220 may retrieve the second subset SS-2 from the disk 233. The CPU 220 may extract all indicators (domain name types) in SS-2, and may insert the indicators into both the space-efficient Bloom filter B / F-2 and the time-efficient Bloom filter B / F-ALL. The CPU 220 may associate the Bloom filter B / F-2 with the packet conversion function BLOCK, the threat metadata "<threat type=certificate phishing site>", and the indicator type domain name. The CPU 220 may associate the indicator type domain name with B / F-ALL. After this operation, B / F-ALL is associated with two types of indicators (certificate identifier and domain name). In step 12-7, the CPU 220 may determine that the main memory 231 has sufficient remaining available capacity to store the Bloom filter B / F-2 (which represents SS-2). Based on the capacity determination, the CPU 220 may store the space-efficient Bloom filter B / F-2 in the main memory 231.

[0165] In step 12-8, based on the policy file header information, the CPU 220 may retrieve the third subset SS-3 from the disk 233. The CPU 220 may extract all indicators (of the domain name type) in SS-3, and may insert the indicators into both the space-efficient Bloom filter B / F-3 and the time-efficient Bloom filter B / F-ALL. The CPU 220 may associate the Bloom filter B / F-3 with the packet conversion function MONITOR, the threat metadata "<threat type=malware CDN>", and the indicator type domain name. The CPU 220 may determine that the indicator type domain name has been associated with B / F-ALL, as discussed in step 12-6 above. In step 12-9, the CPU 220 may determine that the main memory 231 has sufficient remaining available capacity to store the Bloom filter B / F-3 (which represents SS-3). Based on the capacity determination, the CPU 220 may store the space-efficient Bloom filter B / F-3 in the main memory 231. In step 12-10, the CPU 220 may determine that the main memory 231 has sufficient remaining available capacity to store the Bloom filter B / F-ALL (which contains all threat indicators in SS-1, SS-2, and SS-3). Therefore, the CPU 220 may store the time-efficient Bloom filter B / F-ALL in the main memory 231. Based on determining that all rule subsets of the policy have been processed, it may be determined that the TIG 200 is configured to perform effective packet filtering for the current policy and may begin receiving packets for filtering.

[0166] Fig.13 Shows Fig.12 A representative use case of a TIG 200 configured in FIG. 1 and the associated efficient packet filtering process. This illustration generally corresponds to Figure 3 Steps 3-5 to 3-9. Fig.13 The version of the TLS protocol used in may generally correspond to TLS 1.2.

[0167] For example, in step 13-1, a human user operating a web browser hosted by a desktop computer 130 attached to network 112 may initiate a (secure) web / HTTPS session with a website at the domain name www.enterprise-xyz.com, which is hosted by a server 134 attached to network 116. Host 130 may resolve www.enterprise-xyz.com in the DNS to an IP address (e.g., 12.34.56.78), and a TCP connection may be established between 130 and 134 on port 443 / HTTP (not in Fig.13). CPU 220 may filter these packets by extracting the IP addresses from the L3 headers of these packets and searching the policy; however, when B / F-ALL does not contain any IP address indicators, CPU 220 may not search for these IP addresses in the policy. Computer 130 may initiate a TLS tunnel setup handshake by forwarding a packet containing a TLS ClientHello message to website 134.

[0168] In step 13-2, the network interface 212 of the TIG 200 can receive and ingest the packet in transit and can transmit it to the CPU 220 via the BUS 210 for filtering. The CPU 220 can extract the domain name www.enterprise-xyz.com from the SNI field in the ClientHello message. In step 13-3, the CPU 220 can determine whether the filter B / F-ALL contains an indicator of the domain name type, which can return a TRUE value. The CPU 220 can test whether www.enterprise-xyz.com is a member of B / F-ALL, which may return a FALSE value. Based on the false value indicating that the domain name is not in the policy, the CPU 220 can skip the search for the policy. In step 13-4, the CPU 220 can transmit the packet to the network interface 214 via the BUS 210. In step 13-5, network interface 214 may forward the packet into network 110 toward the target destination website www.enterprise-xyz.com, which may be hosted by server 134 having IP address 12.34.56.78. Server 134 may receive and ingest the packet at port 443, may process the ClientHello message, and may respond to the ClientHello by sending a packet containing a TLSServerHello message back to host 130 (not in Fig.13 ).

[0169] In step 13-6, host 134 may send a packet containing a TLS certificate message to host 130. The packet may be received and ingested by TIG (via network interface 214) and processed by CPU 220. CPU 220 may extract the "issuer name" and "serial number" values ​​from the certificate and associate these values ​​with the certificate ID. In step 13-7, CPU 220 may determine whether B / F-ALL contains an indicator of a certificate identifier type associated with the filtered packet, which may return a TRUE result. CPU 220 may determine whether B / F-ALL contains a certificate ID associated with the filtered packet, which may return a TRUE result. Based on the TRUE result, CPU 220 may test the rules associated with B / F-ALL to determine any rules and operations associated with the tested packet data. In step 13-8, CPU 220 may determine whether B / F-1 contains an indicator of a certificate identifier type, which may return a TRUE result. Based on the TRUE result, CPU 220 may determine whether B / F-1 contains a certificate ID, which may return a TRUE result. Based on a TRUE result associated with the certificate ID, it can be determined that the certificate for www.enterprise-xyz.com has been revoked by the certificate authority that issued the certificate, and the system can determine that the site administrator has not yet obtained and installed a new (non-revoked) certificate.

[0170] In step 13-9, CPU 220 may apply a packet conversion function BLOCK associated with B / F-1, which may cause the packet to be dropped and may cause a TCP RST packet to be sent (via network interface 212) to host 130 to terminate the TCP connection, thereby stopping the TLS tunnel setup handshake process.

[0171] According to the present disclosure, Figure 1 akin Fig.14 A representative network environment 100 is depicted. Network environment 100 may include a DNS attack prevention system for protecting a Domain Name System (DNS) of an interconnected network, such as the public Internet and private networks that may be interconnected via the public Internet, from attacks that may use false DNS requests as an attack vector. Figure 1 Similarly, network 110 may be the public Internet, which may connect networks 112, 114, 116, 118, and 120 (which may be connected to Figure 1 those described above) are interconnected.

[0172] The network 112 may be a private enterprise network with multiple hosts. The multiple hosts may include at least one "Internet of Things" (IoT) device IOT 140, such as a security camera or device that can be remotely managed and operated via the Internet, and include a personally identifiable information (PII) host PII 141. The host IOT 140 and other IoT devices (not shown) attached to the network 112 may have been infected with malware that can generate attacks on the DNS by issuing many false DNS requests with domain names intended to attack specific authoritative name servers (such as the host NS 144 attached to the network 116, as described below). The host PII 141 represents a personal computer or server that manages sensitive information (such as personally identifiable information (PII), credit card information, etc.). The host PII 141 may be infected by malware that collects credit card information, and the infected host may use the DNS tunneling method to leak information to the collection server CSVR 148 (described below) attached to the network 170. Both attacks may issue requests with domain names that are not registered in the DNS.

[0173] Network 114 and attached host IOT 142 and PII 143 may be configured similarly and subject to similar compromise as network 112 and host IOT 140 and PII 141. However, network 114 may be configured with a DNS gatekeeper DNS-G / K 150 located at or near the boundary between network 114 and Internet 110. Network 114 may be configured such that DNS-G / K 150 filters all DNS requests that cross the network 114 boundary and may control which DNS requests may access the DNS.

[0174] Network 116 may be operated by a DNS infrastructure provider organization DIPO-X. Authoritative name servers NS 144 and NS 145 may be DNS authoritative name servers for domains example-144.com and example-145.com, and DIPO-X may operate on behalf of the relevant registration organizations. Name servers NS 144 and NS 145 may both be configured to provide their zone files to authorized requesters, which may include a list of all registered domain names managed by these servers. Network 116 may have a DNS gatekeeper DNS-G / K 180 located at or near the boundary between network 116 and Internet 110. Network 116 may be configured so that DNS gatekeeper DNS-G / K 180 filters all DNS requests that cross the boundary, and may control which DNS requests may access the DNS name server for example-144.com (authoritative name server NS 144) and the DNS name server for example-145.com (authoritative name server NS 145).

[0175] Network 118 may be operated by a DNS infrastructure provider organization (e.g., DIPO-Y). Host authoritative name servers NS 146 and NS 147 may be DNS authoritative name servers for domains example-146.com and example-147.com, which DIPO-Y may operate on behalf of the associated registration companies. Name servers NS 146 and NS 147 may both be configured to provide their zone files, which may include a list of all registered domain names managed by these servers, to authorized requesters. Network 118 does not need to have a DNS gatekeeper to control which DNS requests can access the DNS name server for example-146.com (authoritative name server NS 146) and the DNS name server for example-147.com (authoritative name server NS 147).

[0176] The network 170 may be operated by at least one malicious subject. The network 170 may include a collection server host CSVR 148 that may collect data leaked through false DNS requests, and may include a botnet command and control host system CNC 149 that may execute a botnet command and control process that instructs infected devices to launch a DDoS attack on the DNS. The malicious data collection process executed on the collection server CSVR 148 may receive false DNS requests on port 53, which is a well-known DNS port. For the following example, the public IP address of the collection server CSVR 148 may be 21.43.65.87. A network firewall (not shown) associated with the network 170 may be configured to allow inbound packets with a destination IP address of 21.43.65.87 of the collection server CSVR 148 and a destination port 53. The network 170 may not have a DNS gatekeeper to control access to the DNS.

[0177] A bloom filter DNS filter manager (DFM) 160, which may act as a system management server, may be executed as a host attached to a private network (not shown) that interfaces with Internet 110. DFM 160 may receive DNS authoritative name servers (such as DNS authoritative name servers NS 144-147) and / or from associated domain list aggregators (not shown). Figure 1 DFM 160 may encode all registered domain names and may store all registered domain names in at least one data structure DNS-REG, and each Bloom filter DNS-REG may store all registered domain names in a single data structure DNS-REG. PX may have an associated false alarm rate P XDFM 160 may distribute the data structure or Bloom filter to subscribing DNS gatekeepers (eg, DNS-G / K 150 and DNS-G / K 180).

[0178] DFM 160 may also include functionality for managing information provided or uploaded by subscribing DNS gatekeepers (e.g., DNS-G / K 150 and 180). This information may include unregistered domain names detected by subscribing DNS gatekeepers DNS-G / K 150 and 180 and associated metadata (e.g., attack type estimates, timestamps, rate estimates, source hosts, DNS-G / K IDs, etc.), which may be used as threat intelligence to determine or detect other attacks. The information may also include statistics about DNS requests and responses observed by DNS-G / K 150 and 180 and element management information of DNS-G / K 150 and 180. DFM 160 may also include functionality for providing information collected from DNS-G / K 150 and 180 to subscribing applications (not in Figure 1 ), subscribing to applications such as threat intelligence providers, network analysis applications, network element managers, DNS monitoring applications, DNS protection applications, Internet protection applications, etc.

[0179] Fig.14 (and discussed below Fig.15 ) may, but need not be, all or part of a network device such as TIG 200 and / or TIG 250. In general, DNS-G / K 150 and DNS-G / K 180 may be executed on any computing device, including but not limited to those computing devices discussed above or below.

[0180] Fig.15 A representative system component diagram of DNS-G / K 150 and / or DNS-G / K 180 that supports aspects of the present disclosure is shown. Fig.15 The DNS-G / K 150 shown can be used with Figure 1The DNS-G / K 180 and / or DNS-G / K 150 shown are the same or similar. The DNS-G / K 150 components may include a processor CPU 1520 that may execute logic for configuring and operating the DNS-G / K 150, network interfaces 1512 and 1514 for interfacing with the networks 114 and 110, respectively, a management interface MGMT I / F 1540 connected to the network 110, a main memory module MAIN-MEMORY 1530 that may contain a data structure (e.g., a Bloom filter) DNS-REG 1531, and a memory storage MEMORY-STORE 1532 for persistent storage. These components may be connected to a bus BUS 1510 that may be used to transfer information between the components of the DNS-G / K 150. The DFM 160 may provide the Bloom filter DNS-REG 1531 to the DNS-G / K 150 via the management interface 1540 and the network 110. The system components of DNS-G / K 150 can be any combination of processes or applications executed (e.g., co-resident) on the same host, processes executed on different hosts, processes executed on virtual infrastructure such as a hypervisor, or other arrangements of components and software. Similarly, bus BUS 1510 can be not only an integrated / embedded data bus of a printed circuit board (PCB), but also any combination of, for example, an L2 / L3 switching network, an L3 routing network, an L2 / L3 network link connecting logical components, etc. The data bus can be any combination of wired, wireless, physical, logical, virtual, software-defined data buses, etc. Network interfaces 1512 and 1514 can be L3 transparent. Network interfaces 1512 and 1514 can have no assigned IP addresses and can not participate in the L3 routing process. Similarly, network interfaces 1512 and 1514 can be L2 transparent. Network interfaces 1512 and 1514 can have no assigned MAC addresses and can therefore not participate in the ARP process. The management interface MGMT I / F 1540 may have an assigned IP address for communicating with a management device such as the DFM 160 .

[0181] Fig.1616-1 is a flow chart of a representative operational concept of DNS-G / K 150. In step 16-1, DNS-G / K 150 may select a data structure (e.g., Bloom filter) DNS-REG 1531 provided by DFM 160, and may download and store the Bloom filter DNS-REG 1531 in its main memory MAIN-MEMORY 1530. Bloom filter DNS-REG 1531 may contain all domain names currently registered in the DNS, which domain names are available to or determined by DFM 160. DFM 160 may provide multiple versions of DNS-REG 1531 for use by DNS gatekeepers, which may be characterized by false positive rates and corresponding memory requirements for storing all registered domain name sets. The versions of DNS-REG 1531 may also be characterized by creation time, and the selection and downloading of DNS-REG 1531 may be part of the update process of DNS-G / K 150. The bloom filter (e.g., DNS-REG 1531) may be selected by DNS-G / K 150 based on an associated false positive rate P and / or based on a corresponding memory requirement for storing a set of domain names in DNS-REG 1531. In step 16-1, DNS-G / K 150 may also upload the information collected in steps 16-6, 16-7, and 16-8 to DFM 160. The upload may be part of an update process. The upload may include data such as false, fake, or unregistered domain names detected by DNS-G / K 150 and associated metadata (e.g., timestamps, source host identifiers, estimated attack types, rate estimates, etc.), which may be used, for example, as threat intelligence and in threat intelligence reports. The uploaded data may include statistics of DNS requests and responses observed by DNS-G / K 150, and domain names determined by DNS-G / K 150 that may be registered in the DNS but are not included in the local instance of the bloom filter DNS-REG 1531 (e.g., due to synchronization lags). Such domains may have been registered by DNS-G / K150 in Fig.17 Other information may also be uploaded, such as DNS-G / K 150 element management information.

[0182] In step 16-2, DNS-G / K 150 may receive a packet in transit from NETWORK 114 that may be intended to be forwarded to NETWORK 110. DNS-G / K 150 may also process packets transmitted in the opposite direction from NETWORK 110 to NETWORK 114, but for purposes of illustration, only one direction is described. In step 16-3, DNS-G / K 150 may determine whether the packet contains a DNS query request. If the packet does not contain a DNS query request, then in step 16-4F, the packet may be forwarded to NETWORK 110, and DNS-G / K 150 loops back to step 16-2 to process other packets. If the packet does contain a DNS query request, then in step 16-4T, the domain name that may be contained in the QNAME field may be extracted.

[0183] In step 16-5, DNS-G / K 150 may test whether the domain name is a member of a set contained in a Bloom filter (e.g., DNS-REG 1531). Based on the last update according to DFM 160, DNS-REG 1531 may include a set of all domain names currently registered in the DNS. When the membership test returns TRUE, or when it is determined that the name is registered in the DNS, in step 16-6T, the packet and associated DNS requests and metadata may be recorded for analysis and statistical purposes and stored in MEMORY-STORE 1532. In addition, the ratio of the rate of erroneous, false, or unregistered domain name DNS requests to the rate of all DNS requests and / or the ratio of the rate change of erroneous, false, or unregistered domain name DNS requests to the rate of all DNS requests may be updated by DNS-G / K 150, the packet may be forwarded to NETWORK 110, and DNS-G / K 150 may loop back to step 16-2 to process other packets.

[0184] When the membership test returns FALSE, and / or when it is determined that the name is not registered in the DNS and the DNS request is determined to be associated with an erroneous, false, or unregistered domain name, then in step 16-6F, the packet can be dropped or blocked. For statistical purposes, the packet and associated information and metadata (e.g., domain name, time, location, direction, source, attack type estimate, rate estimate, etc.) can be recorded, the packet and associated information and metadata can be recorded and / or captured, and the packet and associated information and metadata can be stored in the MEMORY-STORE 1532. This data can then be transmitted or uploaded to the DFM 160 as in step 16-1 above. The packet and associated information and metadata can be used for network analysis applications, threat intelligence applications, and network protection applications, such as applications for protecting and defending the global DNS and the global Internet (not in Fig.16 ). For example, during a high-velocity DDoS attack using fake DNS requests as an attack vector, logging and / or capturing packets associated with the attack may consume an unacceptably large amount of computing resources. Therefore, statistics that require fewer resources may be collected instead. Network analysis applications may identify infected hosts that may be the initiators of fake DNS requests. The packets and associated information and metadata may be provided to a network threat intelligence database, which may then be used by applications and systems protecting the network. For example, DNS infrastructure providers and Internet service providers may jointly use this data to drive global DNS and global Internet protection applications.

[0185] In step 16-7, the rate of erroneous, false or unregistered domain name DNS requests may be updated to include the occurrence of erroneous, false or unregistered domain name DNS requests. DNS-G / K 150 may determine or calculate the rate of occurrence of erroneous, false or unregistered domain name DNS requests, or the rate of occurrence of all DNS requests. DNS-G / K 150 may determine or calculate the rate of change of the rate of occurrence of false DNS requests, or the rate of change of the rate of occurrence of all DNS requests. The rate of DNS requests and / or the rate of change of the rate of DNS requests may be used in step 16-8 to determine the type of attack associated with erroneous, false or unregistered domain name DNS requests, and may be used to determine the DNS responses that may be generated (if any). In step 16-8, DNS-G / K 150 may determine what type of DNS query response (if any) to generate in response to the erroneous, false or unregistered domain name DNS query request, and may forward any generated DNS responses to network 114. When generating a DNS response, DNS-G / K 150 can spoof the actual DNS responder and can send a spoofed response to the initiator of the associated DNS request and any intermediate device that may have observed the DNS request. In the eyes of the requesting device, the spoofed response may be a response from a verified name server of the DNS.

[0186] Factors used to determine the DNS response may include: the current estimated rate of erroneous, false or unregistered domain name requests and legitimate DNS requests, the rate of change of erroneous, false or unregistered domain name requests and legitimate DNS requests, the false positive rate P of the data structure (e.g., DNS-REG 1531), the estimated lag in the case of including country codes in the domain name suffix, the estimated type of attack using erroneous, false or unregistered domain name DNS requests as an attack vector, and / or syntactic features of the domain name (e.g., information entropy value, correlation with human language words, label length, alphanumeric features, etc.). For example, based on the current value of the rate of erroneous, false or unregistered domain name DNS requests and / or the rate of change of the rate of erroneous, false or unregistered domain name DNS requests, DNS-G / K 150 may determine that a high-volume DDoS attack is occurring. Based on determining that an attack is occurring, DNS-G / K 150 may determine not to generate any DNS responses to send back to the source of the DNS request. Generating DNS responses to DNS requests for erroneous, false, or unregistered domain names (e.g., responses with a response code of NXDOMAIN) during such a DDoS attack may exacerbate the attack by consuming processing resources or filling DNS caches in intermediate DNS proxies with NXDOMAIN responses to DNS requests for erroneous, false, or unregistered domain names, which results in legitimate DNS requests and associated responses not being stored in the DNS cache, thereby causing increased DNS load.

[0187] In another example, a non-DDoS attack (such as a DNS tunnel attack or leak) may occur based on the current value of the rate of wrong, false or unregistered domain name DNS requests and the rate of change of the rate of wrong, false or unregistered domain name DNS requests. Such attacks should be prevented, but it is unlikely to over-consume processing resources or fill up the DNS cache. DNS-G / K 150 can determine that a non-DDoS attack, such as a DNS tunnel attack or leak, may be occurring. Based on determining that such an attack is occurring, DNS-G / K 150 can generate a DNS response with a response code NXDOMAIN, and the response can be transmitted to the source of the DNS request. The DNS cache in any intermediate DNS proxy can store the NXDOMAIN response generated by DNS-G / K 150, and subsequent wrong, false or unregistered domain name DNS requests, including retries of previous wrong, false or unregistered domain name DNS requests, can be processed by the intermediate DNS proxy. On the contrary, DNS-G / K 150 can decide not to generate a DNS response and transmit it back to the source of the DNS request, so as to, for example, prevent the source from realizing that it may have been discovered or otherwise identified as a malicious subject.

[0188] In yet another example, DNS-G / K 150 may determine that an erroneous, false, or unregistered domain name DNS request may be the result of a human error when operating an application (such as a web browser) that generates a DNS request or the result of a simulated human error generated by malware. Based on such a determination, DNS-G / K 150 may generate a DNS response with a response code NXDOMAIN. Based on the response code, the application may not time out when waiting for a response to the request, and may not automatically reissue the same erroneous, false, or unregistered domain name DNS request. For example, if a DNS response to a DNS request is not received, a web browser that generates an erroneous DNS request may repeat this retry cycle multiple times, which may cause a human operator to be temporarily unable to use the browser until the browser gives up. Repeatedly issuing the same erroneous DNS request may be considered a denial of service attack. By generating and sending a DNS response with a response code NXDOMAIN by DNS-G / K 150, repeated issuance of the same erroneous DNS request may be prevented.

[0189] Fig.17 is a flow chart of the operation of a DNS gatekeeper (such as DNS-G / K 150). Fig.17 Steps 17-1 to 17-4 in Fig.16 Steps 16-1 to 16-4 in FIG. 1 are substantially the same; therefore, description of these steps will not be repeated here or in the accompanying drawings. The other steps are related to processing DNS requests that may be forwarded by DNS-G / K 150 to the DNS, even if Fig.16 The domain name affiliation test in step 16-5 returns FALSE.

[0190] As described above, some requests may be related to domain names that are not in the DNS-REG 1531 due to lags (such as country code lags). For example, some gaps in the set of all country code domains may appear in the domain names registered in the local DNS-REG 1531, and the country code domain CCD-X registered in the global DNS may not be included in the domain names registered by DFM 160 in the global DNS. Fig.16 Step 16-1 or Figure 4 1531. Similarly, synchronization lag may be a factor if, for example, DNS-G / K 150 determines based on data that may be received from DFM 160 that a local Bloom filter stored in DNS-G / K 150 (e.g., DNS-REG 1531) has not been updated for a relatively long time. In addition, some requests may involve domain names that are not in DNS-REG 1531 due to the use of unregistered domain names by legitimate information exchange services.

[0191] In step 16-5, the membership test for CCD-X in the Bloom filter DNS-REG 1531 may return a FALSE value, and in step 16-6F, the associated packet may be discarded. However, when the packet is discarded, DNS-G / K150 does not collect new information. As an alternative to discarding, if it is determined that the domain name is actually registered in the DNS with a sufficiently high probability, which is for other possible reasons (such as intelligence gathering), or for example because it may be determined that the unregistered domain name is part of a legitimate information exchange service, then DNS-G / K 150 may decide not to discard the DNS request packet, but to forward the DNS request packet to the DNS. DNS-G / K 150 can monitor and observe the corresponding DNS response, which may contain a resolved IP address, thereby determining that the domain name is registered in the DNS, or contain an error code such as NXDOMAIN, thereby determining that the domain name is not registered in the DNS; or there will be no response, which may indicate a DNS tunnel attack. In view of these considerations, additional processing may be provided in place of the determination in step 16-5. In Fig.17 Such determinations are provided in steps 17.5.1 and 17.5.2.

[0192] In step 17.5.1, DNS-G / K 150 may test whether the domain name is a member of the set contained in Bloom filter DNS-REG 1531, which represents all domain names currently registered in the DNS. If the membership test returns TRUE, so that DNS-G / K 150 determines that the name is registered in the DNS, then in step 17-6T, the packet and associated DNS request may be recorded for statistical purposes and may be stored in MEMORY-STORE 1532. DNS-G / K 150 may forward the packet to NETWORK 110, and DNS-G / K 150 may loop back to step 17-2 and process more packets.

[0193] If the membership test returns FALSE, thereby making it determined that the domain name is not registered in the DNS-REG 1531, then in step 17.5.2, the DNS-G / K 150 may determine whether to forward the packet to the DNS. The forwarding determination may be based on a determination by the DNS-G / K 150 that the FALSE result of the membership test is due to some type of lag. For example, the DNS-G / K 150 may determine whether the domain name includes a suffix that is a country code. Based on the determination that the domain name includes a suffix that is a country code, the DNS-G / K 150 may conclude that the FALSE result may be due to a country code lag (as described above). Alternatively, the DNS-G / K 150 may determine that the local copy of the Bloom filter DNS-REG 1531 has not been updated by the DFM 160 for a considerable period of time, and may conclude that the FALSE result may be due to a synchronization lag (as described above). The forwarding determination may also be based on the DNS-G / K 150 determining that the FALSE result of the membership test is due to the use of an unregistered domain name as part of a legitimate information exchange service. For example, DNS-G / K 150 may determine that the unregistered domain name includes a suffix that corresponds to an authoritative name server operated by a legitimate information exchange service.

[0194] When, in step 17-5.2, DNS-G / K 150 determines not to forward the DNS request packet to the DNS, steps 17-6F, 17-7, and 17-8 may be performed. These steps are similar to Fig.16 Steps 16-6F, 16-7, and 16-8 of are substantially the same; therefore, a detailed description is not repeated here. When DNS-G / K 150 determines to forward the DNS request packet, in step 17-9, DNS-G / K 150 may forward the DNS request packet to the DNS via NETWORK 110. When DNS-G / K 150 determines that the domain name in the request is not registered in DNS-REG 1531 but the request packet should be forwarded, DNS-G / K 150 may activate logic to monitor communications associated with the forwarded request packet, and may detect a corresponding DNS response. In step 17-10, DNS-G / K 150 may detect or observe a corresponding DNS response (if any) transmitted to NETWORK 114.

[0195] In step 17-11, DNS-G / K 150 may check whether the DNS response indicates a resolved IP address, thereby determining that the domain name is not represented in DNS-REG 1531 but is registered in the DNS. If DNS-G / K 150 does not observe a DNS response indicating a resolved IP address, DNS-G / K 150 may perform steps 17-6F, 17-7, and 17-8 (as described above), and DNS-G / K 150 may loop back to step 17-2 to process other packets. If DNS-G / K 150 does observe a DNS response indicating a resolved IP address, then in step 17-12, DNS-G / K 150 may insert the domain name into the local DNS-REG 1531, and may record the domain name for informational purposes, and may upload it to DFM 160 in step 17-1. In step 17-13, DNS-G / K 150 may forward the DNS response packet to its destination (via NETWORK 114), and DNS-G / K 150 may loop back to step 17-2 to process other requests.

[0196] In step 17-10, it may also be the case that no DNS response is observed. For example, when the associated DNS request is a path for a DNS tunnel attack (eg, leakage), the collection server CSVR 148 may not be configured to respond to the DNS request. Fig.17 The logic path for handling this situation is not shown in FIG. 1. When no DNS response is observed, DNS-G / K 150 may determine that the DNS request may be a path for a DNS tunnel attack. DNS-G / K 150 may capture, record, and / or log the DNS request, and may generate an alert to a network analysis application that a DNS tunnel attack may have occurred. In step 17-1, the alert associated with the single domain name or multiple domain names and related information may be uploaded to DFM 160.

[0197] Fig.18 is a flow chart showing the concept of operation of the DNS filter manager (DFM 160). Note that Fig.18 All steps shown in the drawings may be performed in different orders, not just the order shown, and may be performed simultaneously and sequentially. Fig.18 The order of steps shown in the drawings is representative and for descriptive purposes only and is not intended to be limiting or restrictive in any way.

[0198] In step 18-1, DFM 160 may collect and may aggregate a list of domain names currently registered in the DNS. DFM 160 may create a database of domain names that may be accessed by or shared with other computing devices. Sources of these lists may include zone files, which may be discovered by DNS registry operators and / or associated representative and / or authoritative name servers, domain list aggregator services, third-party services that discover country code domains (CCDs), ICANN, etc.

[0199] In step 18-2, DFM 160 may snapshot a database of currently registered domain names, possibly according to a schedule that may be periodic. The snapshot may be used as source data for a DNS-REG 1531 data structure (e.g., a Bloom filter) that may be provided to subscribing computing devices (such as DNS-G / K 150 and 180). Using the data in the snapshot, DFM 160 may generate multiple DNS-REG 1531 instances. Each instance DNS-REG PX The false alarm rate P X For probabilistic data structures such as Bloom filters, the logarithmic size of the false positive rate P can be proportional to the memory requirement for storing the elements of the set. That is, for each DNS-REG 1531 instance, the number of instances of DNS-REG PX The lower the associated false positive rate, the more memory may be required to store a given number of elements in the set. Each instance of the local DNS-G / K 150 may have different local resource requirements. For example, each local DNS-G / K 150 may have different local resource requirements for storing DNS-REG. PX Instances have different limits on the local main memory available. Therefore, DFM 160 may provide multiple DNS-REGs with different false positive rates and different memory sizes. PX Instance, so that a given DNS-G / K 150 can select a DNS-REG that meets local resource requirements PX Example.

[0200] In step 18-3, DFM 160 may publish the newly created DNS-REG PX Instances and associated information / metadata, such as timestamp data, and enable DNS-REG PX The instance is available to authorized DNS-G / K subscribers. In step 18-4, DFM 160 may send the selected DNS-REG to each requesting subscriber. PXThe DFM 160 may log each such request and response, and the log data may be used by global monitoring applications of the DNS-G / K 150 and other subscribers. Such monitoring applications may send alerts to administrators of subscribing DNS-G / K 150 instances that have not recently updated their local DNS-REG instances.

[0201] In step 18-5, DFM 160 may upload and store information sent by subscribing DNS-G / K 150. As described above, this information may include unregistered domain names detected by subscribing DNS-G / K 150 and associated metadata (e.g., timestamp, source host, estimated attack type, etc.), which may be used, for example, as threat intelligence when detecting relevant packets. The information may also include statistics of DNS requests and responses observed by DNS-G / K 150, DNS-G / K 150 element management information, domain names that DNS-G / K 150 may have determined to be registered in the DNS but may not be included in the local instance of Bloom filter DNS-REG 1531. Such domain names may have been in the local instance of Bloom filter DNS-REG 1531 in relation to the DNS-G / K 150. Fig.17 The process described in steps 17-12 is determined.

[0202] In step 18-6, DFM 160 may respond to the request for information, including information already collected in step 18-5 (described above). The request for information may be received from various client systems, such as network analysis applications, threat intelligence provider services, Internet and DNS monitoring systems, domain list aggregator services, etc. DFM 160 may loop back to step 18-1, or the above processes may be performed simultaneously or continuously.

[0203] Fig.18 Some possible variations of the DFM 160 operating concept shown may reduce resource requirements and reduce latency. Fig.18 The most recently generated DNS-REG for all registered domain names is contained in PX Versions can be distributed to each subscriber when requested. At certain scale and high request frequency, continuously distributing full instances of DNS-REG 1531 may be inefficient. To reduce resource consumption, subscribers can request a "delta" set instead of updating the entire DNS-REG. PX That is, the subscriber may indicate to the DFM 160 the DNS-REG PXThe DFM 160 may calculate the delta between version x.04-01-2019.0800 and the latest version (e.g., x.04-02-2019.0800). The delta set may include all newly registered domain names included in version x.04-02-2019.0800 but not included in x.04-01-2019.0800, and may include all recently deregistered domain names included in x.04-01-2019.0800 but not included in x.04-02-2019.0800. This delta set may be sent to the requesting subscriber DNS-G / K 150. The requesting subscriber DNS-G / K 150 may insert newly registered domain names into its local copy of the DNS-REG 1531 and may delete recently deregistered domain names from its local copy of the DNS-REG 1531. In another variation, the DFM 160 may provide a continuous feed of newly registered domain names and newly deregistered domain names. Subscribers to the continuous feed may insert newly registered domain names into their local DNS-REG and delete recently deregistered domain names from their local DNS-REG. As described above, this variation may reduce the lag processing problem.

[0204] Fig.18 Some other possible variations of the DFM 160 operational concepts shown in can reduce DNS-G / K 150 local resource requirements, including memory requirements and processing requirements. For example, some DNS-G / K 150 may not need a full DNS-REG 1531 with all known / available registered domain names to provide effective protection against attacks, but may only need to register a relatively small subset of domain names in its local DNS-REG 1531 instance. Figure 1 , consider a network 116 that may be operated by a first DNS infrastructure provider organization (e.g., DIPO-X). DIPO-X may operate two (2) authoritative name servers NS 144 and NS 145, which may be, for example, authoritative name servers for domains example-144.com and example-145.com, respectively. The DNS may be configured such that all DNS query requests for domain names *.example-144.com and *.example-145.com will be routed to the authoritative name servers NS 144 and NS 145 attached to the network 116. Thus, such requests will pass through or be filtered by the DNS gatekeeper DNS-G / K 180 and will be tested to see if they belong to the local DNS-REG 1531 instance of the DNS gatekeeper DNS-G / K 180.

[0205] However, it is likely that most, if not all, of the DNS query requests entering the network 116 may be both legitimate DNS requests for resolving domain names of the form *.example-144.com and *.example-145.com and erroneous, false, or unregistered domain name DNS requests. Therefore, the local DNS-REG 1531 of the DNS gatekeeper DNS-G / K 180 need not be populated with every registered domain name in the DNS, but instead only needs to be populated with registered domain names of the form *.example-144.com and *.example-145.com. A Bloom filter containing only these domain names will require less local memory and less time to perform membership tests than a full Bloom filter DNS-REG 1531. Therefore, it may be beneficial if the DFM 160 allows subscribers to customize the Bloom filters that the subscribers use to configure their DNS gatekeepers. The DFM 160 can maintain and can update the customized Bloom filters and provide them to the associated subscribers upon request.

[0206] Alternatively, since DIPO-X may be the registrar and administrator of the authoritative name servers example-144.com and example-145.com, DIPO-X may create and manage a Bloom filter containing registered domain names of the form *.example-144.com and *.example-145.com, and use this Bloom filter to configure DNS-G / K 180. DIPO-X may also provide this Bloom filter to DFM 160 as an alternative to publishing zone files. DFM 160 may then include this Bloom filter in the Bloom filter DNS-REG 1531 that DFM 160 distributes to subscribers (e.g., by forming a union). In this way, DIPO-X may retain any privacy and protection it desires by not discovering / publishing its zone files, but may participate in helping to protect the DNS and the Internet together with the administrators of DFM 160 and DNS-REG 1531 filters. In practice, for example, this approach may be useful for registrars of country code zones that do not publish their zone files.

[0207] Fig.19The sequence of events of an interconnected network embodiment with a DNS gatekeeper that can mitigate or prevent DDoS attacks on DNS is shown. In steps 19-1a and 19-1b, malware that has compromised devices IOT 140 and IOT 142 contacts botnet command and control system CNC 149 to notify command and control system CNC 149 of their presence and request instructions. In steps 19-2a and 19-2b, command and control system CNC 149 can instruct IOT 140 and IOT 142 to attack the DNS authoritative name servers of example-144.com and example-146.com at a specified time T with false, false or unregistered domain name DNS requests.

[0208] In step 19-3a, at a specified time T, IOT 140 generates a fake DNS request to resolve the domain name 6pPhAUPd.example-146.com. IOT 140 uses a random string generator to create the string "6pPhAUPd", which is then used as a subdomain label to construct a domain name, which DNS routes to the authoritative name server NS146 of example-146.com for resolution. This method of domain name generation assumes that the resulting domain name is extremely unlikely to have been registered in the DNS. IOT 140 forwards the fake DNS request to the authoritative name server NS146 of example-146.com.

[0209] In step 19-4a, the authoritative name server NS 146 receives the DNS request to resolve 6pPhAUPd.example-146.com. Since 6pPhAUPd.example-146.com has never been registered in the DNS, NS 146 wastes resources searching its DNS database for the corresponding DNS entry but does not find it. NS 146 generates a corresponding DNS response to the request with RCODE=3 ("NXDOMAIN", which signals that the domain name does not exist in the DNS) and forwards the DNS response to the requester IOT 140. Any caching DNS proxy in the network path between the authoritative name server (e.g., NS 146) and the device (e.g., IOT 140) can insert a cache entry for the {6pPhAUPd.example-146.com, NXDOMAIN} pair.

[0210] Therefore, the false DNS request generated by IOT 140 in step 19-3a wastes network bandwidth resources, DNS name server resources, and DNS cache resources. These resources may no longer be available to serve legitimate DNS requests.

[0211] In step 19-3b, at a specified time T, similar to step 19-3a, IOT 142 generates a false DNS request to resolve the domain name B2f8O2vR.example-146.com. IOT 142 may forward the false DNS request to the authoritative name server NS 146 of example-146.com. When the false DNS request leaves network 114, it may be received by DNS gatekeeper DNS-G / K 150. DNS-G / K 150 may test whether B2f8O2vR.example-146.com is a member of its local instance of Bloom filter DNS-REG. The membership test may return FALSE, thereby determining that B2f8O2vR.example-146.com is not a member of Bloom filter DNS-REG and therefore it is not registered in DNS. DNS-G / K 150 may examine its rate of invalid or false DNS requests, the rate of change of the rate of invalid or false DNS requests, and other parameters, and decide that the request may be part of a DDoS attack. Thus, DNS-G / K 150 may drop packets containing DNS requests, but may not generate corresponding DNS responses to send back to the originator of the DNS request (as this would consume resources and thus facilitate the execution of a DDoS attack). DNS-G / K 150 may log, capture, and locally store false DNS requests for possible future retrieval by, for example, a threat intelligence provider service and / or a network analysis service. DNS-G / K 150 thereby prevents false DNS requests from attacking the DNS and wasting resources that could instead be applied to servicing legitimate DNS requests.

[0212] In step 19-3c, at a specified time T, similar to step 19-3a, IOT 140 generates a fake DNS request to resolve the domain name V00zln5k.example-144.com. IOT 140 may forward the fake DNS request to name server NS 144 of example-144.com attached to network 116. Before entering network 116, the request may be received by DNS gatekeeper DNS-G / K 180 (DIPO-X is installed at the boundary of network 116 to protect its DNS infrastructure). DNS gatekeeper DNS-G / K 180 may test whether V00zln5k.example-144.com is a member of its local Bloom filter DNS-REG instance. The membership test returns FALSE, so it can be determined that V00zln5k.example-144.com is not a member of the Bloom filter DNS-REG, and therefore it is not registered in the DNS. DNS gatekeeper DNS-G / K 180 may decide that the fake DNS request may be part of a DDoS attack. Therefore, the DNS gatekeeper DNS-G / K 180 can discard packets containing DNS requests, but does not need to generate corresponding DNS responses to send back to the initiator of the DNS request. The DNS gatekeeper DNS-G / K 180 thereby prevents false DNS requests from attacking the DNS and wasting resources that can be used to serve legitimate DNS requests instead.

[0213] In step 19-3d, at a specified time T, similar to step 19-3b, IOT 142 generates a false DNS request to resolve the domain name I9BVhImV.example-144.com. IOT 142 forwards the false DNS request to name server NS 144 of example-144.com. When the false DNS request leaves network 114, it is received by DNS gatekeeper DNS-G / K 150. DNS-G / K 150 tests whether I9BVhImV.example-144.com is a member of its local Bloom filter DNS-REG instance. The membership test returns FALSE, so it is determined that I9BVhImV.example-144.com is not a member of the Bloom filter DNS-REG, and therefore it is not registered in the DNS. DNS-G / K 150 examines the rate of invalid or false DNS requests and / or the rate of change of the rate of invalid or false DNS requests, and decides that the request may be part of a DDoS attack. Therefore, DNS-G / K 150 discards the packet containing the DNS request, but does not generate a corresponding DNS response to send back to the initiator of the DNS request. DNS-G / K 150 thereby prevents false DNS requests from attacking the DNS and wasting resources that could be used instead to serve legitimate DNS requests.

[0214] Fig. 20 A sequence of events for an embodiment of an internetwork with a DNS gatekeeper that can mitigate or prevent DNS tunneling attacks is shown.

[0215] Recall the above and Fig.14 , host PII 141 attached to network 112 and PII 143 attached to network 114 may be servers that manage sensitive information, such as personally identifiable information (PII), credit card information, etc. Host PII 141 and PII 143 have also been infected by malware that collects credit card information and then uses a DNS tunneling method to exfiltrate the information to a collection server CSVR 148 attached to network 170 (described below). CSVR 148 has been assigned a public IP address 21.43.65.87. Collection server CSVR 148 may be configured with a DNS server that can ingest DNS requests on port 53, a well-known port for DNS. The malware on PII 141 and PII 143 has been configured to send DNS requests to IP address 21.43.65.87, the IP address of collection server CSVR 148.

[0216] In step 20-1, PII 141 attached to network 112 reads the credit card number "1234123412341234" and the associated credit card issuer name "card" from a local database, constructs a (fake) DNS request to resolve, for example, 1234123412341234card.csvr-148.tk, encapsulates the request in a UDP datagram packet with a destination port 53, encapsulates the datagram in an IP packet with a destination IP address of 21.43.65.87, and forwards the packet to 21.43.65.87 (CSVR 148). The packet is routed through the internetwork to collection server CSVR 148 port 53, and the DNS server listening on port 53 receives the fake DNS request to resolve 1234123412341234card.csvr-148.tk. The DNS server extracts the (stolen) credit card information "1234123412341234" from the DNS request and stores the DNS request in a local database, thus successfully completing the exfiltration. The DNS server does not generate a corresponding DNS response for the fake DNS request, for example, in order to avoid detection.

[0217] In step 20-2, PII 143 attached to network 114 may read the credit card number "4321432143214321" and the associated credit card issuer name "card2" from a local database, construct a fake DNS request to resolve, for example, 4321432143214321card2.csvr-148.tk. PII 143 may encapsulate the request in a UDP datagram packet with a destination port 53, may encapsulate the datagram in an IP packet with a destination IP address of 21.43.65.87, and may forward the packet to the IP address (21.43.65.87) of collection server CSVR 148. When the packet leaves network 114, it may be received by DNS gatekeeper DNS-G / K 150. DNS-G / K 150 may test whether the request 4321432143214321card2.csvr-148.tk is a member of its local Bloom filter DNS-REG instance. The membership test may return FALSE, thus determining that 4321432143214321card2.csvr-148.tk is not a member of the Bloom filter DNS-REG, and therefore it is not registered in the DNS. DNS-G / K 150 may check the rate of error, false or unregistered domain name DNS requests and legitimate DNS requests stored by the local gatekeeper and / or the rate of change of the rate of error, false or unregistered domain name DNS requests and legitimate DNS requests and other parameters, and may decide that the request is not part of a DDoS attack, but may be part of a DNS tunnel attack. Therefore, DNS-G / K 150 may discard the packet containing the DNS request, but may not generate a corresponding DNS response to send back to the initiator of the DNS request. DNS-G / K 150 may thereby prevent false DNS requests from leaking stolen credit card numbers to malicious entities. DNS-G / K 150 may record, capture, and locally store false DNS requests. This information may be provided to, for example, a threat intelligence provider service and / or a network analysis service, etc. These services can identify malicious collection server CSVR 148 (IP address 21.43.65.87) and network 170 as being associated with a malicious actor, and identify host PII 143 as being compromised by exfiltrated malware. This information can be used to further protect the network, for example, by the enterprise disinfecting host PII 143 and / or by threat intelligence providers distributing the IP address 21.43.65.87 of collection server CSVR 148 and associated threat metadata (e.g., the attack type is "DNS tunneling") as a threat indicator to subscribers, as well as distributing a block of IP address space of the (malicious) network 170, such as 21.43.65.0 / 24.

[0218] The functions and steps described herein may be embodied in computer-usable data or computer-executable instructions executed by one or more computers or other devices for performing one or more functions described herein, such as in one or more program modules. Typically, program modules include routines, programs, objects, components, data structures, etc., which perform specific tasks or implement specific abstract data types when executed by one or more processors in a computer or other data processing device. Computer-executable instructions may be stored on computer-readable media such as hard disks, optical disks, removable storage media, solid-state memory, RAM, etc. It should be understood that the functions of program modules may be combined or allocated as needed. In addition, the functions may be embodied in firmware or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. Specific data structures may be used to more effectively implement one or more aspects of the present disclosure, and such data structures are considered to be within the scope of computer-executable instructions and computer-usable data described herein.

[0219] Although not required, it will be appreciated by one of ordinary skill in the art that the various aspects described herein may be embodied as a method, system, apparatus, or as one or more computer-readable media storing computer-executable instructions. Thus, the various aspects may take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, or an embodiment combining software, hardware, and firmware aspects in any combination.

[0220] As described herein, various methods and actions may be operated on one or more computing devices and networks. Functionality may be distributed in any manner, or may be located in a single computing device (eg, server, client computer, etc.).

[0221] The computer software, hardware, and networks may be utilized in a variety of different system environments, including stand-alone, networked, remote access (also known as remote desktop), virtualized and / or cloud-based environments, among other environments. Fig.21An example of a system architecture and data processing device that can be used to implement one or more illustrative aspects described herein in an independent and / or networked environment is shown. Various network nodes 2103, 2105, 2107 and 2109 can be interconnected via a wide area network (WAN) 2101 such as the Internet. Other networks can also or alternatively be used, including a dedicated intranet, a company network, a local area network (LAN), a metropolitan area network (MAN), a wireless network, a personal network (PAN), etc. Network 2101 is for illustrative purposes, and can be replaced by less or additional computer networks. Local area network 2133 can have one or more of any known LAN topology, and can use one or more of various different protocols, such as Ethernet. Devices 2103, 2105, 2107 and 2109 and other devices (not shown) can be connected to one or more networks via twisted pair, coaxial cable, optical fiber, radio waves or other communication media.

[0222] The term "network" as used herein and depicted in the accompanying drawings refers not only to a system in which remote storage devices are coupled together via one or more communication paths, but also to independent devices that may be coupled to such a system from time to time with storage capabilities. Thus, the term "network" includes not only a "physical network" but also a "content network", which consists of data (attributable to a single entity) residing in all physical networks.

[0223] These components may include a data server 2103, a web server 2105, and client computers 2107, 2109. For example, DNS-G / K 150 may be implemented as any one of a plurality of different devices, including each of various network nodes 2103, 2105, 2107, and 2109. The data server 2103 provides overall access, control, and management of the database and control software for performing one or more illustrative aspects described herein. The data server 2103 may be connected to the web server 2105, through which the user interacts and obtains the requested data. Alternatively, the data server 2103 itself may act as a web server and be directly connected to the Internet. The data server 2103 may be connected to the web server 2105 via a local area network 2133, a wide area network 2101 (e.g., the Internet) via a direct or indirect connection or via some other network. A user may use a remote computer 2107, 2109, such as a web browser, to connect to the data server 2103 via one or more externally published websites hosted by the web server 2105 to interact with the data server 2103. The client computers 2107, 2109 may be used in conjunction with the data server 2103 to access data stored therein, or may be used for other purposes. For example, through the client device 2107, a user may access the web server 2105 using an internet browser as known in the art or by executing a software application that communicates with the web server 2105 and / or the data server 2103 via a computer network (such as the internet).

[0224] The server and application may be combined on the same physical machine and retain separate virtual or logical addresses, or may reside on separate physical machines. Fig.21 Only one example of the equipment and network architecture that may be used is shown, and those skilled in the art will appreciate that the specific equipment and network architecture and data processing equipment used may vary and are secondary to the functionality provided, as further described herein. For example, the services provided by the web server 2105 and the data server 2103 may be combined on a single server.

[0225] Each component 2103, 2105, 2107, 2109 can be any type of known computer, server or data processing device. The data server 2103 can, for example, include a processor 2111 that controls the overall operation of the data server 2103. The data server 2103 can further include a random access memory (RAM) 2113, a read-only memory (ROM) 2115, a network interface 2117, an input / output interface 2119 (e.g., keyboard, mouse, display, printer, etc.) and a memory 2121. The input / output (I / O) 2119 can include various interface units and drivers for reading, writing, displaying and / or printing data or files. The memory 2121 can further store operating system software 2123 for controlling the overall operation of the data processing device 2103, control logic 2125 for instructing the data server 2103 to perform aspects described herein, and other application software 2127 that provides auxiliary, support and / or other functions that may or may not be used in conjunction with aspects described herein. The control logic may also be referred to as data server software 2125 in this article. The functionality of the data server software may refer to a combination of operations or decisions made automatically based on rules encoded into the control logic, operations or decisions made manually by a user providing input to the system, and / or automatic processing based on user input (e.g., queries, data updates, etc.).

[0226] The memory 2121 may also store data for performing one or more aspects described herein, including a first database 2129 and a second database 2131. The first database may include a second database (e.g., as a separate table, report, etc.). That is, information may be stored in a single database, or may be divided into different logical, virtual or physical databases, depending on the system design. Devices 2105, 2107 and 2109 may have similar or different architectures as described with respect to device 2103. Those skilled in the art will appreciate that the functionality of the data processing device 2103 (or device 2105, 2107 or 2109) as described herein may be distributed across multiple data processing devices, for example, to distribute the processing load across multiple computers, to separate transactions based on geographic location, user access level, quality of service (QoS), etc.

[0227] One or more aspects may be embodied in computer-usable or readable data and / or computer-executable instructions executed by one or more computers or other devices described herein, such as in one or more program modules. Typically, program modules include routines, programs, objects, components, data structures, etc., which perform specific tasks or implement specific abstract data types when executed by a processor in a computer or other device. Modules may be written in a source code programming language and then compiled for execution, or modules may be written in a scripting language, such as (but not limited to) Hypertext Markup Language (HTML) or Extensible Markup Language (XML). Computer executable instructions may be stored on a computer-readable medium such as a non-volatile storage device. Any suitable computer-readable storage medium may be utilized, including a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, and / or any combination thereof. In addition, various transmission (non-storage) media representing data or events described herein may be transmitted between a source and a destination by traveling in the form of electromagnetic waves through a signal-conducting medium such as a metal wire, an optical fiber, and / or a wireless transmission medium (e.g., air and / or space). Various aspects described herein may be embodied as a method, a data processing system, or a computer program product. Thus, the various functions may be embodied in whole or in part in software, firmware, and / or hardware or hardware equivalents, such as integrated circuits, field programmable gate arrays (FPGAs), etc. Certain data structures may be used to more efficiently implement one or more aspects described herein, and such data structures are considered to be within the scope of computer-executable instructions and computer-usable data described herein.

[0228] Further references Fig. 22 , one or more aspects described herein may be implemented in a network or remote access environment. Fig. 22 An exemplary network architecture is depicted, which includes a computing device 2201 in an exemplary computing environment 2200 that can be used according to one or more illustrative aspects described herein. For example, a DNS filter manager (DFM) 160 can be implemented using features of the computing device 2201. The computing device 2201 can be used as a server 2206a in a single-server or multi-server desktop virtualization system (e.g., a remote access or cloud system) configured to provide virtual machines to client access devices. The computing device 2201 can have a processor 2203 for controlling the overall operation of the server and its associated components, including RAM 2205, ROM 2207, input / output (I / O) modules 2209, and storage 2215.

[0229] The I / O module 2209 may include a mouse, a keypad, a touch screen, a scanner, an optical reader, and / or a stylus (or other input device) through which a user of the computing device 2201 can provide input, and may also include one or more speakers for providing audio output and one or more video display devices for providing text, audiovisual, and / or graphical output. Software may be stored in the memory 2215 and / or other storage devices to provide instructions to the processor 2203 for configuring the computing device 2201 as a special-purpose computing device to perform the various functions described herein. For example, the memory 2215 may store software used by the computing device 2201, such as an operating system 2217, application programs 2219, and associated databases 2221.

[0230] The computing device 2201 can operate in a networked environment that supports connections to one or more remote computers, such as a terminal 2240 (also referred to as a client device). The terminal 2240 can be a personal computer, a mobile device, a laptop, a tablet computer, or a server, which includes many or all of the elements described above with respect to the computing device 2103 or 2201. Fig. 9 The network connections depicted in the figure include a local area network (LAN) 2225 and a wide area network (WAN) 2229, but other networks may also be included. When used in a LAN networking environment, the computing device 2201 can be connected to the LAN 2225 via a network interface or adapter 2223. When used in a WAN networking environment, the computing device 2201 may include a modem 2227 or other wide area network interface for establishing communications through a WAN 2229 such as a computer network 2230 (e.g., the Internet). It should be understood that the network connections shown are exemplary and other means of establishing a communication link between computers may be used. The computing device 2201 and / or the terminal 2240 may also be a mobile terminal (e.g., a mobile phone, a smart phone, a personal digital assistant (PDA), a notebook, etc.) including various other components (such as a battery, a speaker, and an antenna (not shown)).

[0231] The various aspects described herein may also operate with many other general or special computing system environments or configurations. Examples of other computing systems, environments, and / or configurations that may be applicable to the aspects described herein include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers (PCs), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0232] like Fig. 22As shown, one or more client devices 2240 can communicate with one or more servers 2206a-2206n (collectively referred to herein as "servers 2206"). In one embodiment, the computing environment 2200 can include a network device installed between the servers 2206 and the client machines 2240. The network device can manage client / server connections and, in some cases, can load balance client connections between multiple back-end servers 2206.

[0233] The client machine 2240 may be referred to as a single client machine 2240 or a single group of client machines 2240, and the server 2206 may be referred to as a single server 2206 or a single group of servers 2206. In one embodiment, a single client machine 2240 communicates with more than one server 2206, while in another embodiment, a single server 2206 communicates with more than one client machine 2240. In yet another embodiment, a single client machine 2240 communicates with a single server 2206.

[0234] Client machine 2240 may be referenced by any of the following non-exhaustive terms: client machine; client; client computer; client device; client computing device; local machine; remote machine; client node; endpoint; or endpoint node. Server 2206 may be referenced by any of the following non-exhaustive terms: server; local machine; remote machine; server farm, or host computing device.

[0235] In one embodiment, the client machine 2240 can be a virtual machine. The virtual machine can be any virtual machine, and in some embodiments, the virtual machine can be any virtual machine managed by a type 1 or type 2 hypervisor (e.g., a hypervisor developed by Citrix, IBM, VMware, or any other hypervisor). In some aspects, the virtual machine can be managed by a hypervisor, and in other aspects, the virtual machine can be managed by a hypervisor executing on the server 2206 or a hypervisor executing on the client 2240.

[0236] The client device 2240 can display application output generated by an application remotely executed on the server 2206 or other remotely located machine, and the client device 2240 can execute a virtual machine receiver program or application to display the output in an application window, a browser, or other output window. In one example, the application is a desktop, while in other examples, the application is an application that generates or presents the desktop. The desktop can include a graphical shell that provides a user interface for an operating system instance in which local and / or remote applications can be integrated. As used herein, an application is a program that is executed after an instance of an operating system (and optionally, a desktop) has been loaded.

[0237] The server 2206 may send data to a thin client or remote display application executing on the client using a remote presentation protocol or other program to present display output generated by the application executing on the server 2206. The thin client or remote display protocol may be a protocol such as the Independent Computing Architecture (ICA) developed by Citrix Systems, Inc. of Fort Lauderdale, Florida; or the Remote Desktop Protocol (RDP) manufactured by Microsoft Corporation of Redmond, Washington.

[0238] The remote computing environment may include more than one server 2206a-2206n, such that the servers 2206a-2206n are logically grouped together into a server farm 2206, for example, in a cloud computing environment. The server farm 2206 may include servers 2206 that are geographically dispersed while being logically grouped together, or servers 2206 that are located adjacent to each other while being logically grouped together. The geographically dispersed servers 2206a-2206n within the server farm 2206 may communicate using a WAN (wide area), MAN (metropolitan area), or LAN (local area), where different geographic areas may be characterized as: different continents; different regions of a continent; different countries; different states; different cities; different campuses; different rooms; or any combination of the above geographic locations. The server farm 2206 may be managed as a single entity, or the server farm 2206 may include multiple server farms.

[0239] The server farm may include servers 2206 that execute substantially similar types of operating system platforms (e.g., WINDOWS, UNIX, LINUX, iOS, ANDROID, SYMBIAN, etc.). The server farm 2206 may include a first group of one or more servers that execute a first type of operating system platform, and a second group of one or more servers that execute a second type of operating system platform.

[0240] Server 2206 can be configured as any type of server as needed, such as a file server, an application server, a web server, a proxy server, a device, a network device, a gateway, an application gateway, a gateway server, a virtualization server, a deployment server, a secure socket layer (SSL) VPN server, a firewall, a web server, an application server or as a main application server, a server executing an active directory or a server executing an application acceleration program, which provides a firewall function, an application function or a load balancing function. Other server types can also be used.

[0241] The first server 2206a can receive a request from a client machine 2240, forward the request to a second server 2206b (not shown), and respond to the request generated by the client machine 2240 with a response from the second server 2206b (not shown). The first server 2206a can obtain an enumeration of applications available to the client machine 2240 and address information associated with the application server 2206 hosting the application identified in the enumeration of applications. The first server 2206a can then present a response to the client request using a web interface and communicate directly with the client 2240 so that the client 2240 can access the identified application. One or more clients 2240 and / or one or more servers 2206 can transmit data over a network 2230 (e.g., network 2101).

[0242] Various aspects of the present disclosure have been described in terms of their exemplary embodiments. Upon review of this disclosure, a person of ordinary skill in the art may conceive of many other embodiments, modifications, and variations within the scope and spirit of the appended claims. For example, a person of ordinary skill in the art will appreciate that the steps shown in the illustrative drawings may be performed in an order different from that listed, and one or more of the steps shown may be optional. Any and all features in the appended claims may be combined or rearranged in any possible manner.

Claims

1. A method for preventing attacks associated with a Domain Name System (DNS), the method comprising: storing, by a gatekeeper, a probabilistic data structure representing a subset of a plurality of domain names in the DNS, wherein the probabilistic data structure representing the subset of the plurality of domain names in the DNS is selected from the plurality of probabilistic data structures based on false positive rates associated with a plurality of probabilistic data structures representing the plurality of subsets of the plurality of domain names in the DNS; receiving a plurality of packets via the gatekeeper; determining, based on the grouping information, whether each corresponding group in the plurality of groups includes a domain name; based on determining that a first packet in the plurality of packets includes a first domain name, testing the probabilistic data structure to determine whether the first domain name can be resolved by the DNS without querying the DNS; as well as Based on determining that the first domain name cannot be resolved by the DNS: - determining the legitimacy of said first group according to at least one criterion; as well as - Based on the legitimacy, discarding the first packet.

2. The method according to claim 1, further comprising: Based on determining that the first domain name cannot be resolved by the DNS, data regarding the first packet is transmitted to a management server.

3. The method according to claim 1 or claim 2, further comprising: Based on determining that a second packet in the plurality of packets does not include a domain name, the second packet is transmitted to its intended destination.

4. The method according to any one of claims 1 to 3, further comprising: testing the probabilistic data structure to determine whether a second domain name of a second packet is represented in the probabilistic data structure without querying the DNS; determining whether the second packet is legitimate based on determining that the second domain name is not represented in the probabilistic data structure; as well as Based on determining that the second packet is legitimate, the second packet is transmitted to its intended destination.

5. The method according to claim 4, further comprising: monitoring network traffic for a response to the second packet; inserting the second domain name into the probabilistic data structure based on the response to the second packet; as well as The second domain name is transmitted to the management server.

6. The method according to any one of claims 1 to 5, wherein determining the legitimacy of the first packet comprises: determining a rate of received invalid DNS query requests associated with the first domain name, wherein the received invalid DNS query requests include DNS query requests that cannot be resolved by the DNS; Determining whether a rate of the received invalid DNS query requests meets a threshold; as well as Based on determining that the rate of the received invalid DNS query requests satisfies the threshold, the legitimacy of the first packet is determined.

7. The method according to any one of claims 1 to 6, wherein determining the legitimacy of the first packet comprises: determining whether a country code is associated with the first domain name; determining whether the country code indicates a hysteresis; as well as Based on determining that the country code does not indicate hysteresis, the legitimacy of the first packet is determined.

8. The method according to any one of claims 1 to 7, wherein the probabilistic data structure is one of: Bloom filter; or Cuckoo filter.

9. The method according to any one of claims 1 to 8, wherein the at least one criterion comprises one or more of the following: The rate of incorrect, false, or unresolved DNS requests for domain names; whether one or more portions of the first grouping are associated with human language words; one or more label lengths associated with the first packet; a country code associated with said first grouping; the timing of updates received to the probabilistic data structure; or The syntactic features of the first domain name.

10. The method according to any one of claims 1 to 9, wherein the at least one criterion comprises one or more of the following: Internet Protocol (IP) address, 5-tuple, Domain name, or Uniform Resource Identifier (URI).

11. The method of any one of claims 1-10, wherein the subset of the plurality of domain names comprises a subset of domain names resolvable in the DNS.

12. The method of any one of claims 1-11, wherein the first packet comprises a first DNS query request, and wherein determining the legitimacy of the first packet comprises determining whether the first DNS query request is associated with one or more of: Legitimate attempts to resolve domain name requests; Deny a request submitted in an attempt to provide service to another person; or Attempts to leak information.

13. A computing device comprising: at least one processor; and A memory comprising instructions which, when executed by the at least one processor, cause the computing device to perform a method according to any one of claims 1-12.

14. A system for preventing attacks associated with a Domain Name System (DNS), comprising: The computing device according to claim 13; as well as A second computing device is configured to transmit at least one packet of the plurality of packets.

15. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to perform the method of any one of claims 1-12.

Images