Method and computing unit for the automated finding and assignment of domains

Abstract

The procedure for automatically locating and assigning domains belonging to a domain owner comprises the following steps: • Maintaining a known domain of the domain owner; • Determining a nameserver (3a, 3b) that manages the domain by passing the known domain to a computer unit (1) set up to determine the nameserver (3a, 3b), which is connected to the Internet (2) and starting an Internet query for the nameserver by the computer unit (1); • Designating the identified nameserver (3a, 3b) as the primary nameserver (3a, 3b) of the domain owner; • Automated extraction of all domains and their associated information on the primary nameserver (3a, 3b) using the computer unit (1) set up for this purpose, whereby the extracted domains and their associated information are stored in a list (4); • Starting an intelligent self-learning algorithm (7) on the computing unit (1), which groups the extracted domains from the list (4) into a number of domain clusters (8a, 8b) based on the associated information, wherein the intelligent algorithm (7) assigns a subset of the number of domain clusters (8a, 8b) to the domain owner and stores them as owner domains, wherein the information associated with the domains is used to calculate a probability that two different domains belong to a common owner, wherein the two domains are grouped into a domain cluster (8a, 8b) if the probability is above a predefined threshold, wherein the domain owner's domain cluster is transferred to another algorithm, which performs security-relevant tests on the domains of the domain cluster and, if necessary, also automatically makes security-relevant changes to the corresponding domains,which raise this to a better level of security.

Description

[0001] The number of domains on the internet continues to grow steadily. This can make it a significant challenge for individual domain owners or companies to monitor and maintain a list of all their domains. However, knowing their own domains is crucial for businesses for a variety of reasons. For example, if content needs to be updated, it's essential that this update is applied to all domains. Legal and technical aspects of individual domains can also be important for businesses.

[0002] While such a list of one's own domains can theoretically be created and updated manually, this would represent an extremely high administrative burden, considering that companies like Deutsche Telekom have more than 10,000 different domains. If such a list doesn't exist, a company's domains can, in principle, be extracted manually from the internet. However, such a manual search, for example by an administrator, is very time-consuming and, due to the sheer number of domains, also very prone to errors.

[0003] "SHI, Yong; CHEN, Gong; LI, Juntao. Malicious domain name detection based on extreme machine learning. Neural Processing Letters, 2018, Vol. 48, No. 3, pp. 1347-1357. - DOI: 10.1007 / s11063-017-9666-7" describes a machine learning method for detecting malicious domain names, in which an Extreme Learning Machine (ELM) classifies domain names as "benign" versus "malicious" based on features extracted from multiple information sources. In their experiments, the authors report high detection performance (including >95% accuracy / detection rate).

[0004] “XIE, Xin, et al. Co-Clustering Host-Domain Graphs to Discover Malware Infection. In: Proceedings of the 2019 International Conference on Artificial Intelligence and Advanced Manufacturing. 2019. pp. 1-6. - DOI: 10.1145 / 3358331.3358380” propose a (semi-supervised) co-clustering method that creates a host-domain graph from active DNS data and, using partial domain labels (blacklist, popular domain lists, Alexa ranking) without fully labeled training data, simultaneously identifies potentially malicious domains and malware-infected hosts in the monitored network; in experiments with hundreds of hosts and 145 malware domains, up to 97.2% true positives for malware domain detection are reported. To explain the terms:

[0005] A subdomain is a domain that lies below another in the hierarchy. In common usage, this usually refers to domains at the third or higher level. A domain that lies directly below the top-level domain is colloquially referred to not as a subdomain, but as a second-level domain or simply as a domain.

[0006] Let's consider the top-level domains .com and .org as an example. These domains contain the second-level domains example.com and example.org, respectively. Each subdomain is an independent domain in its own right, which can contain other domains and hostnames. For example, example.org is a second-level domain of the top-level domain .org, and de.example.org is a subdomain of example.org.

[0007] To logically and physically separate services within an organization's domain, subdomains are traditionally used, e.g., www.example.at for the web server or mail.example.at for the mail server. However, this is merely a convention; a web server could just as easily be run on a computer with the domain name web.example.at or example.at.

[0008] Conventional names for different services: Web server: www. Web server specifically for mobile devices: m. Mail server: mail. / smtp. / pop3. / imap. FTP server: ftp.

[0009] A name server is a server that provides name resolution. Name resolution is the process that allows the names of computers or services to be resolved into a computer-readable address (e.g., www.wikipedia.org to 91.198.174.192).

[0010] Most nameservers are part of the domain system, which is also used on the Internet.

[0011] Nameservers are programs that answer queries about the domain namespace based on a DNS database; however, in common usage, the computers on which these programs run are also referred to as nameservers. A distinction is made between authoritative and non-authoritative nameservers.

[0012] A subdomain, like any domain, can be managed on any nameserver. For example, de.example.com could be managed on a nameserver in Germany, while example.com is located in the USA.

[0013] For large companies like Telekom, the effort required to manage their second-level domains, such as telekom.de, may still be acceptable, but this effort increases disproportionately for subdomains at the third or further level.

[0014] The object of the invention is therefore to specify a method, a computing unit and / or a computer program product that makes the search for domains of a specific domain owner more efficient.

[0015] This task is solved using the characteristics of independent claims.

[0016] According to the invention, the method for automatically finding and assigning domains of a domain owner comprises the following steps: - A known domain of the domain owner is maintained. For example, it is known that the domain "de.wikipedia.org" also belongs to the domain owner "Wikipedia". It is also possible to maintain several known domains of the domain owner. - In the next step, a nameserver is identified that manages this known domain for name resolution. Multiple nameservers that manage the known domain(s) for name resolution can also be identified. The nameserver identification is performed by a dedicated computer unit connected to the internet, which initiates an internet query for the server managing the known domain. For example, such a query can be executed using the "who.is" command. The selection of which nameserver is queried first can be randomized. A nameserver located geographically close to the domain owner is preferred. The identified nameserver is then designated as the primary nameserver, which is also preferably used to manage all other domains for the company. This designation is based on the understanding that companies typically prefer to use a single nameserver for all their domains. - Using the computer unit set up for this purpose, the domains managed on the primary nameserver, including the information associated with the domains, are automatically extracted from an external service provided by the primary nameserver and stored in a list. - An intelligent, self-learning algorithm is started on the computer unit, which groups the extracted domains from the list into a number of domain clusters based on the associated information. The intelligent algorithm then assigns a subset of these domain clusters to the domain owner and stores them as owner domains. The intelligent, self-learning algorithm can use the information associated with the known domains as a basis for determining which domains should be assigned to the domain owner.

[0017] This method thus advantageously enables the automated retrieval of a domain owner's domains, significantly reducing manual effort. It is essentially a "brute-force" method for identifying as many domains as possible belonging to a domain owner, even from outside the company. Initial trials of the method according to the invention have already shown that, with significantly reduced time expenditure, 75% more domains can be successfully identified compared to the previous manual approach. The self-learning algorithm can continuously improve, thus constantly increasing the success rate of identified domains. Furthermore, the method allows companies to commission service providers to locate domains, since the search, as described above, is conducted "externally" and only a short list of the domains already registered by the owner is required.Preferably, companies can provide the intelligent algorithm with further information regarding their domains, which the algorithm can then consider when creating domain clusters. This information can also include exclusion criteria, such that a domain found with a specific exclusion criterion is definitely not a domain belonging to the domain owner.

[0018] Preferably, the information associated with the domains is used to calculate a probability that two different domains belong to a common owner, with the two domains being grouped into a domain cluster if the probability exceeds an adjustable predefined threshold.

[0019] This has the advantage that, for example, an administrator can manually review the identified domain clusters and assess the accuracy of the matching process, meaning that the domains are correctly assigned to the domain owner. The administrator can adjust the customizable threshold and check whether this improves the result. Lowering the threshold will result in more domains being assigned to the domain owner, but this will also include several incorrectly assigned domains. Conversely, raising the threshold will result in fewer domains being assigned to the domain owner, but these will be assigned with a higher probability of being correctly assigned. Depending on the company's needs, the threshold can therefore be adjusted to achieve an optimal result.

[0020] Preferably, different domain clusters are merged into a single, shared domain cluster. This can be achieved, for example, through input from an administrator or by informing the algorithm that domains which, in themselves, have no common information, nevertheless belong together. This is particularly advantageous for companies with diverse business areas that sometimes have no overlap. Otherwise, the domains assigned to the different business areas would not be uniformly assigned to that company. For example, if a company manufactures mobile phones and lawnmowers, the self-learning algorithm can be instructed that the domains containing information about lawnmowers and mobile phones, respectively, belong together.The combined domain cluster therefore has a higher number of domains belonging to the domain owner than any single domain cluster alone.

[0021] The intelligent algorithm is preferably provided with information from the domain owner via an input device, such as a keyboard or touchpad, connected to the computer unit. This has the advantage that an administrator can always provide new information to the intelligent algorithm, thereby improving and streamlining the process.

[0022] In one implementation of the process, the intelligent self-learning algorithm takes into account feedback on domains correctly listed in a cluster. This feedback can be provided to the algorithm, for example, by an administrator via input devices on the computer unit, and serves to adjust the algorithm's parameters accordingly, thereby improving it.

[0023] Furthermore, the computing unit can extract domains from additional sources. This has the advantage of also identifying domains belonging to the domain owner that are not managed on name servers. In this situation, it is particularly possible for the domain owner's domains to be identified and listed multiple times. In such cases, it is advisable for the self-learning algorithm to include a `Unique(Domain-List)` command to which the identified domain list is passed. This `Unique` command incorporates a filter function, ensuring that only one identical domain is retained in the case of multiple instances of the same domain.

[0024] In a preferred implementation of the process, the computer unit automatically performs security tests on the owner's domains. This has the advantage that a domain owner can react very quickly and efficiently to new threats from the internet. New security vulnerabilities are regularly discovered, and new security-relevant threats emerge on the internet that also affect the security of internet domains. Until now, it was not possible for domain owners to react quickly and efficiently to these threats if they could not provide an up-to-date list of their own domains. This process now advantageously enables the domains to be automatically and quickly identified and, in a subsequent automated step, subjected to security tests or automatically upgraded to a new security level.

[0025] According to a second aspect of the invention, a computing unit is provided which includes means for carrying out the steps of the method described above and is suitably configured for this purpose. This advantageously enables a fast and efficient execution of the method.

[0026] According to a third aspect of the invention, a computer program product is provided which includes instructions that, when the program is executed by a computer, cause it to perform the method described above. This has the advantage that the computer program product can be flexibly installed on suitable computing units and executes the method according to the invention.

[0027] Preferred embodiments of the present invention are explained below with reference to the accompanying figures: Fig. Figure 1 shows a computer unit that determines domains from name servers. Fig. 2 shows the view from the computer unit Fig. 1 identified domains in a list view. Fig. Figure 3 shows how a self-learning algorithm can be derived from the domains of the list representation according to Fig. 2 domain clusters created.

[0028] Numerous features of the present invention are explained in detail below with reference to preferred embodiments. The present disclosure is not limited to the specific combinations of features mentioned. Rather, the features mentioned here can be combined arbitrarily to form embodiments according to the invention, unless expressly excluded below.

[0029] Fig. Figure 1 shows a computer unit 1, which is connected via the internet 2 to two different name servers 3a and 3b and can query information from them. Computer unit 1 is first provided with a known domain belonging to a domain owner, for example, "xy1.de". Computer unit 1 then establishes a connection to one of the data sources 9a or 9b and queries which name server is responsible for the domain. One or more of the data sources 9a and 9b return the responsible name servers 3a and 3b. Computer unit 1 is configured to query the data sources for all domains managed by the returned name servers 3a and 3b.

[0030] In Fig. Figure 2 shows a possible configuration of this list 4. The number of rows in list 4 corresponds to the number of domains managed on nameserver 3a. List 4 has two columns. The first column is the domain column 5 and accordingly contains the name of a domain managed on nameserver 3a. The second column is the information column 6 and contains all possible information that can be determined about the domain from the corresponding domain column 5. This information can also be referred to as domain indicators. For example, based on the Fig. 2. To recognize that the domain “xy1.de” has no common indicators with the domain “xy2.de”, one common indicator with the domain “xy3.de”, two common indicators with the domain “xy4.de” and three common indicators with the domain “xy5.de”.

[0031] This list 4 is given as input to a self-learning intelligent algorithm 7 according to Fig. 3. The self-learning intelligent algorithm 7, implemented on computing unit 1, creates 4 domain clusters from the list as shown in Fig. Figure 3 shows that the self-learning intelligent algorithm 7 implements an adjustable, predefined threshold that determines the probability at which the domains from list 4 are grouped into a common domain cluster. In this example, the threshold is defined such that domains are grouped into a common domain cluster if two or more matching pieces of information or indicators are found. Accordingly, the intelligent self-learning algorithm 7 assigns the domains "xy1.de", "xy4.de", and "xy5.de" to the domain owner's domain cluster 8a and the remaining domains "xy2.de", "xy3.de", and "xy6.de" to the further domain cluster 8b.

[0032] An administrator can then check to what extent the domain owner's domain cluster 8a has been correctly assembled by the self-learning algorithm 7 and provide corresponding feedback to the self-learning algorithm 7. Alternatively, the administrator can provide the self-learning algorithm 7 with further indicators that will help it assemble the domain cluster 8a more correctly.

[0033] The domain cluster 8a of the domain owner can be automatically transferred by computer unit 1 to another algorithm, which performs security-relevant tests on the domains of the domain cluster and, if necessary, also automatically makes security-relevant changes to the corresponding domains, raising them to a better security level.

[0034] The method according to the invention thus makes it possible to automatically check and test a large number of domains belonging to a domain owner in a very time-efficient and effective manner.

Claims

[1] A method for automatically locating and assigning domains belonging to a domain owner comprises the following steps: • Maintaining a known domain of the domain owner; • Determining a nameserver (3a, 3b) that manages the domain by passing the known domain to a computer unit (1) set up to determine the nameserver (3a, 3b), which is connected to the Internet (2) and starting an Internet query for the nameserver by the computer unit (1); • Designating the identified nameserver (3a, 3b) as the primary nameserver (3a, 3b) of the domain owner; • Automated extraction of all domains and their associated information on the primary nameserver (3a, 3b) using the computer unit (1) set up for this purpose, whereby the extracted domains and their associated information are stored in a list (4); • Starting an intelligent self-learning algorithm (7) on the computing unit (1), which groups the extracted domains from the list (4) into a number of domain clusters (8a, 8b) based on the associated information, wherein the intelligent algorithm (7) assigns a subset of the number of domain clusters (8a, 8b) to the domain owner and stores them as owner domains, wherein the information associated with the domains is used to calculate a probability that two different domains belong to a common owner, wherein the two domains are grouped into a domain cluster (8a, 8b) if the probability is above a predefined threshold, wherein the domain owner's domain cluster is transferred to another algorithm, which performs security-relevant tests on the domains of the domain cluster and, if necessary, also automatically makes security-relevant changes to the corresponding domains,which raise this to a better level of security. [2] Method according to claim 1, characterized by , that the predefined threshold is adjusted variably. [3] Method according to any one of claims 1 to 2, characterized by , that different domain clusters (8a, 8b) are combined into one domain cluster (8a, 8b). [4] Method according to any one of the preceding claims, characterized by , that the intelligent algorithm (7) is made aware of information from the domain owner via an input device of the computer unit (1). [5] Method according to any one of the preceding claims, characterized by , that the intelligent self-learning algorithm (7) takes into account feedback about domains correctly listed in a cluster. [6] Method according to any one of the preceding claims, characterized by , that the computing unit (1) extracts domains from other data sources (9a, 9b). [7] Computing unit (1) comprising means for carrying out the steps of the method according to any one of claims 1-6. [8] Computer program product comprising instructions which, when the program is executed by a computer, cause it to perform the steps of the method according to any one of claims 1 to 6.

Images