A domain name resolution log filtering method, device, equipment and storage medium

Abstract

The application provides a domain name resolution log filtering method and device, equipment and a storage medium, and relates to the technical field of information security. The method comprises the following steps: obtaining a first resolution table; determining a plurality of whitelist master domain names according to a plurality of domain names in the first resolution table, and determining a first domain name table according to the plurality of whitelist master domain names; determining a plurality of content distribution network addresses according to a plurality of internet protocol addresses in the first resolution table, and determining a first address table according to the plurality of content distribution network addresses; filtering resolution records corresponding to a plurality of master domain names, invalid domain names or generic domain names according to the first domain name table, and filtering resolution records corresponding to a plurality of cluster addresses according to the first address table. The method of the application solves the problem that the huge amount of data of the domain name resolution log causes the existing computing power to be unable to meet the data processing demand, and further causes the malicious domain name to be unable to be accurately identified from the resolution domain name.

Description

Technical Field

[0001] This application relates to the field of information security, and in particular to a method, apparatus, device, and storage medium for filtering domain name resolution logs. Background Technology

[0002] Domain name resolution logs are the logs generated when a domain name (DN) is resolved into an Internet Protocol (IP) address. Data modeling and analysis of these logs can help identify malicious domains from among the multiple domains resolved.

[0003] With the rapid development of the internet, the number of real-time domain name resolution logs has also increased rapidly. Currently, the daily volume of domain name resolution logs is approaching the petabyte (PB) level. This massive volume of data makes it impossible for existing computing power to meet the data processing demands, thus hindering the accurate identification of malicious domains from among the multiple domains being resolved. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for filtering domain name resolution logs, in order to solve the problem that the large amount of data in domain name resolution logs leads to the inability of existing computing power to meet data processing needs, and consequently, the inability to accurately identify malicious domain names from multiple resolved domain names.

[0005] Firstly, this application provides a method for filtering domain name resolution logs, the method comprising:

[0006] Obtain the first DNS table, which includes multiple DNS records. Each DNS record includes a domain name and the corresponding Internet Protocol address.

[0007] Multiple whitelisted main domains are determined based on multiple domains in the first resolution table, and the first domain table is determined based on the multiple whitelisted main domains;

[0008] Multiple content delivery network addresses are determined based on multiple Internet Protocol addresses in the first resolution table, and a first address table is determined based on the multiple content delivery network addresses;

[0009] The first domain name table filters the DNS records corresponding to multiple main domain names, invalid domain names, or wildcard domain names, and the first address table filters the DNS records corresponding to multiple clustered addresses.

[0010] In one possible design, the DNS records corresponding to multiple primary domains, invalid domains, or wildcard domains are filtered based on the first domain table, including:

[0011] Based on the first DNS table, multiple primary domains, invalid domains, and wildcard domains were identified.

[0012] When the first domain is in the first domain table, filter the DNS records corresponding to the first domain. The first domain can be any one of multiple main domains.

[0013] Filter the DNS records corresponding to the second domain name. The second domain name is any one of multiple invalid domain names. Invalid domain names refer to domain names on the local area network and domain names that failed to resolve.

[0014] When multiple third-party domains correspond to the same primary domain, filter the DNS records corresponding to the third-party domains. The third-party domain can be any one of the multiple wildcard domains.

[0015] In one possible design, filtering the DNS records corresponding to third-party domains includes:

[0016] The fourth domain is determined based on the third domain, and the fourth domain is the main domain corresponding to the third domain.

[0017] When the fourth domain is listed in the first domain table, filter the DNS records corresponding to the third domain.

[0018] When the fourth domain is not in the first domain table, the third domain is replaced with the fourth domain in the DNS record corresponding to the third domain.

[0019] In one possible design, the parsing records corresponding to multiple clustered addresses are filtered according to the first address table, including:

[0020] Multiple cluster addresses are determined based on the first resolution table, and each cluster address corresponds to multiple domain names;

[0021] When the first address is in the first address table, filter the parsing record corresponding to the first address. The first address is any one of multiple cluster addresses.

[0022] In one possible design, obtaining the first parsing table includes:

[0023] Upon receiving the first instruction information, a second instruction information is sent to the log collection device so that the log collection device can collect the full domain name resolution logs for a preset time period.

[0024] Based on the domain name and Internet Protocol address in each domain name resolution log, multiple resolution records are determined;

[0025] The first parsing table is determined based on multiple parsing records.

[0026] In one possible design, the resolution record also includes: the number of domain name resolution logs corresponding to the domain name;

[0027] Multiple whitelisted primary domains are determined based on multiple domains in the first DNS table, and the first domain table is determined based on these multiple whitelisted primary domains, including:

[0028] Multiple primary domains are determined based on the multiple domains in the first resolution table, and the multiple primary domains are sorted in descending order according to the number of domain name resolution logs corresponding to each primary domain to obtain the sorting result;

[0029] Extract a preset number of whitelisted main domains in order of sorting results, and determine the second domain table based on the preset number of whitelisted main domains;

[0030] The second domain name table is sent to the address information database so that the address information database can update the pre-stored whitelist table with a preset number of whitelisted main domain names, thus obtaining the first domain name table.

[0031] In one possible design, multiple content delivery network addresses are determined based on multiple Internet Protocol addresses in a first resolution table, and a first address table is determined based on the multiple content delivery network addresses, including:

[0032] Multiple cluster addresses are determined based on the first resolution table, and each cluster address corresponds to multiple domain names;

[0033] Determine the number of mappings and the threshold number for the second address, where the second address is any one of multiple clustering addresses;

[0034] When the number of mappings exceeds the threshold, the second address is determined as the content delivery network address;

[0035] The second address table is determined based on multiple content delivery network addresses and multiple domain names corresponding to each content delivery network address;

[0036] The second address table is sent to the address information database so that the address information database can update the pre-stored content delivery network address table with multiple content delivery network addresses and multiple domain names corresponding to each content delivery network address, thus obtaining the first address table.

[0037] In one possible design, the multiple domain names corresponding to the second address include a first number of fourth domain names and a second number of fifth domain names, wherein each fourth domain name corresponds to one domain name resolution log and each fifth domain name corresponds to multiple domain name resolution logs;

[0038] Determine the number of mappings and the threshold number for the second address, including:

[0039] The number of times the second address is mapped is determined based on the first quantity and the second quantity, and the number of mappings is equal to the sum of the first quantity and the second quantity;

[0040] The threshold number R for the second address is determined using the following formula:

[0041]

[0042] Where A is the median of the number of DNS resolution logs corresponding to the second number of fifth domains, B is the average of the number of DNS resolution logs corresponding to the second number of fifth domains, m is the first number, n is the second number, k is the preset adjustment coefficient, and h is the second number. i Let h be the number of DNS resolution logs corresponding to the i-th fifth domain name, where A, B, m, and n are positive integers, i is a positive integer not greater than n, k is a positive number less than 1, and h is a positive number. i It is a positive integer greater than 1.

[0043] In one possible design, the DNS records corresponding to multiple primary domains, invalid domains, or wildcard domains are filtered according to the first domain name table, and the DNS records corresponding to multiple clustered addresses are filtered according to the first address table, including:

[0044] The second resolution table is obtained by filtering multiple resolution records in the first resolution table based on the first domain name table;

[0045] Multiple resolution records in the second resolution table are filtered based on the first address table.

[0046] In one possible design, filtering DNS records corresponding to multiple primary domains, invalid domains, or wildcard domains based on the first domain name table, and filtering DNS records corresponding to multiple clustered addresses based on the first address table, further includes:

[0047] The third resolution table is obtained by filtering multiple resolution records in the first resolution table based on the first address table;

[0048] Filter multiple DNS records in the third DNS table based on the first domain name table.

[0049] In one possible design, filtering DNS records corresponding to multiple primary domains, invalid domains, or wildcard domains based on the first domain name table, and filtering DNS records corresponding to multiple clustered addresses based on the first address table, further includes:

[0050] The second resolution table is obtained by filtering multiple resolution records in the first resolution table based on the first domain name table;

[0051] The third resolution table is obtained by filtering multiple resolution records in the first resolution table based on the first address table;

[0052] The fourth parsing table is determined based on the second and third parsing tables. The fourth parsing table includes multiple parsing records that are common to the second and third parsing tables.

[0053] Secondly, this application provides a data processing apparatus, comprising:

[0054] The acquisition module is used to acquire the first resolution table, which includes multiple resolution records. Each resolution record includes a domain name and the corresponding Internet Protocol address.

[0055] The determination module is used to determine multiple whitelisted main domains based on multiple domains in the first resolution table, and to determine the first domain table based on the multiple whitelisted main domains;

[0056] The determining module is further configured to determine multiple content delivery network addresses based on multiple Internet Protocol addresses in the first parsing table, and to determine the first address table based on the multiple content delivery network addresses;

[0057] The filtering module is used to filter the DNS records corresponding to multiple main domains, invalid domains or wildcard domains according to the first domain name table, and to filter the DNS records corresponding to multiple clustered addresses according to the first address table.

[0058] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0059] The memory stores instructions that the computer executes;

[0060] The processor executes computer execution instructions stored in memory to implement a domain name resolution log filtering method according to the first aspect of the invention.

[0061] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement a domain name resolution log filtering method according to the first aspect of the invention.

[0062] Fifthly, this application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement a method for filtering domain name resolution logs as described in the first aspect of the invention.

[0063] This application provides a method, apparatus, device, and storage medium for filtering domain name resolution logs. It obtains a first resolution table, determines multiple whitelisted primary domains based on multiple domains in the first resolution table, determines a first domain name table based on the multiple whitelisted primary domains, determines multiple content delivery network addresses based on multiple Internet Protocol (IP) addresses in the first resolution table, determines a first address table based on the multiple IP addresses, filters resolution records corresponding to multiple primary domains, invalid domains, or wildcard domains based on the first domain name table, and filters resolution records corresponding to multiple clustered addresses based on the first address table. This achieves the following technical effects: filtering multiple resolution records corresponding to primary domains, invalid domains, and wildcard domains based on the first domain name table, and filtering multiple resolution records corresponding to content delivery network addresses based on the first address table, reduces the number of domain name resolution logs and improves the accuracy and efficiency of malicious domain identification; determining the first domain name table based on whitelisted primary domains and the first address table based on content delivery network addresses solves the problem of lacking filtering criteria based on resolution records; resolving domain name resolution logs into resolution records and then filtering these records improves data processing efficiency and solves the problem of low data processing efficiency when directly filtering domain name resolution logs. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 A schematic diagram of the system architecture for the domain name resolution log filtering method provided in this application embodiment;

[0066] Figure 2 A flowchart illustrating the domain name resolution log filtering method provided in this application embodiment. Figure 1 ;

[0067] Figure 3 A flowchart illustrating the domain name resolution log filtering method provided in this application embodiment. Figure 2 ;

[0068] Figure 4 A flowchart illustrating the determination of the first parsing table provided in an embodiment of this application;

[0069] Figure 5 A flowchart illustrating the process of determining the first domain name table provided in an embodiment of this application;

[0070] Figure 6 A flowchart illustrating the process of determining the first address table provided in an embodiment of this application;

[0071] Figure 7 A flowchart illustrating the execution order of single filtering and global filtering provided in the embodiments of this application. Figure 1 ;

[0072] Figure 8 A flowchart illustrating the execution order of single filtering and global filtering provided in the embodiments of this application. Figure 2 ;

[0073] Figure 9 A flowchart illustrating the execution order of single filtering and global filtering provided in the embodiments of this application. Figure 3 ;

[0074] Figure 10 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application;

[0075] Figure 11 This is a schematic diagram of the structure of the electronic device hardware provided in the embodiments of this application.

[0076] Figure label:

[0077] 110 - Root Server; 120 - Log Collection Module; 130 - Data Processing Module; 140 - Address Information Database; 150 - Scheduling Module; 131 - Acquisition Module; 132 - Determination Module; 133 - Filtering Module;

[0078] 200 - Electronic device; 210 - Processor; 220 - Memory; 230 - Communication component; 240 - Bus. Detailed Implementation

[0079] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0080] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. For example, the first domain name and the second domain name are merely to distinguish domain names with different IP addresses and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" do not necessarily imply that they are different. It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that they are examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. In the embodiments of this application, "at least one" means one or more, and "more than one" means two or more.

[0081] It should be noted that the phrase "at...time" in the embodiments of this application can refer to the instant at which a certain situation occurs, or to a period of time after the occurrence of a certain situation; the embodiments of this application do not specifically limit this. Furthermore, the domain name resolution log filtering method provided in the embodiments of this application is merely an example; the domain name resolution log filtering method may also include more or less content.

[0082] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0083] Malicious domains refer to domains containing malicious links. A common tactic used by malicious domains is to lure users to unsafe websites, where malicious code can then damage the user's computer network, leak their private information, and sensitive data.

[0084] Content Delivery Network (CDN) address: This refers to the IP address of the node server that provides network services. Each node server corresponds to a domain name, and multiple node servers correspond to one IP address.

[0085] Clustered addresses: These are addresses that are grouped together based on their similarity. One clustered address can correspond to multiple domain names.

[0086] Domain name resolution log filtering is a series of filters implemented to cope with the massive amount of domain name resolution logs, based on different business scenarios and professional application needs, in order to reduce the number of domain name resolution logs and improve the efficiency of identifying malicious domains.

[0087] In existing technologies, the pain point of identifying massive malicious domain names at the carrier level is the huge amount of data generated in real time. The amount of domain name resolution logs generated daily is close to the petabyte level. Due to the huge amount of data, it is impossible to perform data modeling and analysis of domain name resolution logs.

[0088] Based on this, embodiments of this application provide a method, apparatus, device, and storage medium for filtering domain name resolution logs, which can be used in the field of information security technology. The aim is to solve the problem that the large amount of data in the domain name resolution logs of the prior art leads to the inability of existing computing power to meet the data processing needs, and thus makes it impossible to accurately identify malicious domain names from multiple resolved domain names.

[0089] Figure 1 This is a schematic diagram of the system architecture for the domain name resolution log filtering method provided in this application embodiment. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this application, in order to help those skilled in the art understand the technical content of this application, but do not mean that the embodiments of this application cannot be used in other devices, systems, environments or scenarios.

[0090] like Figure 1 As shown, the system architecture of the domain name resolution log filtering method includes: a root server 110, a log collection module 120, a data processing module 130, an address information database 140, and a scheduling module 150. The data processing module 130 is communicatively connected to the log collection module 120, the address information database 140, and the scheduling module 150. The log collection module 120 is communicatively connected to the root server 110. These communication connections can include various connection types, such as wired or wireless communication links or fiber optic cables.

[0091] In this embodiment, the root server 110 can be a server that provides domain name resolution services. The root server 110 can receive domain name resolution requests sent by user terminals, determine the domain name to be resolved according to the domain name resolution request, and resolve the domain name to obtain the IP address corresponding to the domain name; the root server 110 can also generate domain name resolution logs according to the domain name to be resolved and the IP address corresponding to the domain name; the root server 110 can also store the domain name resolution logs in a cloud server or its own storage.

[0092] The log collection module 120 can be software or hardware for collecting data. The log collection module 120 can collect domain name resolution logs from the root server 110 and send the collected domain name resolution logs to the data processing module 130; the log collection module 120 can collect domain name resolution logs actively at regular intervals or passively when it receives an instruction.

[0093] The data processing module 130 can be software or hardware for filtering domain name resolution logs. The data processing module 130 can generate resolution records based on the domain name resolution logs and generate a resolution table based on multiple resolution records; the data processing module 130 can also filter multiple resolution records to reduce the number of domain name resolution logs.

[0094] Address information database 140 can be software or hardware that stores and updates domain name tables and address tables. Address information database 140 can store domain name tables and address tables sent by data processing module 130; address information database 140 can also update a pre-stored whitelist table according to the domain name table; address information database 140 can also update a pre-stored CDN address table according to the address table.

[0095] The scheduling module 150 can be software or hardware responsible for overall process scheduling. The scheduling module 150 can send a first instruction to the data processing module 130, so that the data processing module 130 can send a second instruction to the log collection module 120; the scheduling module 150 can also obtain the parsed table filtered by the data processing module 130.

[0096] Figure 2 A flowchart illustrating the domain name resolution log filtering method provided in this application embodiment. Figure 1 .

[0097] like Figure 2 As shown, the method includes:

[0098] S101. Obtain the first resolution table. The first resolution table includes multiple resolution records. Each resolution record includes: the domain name and the Internet Protocol address corresponding to the domain name.

[0099] Specifically, the collected domain name resolution logs are parsed to obtain the first resolution table. The specific process is as follows: each domain name resolution log includes a domain name and an IP address corresponding to that domain name. The domain name resolution log is parsed to generate the corresponding resolution record. The resolution record includes the aforementioned domain name and the IP address corresponding to that domain name.

[0100] Instead of filtering the domain name resolution logs directly, the domain name resolution logs are first parsed into resolution records, and then the resolution records are filtered. The goal of this approach is that the first resolution table in data table format is easier to process than the log format of the domain name resolution logs, thus enabling more efficient filtering of the domain name resolution logs and improving the efficiency of malicious domain name identification.

[0101] S102. Determine multiple whitelisted main domains based on multiple domains in the first resolution table, and determine the first domain table based on the multiple whitelisted main domains;

[0102] Specifically, multiple primary domains are first identified from the full set of domains in the first DNS table, and then multiple whitelisted primary domains are selected from these primary domains. The whitelist corresponds to the blacklist; domains (or addresses, applications, information, etc.) on the whitelist are given priority and are not blocked, significantly improving their security and speed. Understandably, the security of whitelisted primary domains is guaranteed, as they are unlikely to be malicious. Therefore, filtering out the DNS records corresponding to whitelisted primary domains improves the accuracy and efficiency of malicious domain identification.

[0103] The first domain name table is determined based on the aforementioned multiple whitelisted main domain names. The purpose of the first domain name table is to provide a basis for filtering DNS records.

[0104] S103. Determine multiple content delivery network addresses based on multiple Internet Protocol addresses in the first parsing table, and determine the first address table based on the multiple content delivery network addresses;

[0105] Specifically

[0106] First, multiple identical IP addresses are identified from the full set of IP addresses in the first resolution table. Clustering is then performed to obtain multiple cluster addresses, each containing multiple distinct domain names. Next, each cluster address is used to determine its CDN address. When any cluster address is determined to be a CDN address, a first address table is established based on that cluster address. Each first address table contains multiple identical IP addresses and the corresponding domain name for each IP address. The purpose of the first address table is to provide a basis for filtering DNS records.

[0107] S104. Filter the resolution records corresponding to multiple main domains, invalid domains or wildcard domains according to the first domain name table, and filter the resolution records corresponding to multiple clustered addresses according to the first address table;

[0108] Specifically, filtering multiple DNS records based on the first domain table is also known as single filtering. Single filtering means first filtering the DNS records corresponding to whitelisted main domains among multiple main domains, then filtering the DNS records corresponding to invalid domains, and finally filtering the DNS records corresponding to wildcard domains.

[0109] Filtering multiple parsing records based on the first address table is also known as global filtering. Specifically, global filtering refers to filtering the parsing records corresponding to CDN addresses in multiple clustered addresses.

[0110] There is no requirement for the execution order of single filtering and global filtering. Single filtering can be performed first and then global filtering, or global filtering can be performed first and then single filtering, or both single filtering and global filtering can be performed simultaneously.

[0111] This embodiment provides a method for filtering domain name resolution logs. It obtains a first resolution table, determines multiple whitelisted primary domains based on multiple domains in the first resolution table, and determines a first domain name table based on the multiple whitelisted primary domains. It also determines multiple content delivery network (CDN) addresses based on multiple Internet Protocol (IP) addresses in the first resolution table, and determines a first address table based on the multiple CDN addresses. The method filters resolution records corresponding to multiple primary domains, invalid domains, or wildcard domains based on the first domain name table, and filters resolution records corresponding to multiple clustered addresses based on the first address table. This achieves the following technical effects: filtering multiple resolution records corresponding to primary domains, invalid domains, and wildcard domains based on the first domain name table, and filtering multiple resolution records corresponding to CDN addresses based on the first address table, reduces the number of domain name resolution logs and improves the accuracy and efficiency of malicious domain identification; determining the first domain name table based on whitelisted primary domains and the first address table based on CDN addresses solves the problem of lacking filtering criteria based on resolution records; resolving domain name resolution logs into resolution records and then filtering these records improves data processing efficiency and solves the problem of low data processing efficiency when directly filtering domain name resolution logs.

[0112] Figure 3 A flowchart illustrating the domain name resolution log filtering method provided in this application embodiment. Figure 2 .like Figure 3 As shown, in this embodiment... Figure 2 Based on the embodiments, the method for filtering domain name resolution logs is described in detail. This embodiment provides a method for filtering domain name resolution logs, including:

[0113] S201. Obtain the first resolution table. The first resolution table includes multiple resolution records. Each resolution record includes: the domain name and the Internet Protocol address corresponding to the domain name.

[0114] S202. Determine multiple whitelisted main domains based on multiple domains in the first DNS table, and determine the first domain table based on the multiple whitelisted main domains;

[0115] S203. Determine multiple content delivery network addresses based on multiple Internet Protocol addresses in the first parsing table, and determine the first address table based on the multiple content delivery network addresses;

[0116] The contents of S201-S203 are similar to those of S101-S103, and will not be described again in this embodiment.

[0117] S204. Based on the first DNS table, identify multiple primary domains, invalid domains, and wildcard domains;

[0118] Specifically, first, the primary domain is determined from multiple domains, with the primary domain format being a first-level domain; second, invalid domains are determined from multiple domains, referring to domains on the local area network and domains that failed to resolve; finally, wildcard domains are determined from multiple domains, with wildcard domains formatting as second-level domains and higher. For example, Table 1 shows the DNS records corresponding to the determined primary domains, invalid domains, and wildcard domains; Table 1:

[0119] 11 111.com 111.111.111.111 12 112.com 112.112.112.112 13 211.com 192.168.0.1 14 213.com 0.0.0.0 15 aaa.311.com 311.311.311.311 16 bbb.311.com 311.311.311.311 17 cc.ddd.312.com 312.312.312.312 18 ee.fff.312.com 312.312.312.312 19 g.hh.iii.313.com 313.313.313.313

[0120] As shown in Table 1, records 11 and 12, with domain names in the format of first-level domains, indicate that the domains are primary domains. Record 13, with an IP address range belonging to the local area network (LAN), indicates that the domain is a LAN domain, i.e., an invalid domain. Record 14, with an IP address indicating domain name resolution failure, indicates that the domain is a domain with resolution failure, i.e., an invalid domain. Records 15 and 16, with domain names in the format of second-level domains, records 17 and 18, with domain names in the format of third-level domains, and record 19, with a domain name in the format of a fourth-level domain, indicate that the domains are wildcard domains.

[0121] S205. When the first domain is in the first domain table, filter the DNS records corresponding to the first domain. The first domain can be any one of multiple main domains.

[0122] Specifically, it determines whether the first domain is in the first domain table; if the first domain is in the first domain table, the corresponding DNS records are filtered; if the first domain is not in the first domain table, the corresponding DNS records are not filtered, as the first domain may or may not be a malicious domain. For example, Table 2 is the first domain table; Table 2:

[0123] 111.com 222.com 311.com

[0124] As shown in Table 1, the domain names of DNS records with serial numbers 11 and 12 are primary domain names. As shown in Table 2, the domain name of the DNS record with serial number 11 is in Table 2, so the DNS record with serial number 11 is filtered; the domain name of the DNS record with serial number 12 is not in Table 2, so the DNS record with serial number 12 is not filtered.

[0125] S206. Filter the DNS records corresponding to the second domain name, where the second domain name is any one of multiple invalid domain names;

[0126] Specifically, filter each second domain name. For example, according to Table 1, the DNS records with serial numbers 13 and 14 are invalid domain names, so filter the DNS records with serial numbers 13 and 14.

[0127] S207. When multiple third domains correspond to the same primary domain, the fourth domain is determined based on the third domain. The fourth domain is the primary domain corresponding to the third domain, and the third domain is any one of the multiple wildcard domains.

[0128] Specifically, it determines whether the main domains corresponding to multiple third-party domains are the same; when the main domains corresponding to multiple third-party domains are different, the multiple third-party domains are not filtered, as the third-party domains may be malicious or not; when the main domains corresponding to multiple third-party domains are the same, the fourth domain is determined.

[0129] For example, according to Table 1, DNS records with serial numbers 15, 16, 17, 18, and 19 have wildcard domains. Specifically, DNS records with serial numbers 15 and 16 have the same primary domain, 311.com; DNS records with serial numbers 17 and 18 have the same primary domain, 312.com; and DNS record with serial number 19 has a primary domain that is different from all other primary domains, so DNS record 19 is not filtered.

[0130] S208. When the fourth domain name is in the first domain name table, filter the DNS records corresponding to the third domain name;

[0131] S209. When the fourth domain name is not in the first domain name table, replace the third domain name with the fourth domain name in the DNS record corresponding to the third domain name;

[0132] Specifically, it determines whether the fourth domain is in the first domain table; if the fourth domain is in the first domain table, it filters the corresponding DNS records; if the fourth domain is not in the first domain table, it does not filter the corresponding DNS records, and the fourth domain may be a malicious domain or not.

[0133] For example, according to Table 2, the fourth domain name for 311.com is in Table 2, so the DNS records with sequence numbers 15 and 16 corresponding to this fourth domain name are filtered out; the fourth domain name for 312.com is not in Table 2, so the DNS records with sequence numbers 17 and 18 are replaced with 312.com by replacing the domain name with either cc.ddd.312.com or ee.fff.312.com. At the same time, only one DNS record is retained in the two records.

[0134] A single filter is applied to the parsed records in Table 1 to obtain Table 3, which contains the parsed records after the single filter. Table 3:

[0135] 11 112.com 112.112.112.112 12 312.com 312.312.312.312 13 g.hh.iii.313.com 313.313.313.313

[0136] S210. Determine multiple cluster addresses based on the first resolution table, and each cluster address corresponds to multiple domain names;

[0137] Specifically, firstly, clustering is performed on the same IP addresses from multiple IP addresses to determine multiple cluster addresses, each of which includes multiple different domain names.

[0138] For example, Table 4 shows the resolution records corresponding to multiple cluster addresses; Table 4:

[0139] 11 411.com 411.411.411.411 12 412.com 411.411.411.411 13 413.com 411.411.411.411 14 511.com 511.511.511.511 15 512.com 511.511.511.511 16 aa.512.com 511.511.511.511

[0140] As shown in Table 4, the DNS records with serial numbers 11, 12, and 13 have the same IP address and belong to the first cluster address; the DNS records with serial numbers 14, 15, and 16 have the same IP address and belong to the second cluster address.

[0141] S211. When the first address is located in the first address table, filter the parsing record corresponding to the first address. The first address is any one of multiple cluster addresses.

[0142] Specifically, the process involves determining whether the first address is in the first address table. If the first address is in the first address table, the DNS records corresponding to that first address are filtered. If the first address is not in the first address table, the DNS records corresponding to that first address are not filtered, as the domain name corresponding to that first address may or may not be malicious. Filtering the DNS records corresponding to CDN addresses can prevent the domain names corresponding to CDN addresses from being treated as malicious domain names, reducing the possibility of wrongly blocking domain names or generating incorrect intelligence data.

[0143] Figure 4 This is a flowchart illustrating the process of determining the first parsing table as provided in an embodiment of this application. Figure 4 As shown, in this embodiment... Figure 3 Based on the implementation examples, combined with Figure 1 The following provides a detailed explanation of obtaining the first DNS record: S201. Obtain the first DNS record, which includes multiple DNS records. Each DNS record includes: a domain name and the corresponding Internet Protocol address, including:

[0144] S301. Upon receiving the first instruction information, send the second instruction information to the log collection device so that the log collection device can collect the full domain name resolution logs for a preset time period.

[0145] Specifically, when the data processing module 130 receives the first instruction information sent by the scheduling module 150, the data processing module 130 executes the log collection process, including: sending the second instruction information to the log collection module 120 so that the log collection module 120 can collect the full domain name resolution logs for a preset time period from the root server 110.

[0146] S302. Based on the domain name and Internet Protocol address in each domain name resolution log, determine multiple resolution records;

[0147] Specifically, each domain name resolution log includes a domain name and the IP address corresponding to that domain name. A resolution record is determined based on a domain name and the IP address corresponding to that domain name.

[0148] S303. Determine the first parsing table based on multiple parsing records.

[0149] Figure 5 This is a schematic diagram illustrating the process of determining the first domain name table provided in an embodiment of this application. Figure 5 As shown, in this embodiment... Figure 3 Based on the implementation examples, combined with Figure 1 The process of determining the first domain name table is explained in detail; S202, multiple whitelisted primary domain names are determined based on multiple domain names in the first DNS table, and the first domain name table is determined based on the multiple whitelisted primary domain names, including:

[0150] S401. Determine multiple primary domains based on the multiple domains in the first DNS table, and sort the multiple primary domains in descending order according to the number of DNS resolution logs corresponding to each primary domain to obtain the sorting result;

[0151] Specifically, the resolution record also includes: the number of domain name resolution logs corresponding to the domain name; determining multiple main domain names from multiple domain names in the first resolution table, and determining the number of domain name resolution logs corresponding to each main domain name; sorting the multiple main domain names in descending order according to the number of corresponding domain name resolution logs to obtain the sorting result.

[0152] S402. Extract a preset number of whitelisted main domains according to the sorting results, and determine the second domain table according to the preset number of whitelisted main domains;

[0153] Specifically, the number of main domain names to be extracted is determined to be a preset number. The preset number can be manually set by technicians in advance, or it can be automatically generated by the data processing module 130 or scheduling module 150 or other devices / modules according to specific parameters, including but not limited to the collection period and the target root server.

[0154] Extract a predetermined number of primary domains from the DNS resolution results in a sequential order, designate these as whitelisted primary domains, and use them to determine the second domain table. Whitelisted primary domains with high access frequency are unlikely to be malicious domains.

[0155] S403. Send the second domain name table to the address information database so that the address information database can update the preset number of whitelisted main domain names to the pre-stored whitelist table and obtain the first domain name table;

[0156] Specifically, the whitelisted primary domains in the second domain name table are determined from the domain name resolution logs collected this time, and do not mean that the whitelisted primary domains cannot include other primary domains that have not been resolved. Therefore, the second domain name table is sent to the address information database 140 so that the address information database 140 can update the pre-stored whitelist table according to the preset number of whitelists in the second domain name table, thus obtaining the first domain name table. The pre-stored whitelist table includes the whitelisted primary domains resolved during previous domain name resolution log collections. The first domain name table obtained in this way is the latest and most complete whitelisted primary domain name table as of the time of this domain name resolution log collection.

[0157] Figure 6 This is a schematic flowchart illustrating the process of determining the first address table as provided in an embodiment of this application. Figure 6 As shown, in this embodiment... Figure 3 Based on the implementation examples, combined with Figure 1 Table 4 provides a detailed explanation of determining the first address table; S203, determining multiple content delivery network addresses based on multiple Internet Protocol addresses in the first resolution table, and determining the first address table based on the multiple content delivery network addresses, including:

[0158] S501. Determine multiple cluster addresses based on the first resolution table, and each cluster address corresponds to multiple domain names;

[0159] S501 is similar to S210, and will not be described again in this embodiment.

[0160] S502. Determine the number of times the second address is mapped based on the first quantity and the second quantity. The number of mappings is equal to the sum of the first quantity and the second quantity. The second address is any one of multiple clustering addresses.

[0161] Specifically, the second address corresponds to multiple domain names, including a first number of fourth domain names and a second number of fifth domain names, wherein each fourth domain name corresponds to one domain name resolution log and each fifth domain name corresponds to multiple domain name resolution logs.

[0162] As shown in Table 4, Table 4 includes two cluster addresses: 411.411.411.411 and 511.511.511.511, each corresponding to three domain names. Taking cluster address 411.411.411.411 as the second address, Table 5 is obtained based on the number of DNS resolution logs corresponding to each domain name at the second address. Table 5 shows the DNS resolution logs corresponding to the second address. Table 5:

[0163] 11 411.com 411.411.411.411 1 12 412.com 411.411.411.411 <![CDATA[h1(h1>1)]]> 13 413.com 411.411.411.411 <![CDATA[h2(h2>1)]]>

[0164] As shown in Table 5, the DNS record with serial number 11 has 1 DNS resolution log entry, and this domain is the fourth domain; the DNS record with serial number 12 has h1 DNS resolution log entry, and this domain is the fifth domain; the DNS record with serial number 13 has h2 DNS resolution log entry, and this domain is the fifth domain; therefore, the first quantity m = 1, the second quantity n = 2, and the number of mappings Q = m + n = 3.

[0165] S503. The threshold number R for the second address is determined using the following formula:

[0166]

[0167] Specifically, A is the median of the number of DNS resolution logs corresponding to the second number of fifth domains, B is the average of the number of DNS resolution logs corresponding to the second number of fifth domains, m is the first number, n is the second number, k is a preset adjustment coefficient, and h... i Let h be the number of DNS resolution logs corresponding to the i-th fifth domain name, where A, B, m, and n are positive integers, i is a positive integer not greater than n, k is a positive number less than 1, and h is a positive number. i It is a positive integer greater than 1.

[0168] First, determine the median A, where A is h. 11 h 12 and h 13 The median is obtained by rounding down the median for easier calculation.

[0169] Secondly, determine the mean B, where B is h 11 h 12 and h 13 The average is the mean, and to facilitate calculation, the mean is rounded down to obtain the mean B;

[0170] Finally, the adjustment coefficient k is determined. The adjustment coefficient k can be 0.3, 0.4 or other positive numbers less than 1. The adjustment coefficient k can be manually set in advance by technicians, or it can be automatically generated by the data processing module 130 or scheduling module 150 or other devices / modules based on specific parameters, including but not limited to the acquisition cycle and the target root server.

[0171] The steps for calculating the threshold number R are as follows:

[0172] First, determine the number of domain name resolution logs N1 corresponding to the multiple second addresses, where N1 = m + ∑h i

[0173] Secondly, determine the number of domain name resolution logs N2 corresponding to multiple fifth domain names, N2 = ∑h i

[0174] Secondly, determine the corrected median A″.

[0175]

[0176] Secondly, determine the corrected mean B″.

[0177]

[0178] Finally, determine the threshold number R, R = A″×k + B″×(1-k)

[0179] S504. When the number of mappings exceeds the threshold number, the second address is determined as the content delivery network address;

[0180] Specifically, when Q>R, it means the second address is a CDN address; when Q≤R, it means the second address is not a CDN address.

[0181] S505. Determine a second address table based on multiple content delivery network addresses and multiple domain names corresponding to each content delivery network address;

[0182] Specifically, the first resolution table can identify multiple CDN addresses, and based on these multiple CDN addresses and the multiple domain names corresponding to each CDN address, the second address table is determined.

[0183] S506. Send the second address table to the address information database so that the address information database can update the multiple content delivery network addresses and the multiple domain names corresponding to each content delivery network address to the pre-stored content delivery network address table to obtain the first address table.

[0184] Specifically, the CDN addresses in the second address table are determined from the domain name resolution logs collected this time, and this does not mean that the CDN addresses cannot include other unresolved addresses. Therefore, the second address table is sent to the address information database 140 so that the address information database 140 can update the pre-stored CDN address table based on the multiple CDN addresses in the second address table to obtain the first address table. The pre-stored CDN address table includes CDN addresses resolved during previous domain name resolution log collections. Therefore, the first address table obtained in this way is the latest and most comprehensive CDN address table up to the time of this domain name resolution log collection.

[0185] Figure 7 A flowchart illustrating the execution order of single filtering and global filtering provided in the embodiments of this application. Figure 1 .like Figure 7 As shown, in this embodiment... Figure 3 Based on the examples, the execution order of single filtering and global filtering is explained; the execution order of single filtering and global filtering includes:

[0186] S601. Filter multiple resolution records in the first resolution table based on the first domain name table to obtain the second resolution table;

[0187] S602. Filter multiple resolution records in the second resolution table according to the first address table;

[0188] Specifically, single filtering determines whether to filter DNS records based on the domain name, which requires less computing power; global filtering determines whether to filter DNS records based on the IP address and the corresponding domain name, which requires more computing power.

[0189] Depending on the business scenario, perform single-level filtering first, and then perform global filtering based on the single-level filtering. This business scenario could be one with limited available computing power, or one that requires obtaining the domain name resolution logs corresponding to malicious domain names and CDN addresses.

[0190] Figure 8 A flowchart illustrating the execution order of single filtering and global filtering provided in the embodiments of this application. Figure 2 .like Figure 8 As shown, in this embodiment... Figure 3 Based on the examples, the execution order of single filtering and global filtering is explained; the execution order of single filtering and global filtering also includes:

[0191] S701. Filter multiple resolution records of the first resolution table according to the first address table to obtain the third resolution table;

[0192] S702. Filter multiple DNS records in the third DNS table based on the first DNS table;

[0193] Specifically, depending on the business scenario, global filtering is performed first, followed by individual filtering based on the global filtering. This business scenario could be one that requires obtaining the domain name resolution logs corresponding to malicious domains, as well as the domain name resolution logs corresponding to whitelisted main domains, invalid domains, and wildcard domains.

[0194] Figure 9 A flowchart illustrating the execution order of single filtering and global filtering provided in the embodiments of this application. Figure 3 .like Figure 9 As shown, in this embodiment... Figure 3 Based on the examples, the execution order of single filtering and global filtering is explained; the execution order of single filtering and global filtering also includes:

[0195] S801. Filter multiple resolution records in the first resolution table based on the first domain name table to obtain the second resolution table;

[0196] S802. Filter multiple resolution records of the first resolution table according to the first address table to obtain the third resolution table;

[0197] S803. Determine the fourth parsing table based on the second and third parsing tables. The fourth parsing table includes multiple parsing records that are common to the second and third parsing tables.

[0198] Specifically, depending on the business scenario, both single filtering and global filtering are performed simultaneously. A second parsing table obtained from single filtering and a third parsing table obtained from global filtering are used, and a fourth parsing table is determined based on the parsing records shared by both. This business scenario can be one where high filtering accuracy is required.

[0199] This embodiment provides a method for filtering domain name resolution logs. It obtains a first resolution table, determines multiple whitelisted primary domains based on multiple domains in the first resolution table, and determines a first domain name table based on the multiple whitelisted primary domains. It also determines multiple content delivery network (CDN) addresses based on multiple Internet Protocol (IP) addresses in the first resolution table, and determines a first address table based on the multiple CDN addresses. The method filters resolution records corresponding to multiple primary domains, invalid domains, or wildcard domains based on the first domain name table, and filters resolution records corresponding to multiple clustered addresses based on the first address table. This achieves the following technical effects: filtering multiple resolution records corresponding to primary domains, invalid domains, and wildcard domains based on the first domain name table, and filtering multiple resolution records corresponding to CDN addresses based on the first address table, reduces the number of domain name resolution logs and improves the accuracy and efficiency of malicious domain identification; determining the first domain name table based on whitelisted primary domains and the first address table based on CDN addresses solves the problem of lacking filtering criteria for resolution records; parsing the domain name resolution logs into resolution records and then filtering these records improves data processing efficiency and solves the problem of low data processing efficiency when directly filtering domain name resolution logs; and extracting resolution records sequentially from descending primary domains... A preset number of primary domains are defined, and the extracted primary domains are identified as whitelisted primary domains, thus resolving the issue of determining whitelisted primary domains. By calculating the number of mappings and a threshold number, the clustered address is determined to be the content distribution address when the number of mappings exceeds the threshold number, thus resolving the issue of determining the content distribution address. A second domain name table is determined based on the whitelisted primary domains, and a second address table is determined based on the content distribution network address. These two tables are then sent to the address information database to facilitate the database's determination of the first domain name table and the first address table, thus resolving the issue of incomplete whitelisted primary domains and content distribution network addresses. Filtering domain name resolution logs in different orders addresses the issue of varying business scenario requirements.

[0200] In this embodiment of the invention, electronic devices or main control devices can be divided into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment of the invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0201] Figure 10 This is a schematic diagram of the structure of the data processing apparatus provided in an embodiment of this application. Figure 10 As shown, the data processing apparatus 130 provided in this application embodiment includes: an acquisition module 131, a determination module 132, and a filtering module 133;

[0202] Module 131 is used to obtain a first resolution table, which includes multiple resolution records. Each resolution record includes a domain name and the Internet Protocol address corresponding to the domain name.

[0203] The determination module 132 is used to determine multiple whitelisted main domains based on multiple domains in the first resolution table, and to determine the first domain table based on the multiple whitelisted main domains;

[0204] The determining module 132 is further configured to determine multiple content delivery network addresses based on multiple Internet Protocol addresses in the first parsing table, and to determine the first address table based on the multiple content delivery network addresses;

[0205] The filtering module 133 is used to filter the resolution records corresponding to multiple main domains, invalid domains or wildcard domains according to the first domain name table, and to filter the resolution records corresponding to multiple clustered addresses according to the first address table.

[0206] In one possible design, the filtering module 133 includes: a domain name determination module and a single filtering module;

[0207] The domain name determination module is used to determine multiple primary domain names, invalid domain names, and wildcard domain names based on the first DNS table.

[0208] The single filtering module is used to filter the DNS records corresponding to the first domain when the first domain is in the first domain table. The first domain can be any one of multiple main domains.

[0209] The single filtering module is also used to filter the DNS records corresponding to the second domain name. The second domain name is any one of multiple invalid domain names. Invalid domain names refer to domain names on the local area network and domain names that have failed to resolve.

[0210] The single filtering module is also used to filter the DNS records corresponding to multiple third-party domains when the main domains corresponding to multiple third-party domains are the same. The third-party domain can be any one of the multiple wildcard domains.

[0211] In one possible design, the domain name determination module is used to determine the fourth domain name based on the third domain name, where the fourth domain name is the main domain name corresponding to the third domain name;

[0212] A single filtering module is used to filter the DNS records corresponding to the third domain when the fourth domain is in the first domain table;

[0213] The single filtering module is also used to replace the third domain name with the fourth domain name in the DNS record corresponding to the third domain name when the fourth domain name is not in the first domain name table.

[0214] In one possible design, the filtering module 133 further includes: an address determination module and a global filtering module;

[0215] The address determination module is used to determine multiple cluster addresses based on the first resolution table, and each cluster address corresponds to multiple domain names;

[0216] The global filtering module is used to filter the parsing records corresponding to the first address when the first address is located in the first address table. The first address is any one of multiple cluster addresses.

[0217] In one possible design, the acquisition module 131 includes: an indication information module, a parsing record determination module, and a parsing table determination module;

[0218] The instruction information module is used to send a second instruction information to the log collection device when it receives the first instruction information, so that the log collection device can collect the full domain name resolution logs for a preset time period.

[0219] The DNS record determination module is used to determine multiple DNS records based on the domain name and Internet Protocol address in each DNS resolution log.

[0220] The parsing table determination module is used to determine the first parsing table based on multiple parsing records.

[0221] In one possible design, the resolution record also includes: the number of domain name resolution logs corresponding to the domain name;

[0222] Module 132 is defined, including: a domain name sorting module, a domain name extraction module, and an information sending and receiving module;

[0223] The domain name sorting module is used to determine multiple primary domain names based on multiple domain names in the first resolution table, and sort the multiple primary domain names in descending order according to the number of domain name resolution logs corresponding to each primary domain name to obtain the sorting result;

[0224] The domain name extraction module is used to extract a preset number of whitelisted main domain names in order of sorting results, and to determine the second domain name table based on the preset number of whitelisted main domain names;

[0225] The information sending and receiving module is used to send the second domain name table to the address information database, so that the address information database can update the pre-stored whitelist table with a preset number of whitelisted main domain names, and obtain the first domain name table.

[0226] In one possible design, the determining module 132 further includes: a clustering module, a calculation module, a comparison module, and a second address determining module;

[0227] The clustering module is used to determine multiple cluster addresses based on the first resolution table, and each cluster address corresponds to multiple domain names;

[0228] The calculation module is used to determine the number of mappings and the threshold number of the second address, which is any one of multiple clustering addresses;

[0229] The comparison module is used to determine the second address as the content delivery network address when the number of mappings exceeds a threshold number;

[0230] The second address determination module is used to determine the second address table based on multiple content distribution network addresses and multiple domain names corresponding to each content distribution network address;

[0231] The information sending and receiving module is used to send the second address table to the address information database, so that the address information database can update the pre-stored content distribution network address table with multiple content distribution network addresses and multiple domain names corresponding to each content distribution network address, thus obtaining the first address table.

[0232] In one possible design, the multiple domain names corresponding to the second address include a first number of fourth domain names and a second number of fifth domain names, wherein each fourth domain name corresponds to one domain name resolution log and each fifth domain name corresponds to multiple domain name resolution logs;

[0233] The calculation module is used to determine the number of times the second address is mapped based on the first quantity and the second quantity, and the number of mappings is equal to the sum of the first quantity and the second quantity;

[0234] The calculation module is also used to determine the threshold number R for the second address using the following formula:

[0235]

[0236] Where A is the median of the number of DNS resolution logs corresponding to the second number of fifth domains, B is the average of the number of DNS resolution logs corresponding to the second number of fifth domains, m is the first number, n is the second number, k is the preset adjustment coefficient, and h is the second number. iLet h be the number of DNS resolution logs corresponding to the i-th fifth domain name, where A, B, m, and n are positive integers, i is a positive integer not greater than n, k is a positive number less than 1, and h is a positive number. i It is a positive integer greater than 1.

[0237] In one possible design, the filtering module 133 is used to filter multiple resolution records of the first resolution table according to the first domain name table to obtain the second resolution table;

[0238] The filtering module 133 is also used to filter multiple parsing records of the second parsing table according to the first address table.

[0239] In one possible design, the filtering module 133 is used to filter multiple parsing records of the first parsing table according to the first address table to obtain a third parsing table;

[0240] The filtering module 133 is also used to filter multiple DNS records in the third DNS table based on the first domain name table.

[0241] In one possible design, the filtering module 133 is used to filter multiple resolution records of the first resolution table according to the first domain name table to obtain the second resolution table;

[0242] The filtering module 133 is also used to filter multiple parsing records of the first parsing table according to the first address table to obtain the third parsing table;

[0243] The filtering module 133 is also used to determine a fourth parsing table based on the second parsing table and the third parsing table. The fourth parsing table includes multiple parsing records that are common to the second parsing table and the third parsing table.

[0244] The data processing device provided in this embodiment can execute a domain name resolution log filtering method of the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0245] In the specific implementation of the aforementioned domain name resolution log filtering method, each module can be implemented as a processor. The processor can execute computer execution instructions stored in the memory, thereby enabling the processor to execute the aforementioned domain name resolution log filtering method.

[0246] Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application. For example... Figure 11 As shown, the electronic device 200 includes at least one processor 210 and a memory 220. The electronic device 200 also includes a communication component 230. The processor 210, memory 220, and communication component 230 are connected via a bus 240.

[0247] In the specific implementation process, at least one processor 210 executes computer execution instructions stored in memory 220, causing at least one processor 210 to execute a domain name resolution log filtering method as executed on the electronic device side as described above.

[0248] The specific implementation process of processor 210 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0249] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0250] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0251] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0252] The above description of the functions implemented by electronic devices and main control devices has introduced the solutions provided by the embodiments of the present invention. It is understood that, in order to implement the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present invention.

[0253] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the domain name resolution log filtering method described above.

[0254] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0255] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0256] This application also provides a computer program product, comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the scheme provided in any of the above embodiments.

[0257] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0258] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. The above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for filtering domain name resolution logs, characterized in that, The method includes: Obtain a first DNS table, which includes multiple DNS records. Each DNS record includes: a domain name, the Internet Protocol address corresponding to the domain name, and the number of DNS resolution logs corresponding to the domain name. Multiple primary domains are determined based on multiple domains in the first resolution table, and the multiple primary domains are sorted in descending order according to the number of domain name resolution logs corresponding to each primary domain to obtain a sorting result; a preset number of whitelisted primary domains are extracted sequentially according to the sorting result, and a second domain table is determined based on the preset number of whitelisted primary domains; the second domain table is sent to the address information database so that the address information database updates the preset number of whitelisted primary domains to the pre-stored whitelist table to obtain a first domain table; Multiple cluster addresses are determined based on the first resolution table, and each cluster address corresponds to multiple domain names; The mapping count and threshold count of a second address are determined, where the second address is any one of the plurality of clustered addresses; when the mapping count is greater than the threshold count, the second address is determined to be a content delivery network address; a second address table is determined based on the plurality of content delivery network addresses and the plurality of domain names corresponding to each content delivery network address; the second address table is sent to an address information database so that the address information database updates the plurality of content delivery network addresses and the plurality of domain names corresponding to each content delivery network address to a pre-stored content delivery network address table, thereby obtaining a first address table; The first domain name table is used to filter the resolution records corresponding to multiple main domain names or wildcard domain names, and the resolution records corresponding to multiple invalid domain names are also filtered. The first address table is used to filter the resolution records corresponding to multiple clustered addresses. Among them, the multiple domain names corresponding to the second address include a first number of fourth domain names and a second number of fifth domain names, each fourth domain name corresponds to a domain name resolution log, and each fifth domain name corresponds to multiple domain name resolution logs; The determination of the mapping count and threshold count for the second address includes: The number of times the second address is mapped is determined based on the first quantity and the second quantity, and the number of mappings is equal to the sum of the first quantity and the second quantity; The threshold number of times for the second address is determined using the following formula. : Among them, the The median of the number of DNS resolution logs corresponding to the second number of fifth domain names, the The average number of domain name resolution logs corresponding to the second number of fifth domain names, the For the first quantity, the For the second quantity, the The preset adjustment coefficient, the For the first The number of domain name resolution logs corresponding to each fifth domain name, the The above The above and stated The value is a positive integer. Not greater than the stated positive integers, the For positive numbers less than 1, the It is a positive integer greater than 1.

2. The method according to claim 1, characterized in that, The step of filtering DNS records corresponding to multiple primary domains or wildcard domains according to the first domain name table, and filtering DNS records corresponding to multiple invalid domains, includes: Based on the first resolution table, multiple primary domains, invalid domains, and wildcard domains are identified; When the first domain name is in the first domain name table, filter the DNS records corresponding to the first domain name. The first domain name is any one of multiple main domain names. Filter the DNS records corresponding to the second domain name, which is any one of a number of invalid domain names. The invalid domain names refer to domain names on the local area network and domain names that failed to resolve. When multiple third domains correspond to the same primary domain, a sixth domain is determined based on the third domain. The sixth domain is the primary domain corresponding to the third domain. When the sixth domain is in the first domain table, the DNS records corresponding to the third domain are filtered. When the sixth domain is not in the first domain table, the third domain is replaced with the sixth domain in the DNS records corresponding to the third domain. The third domain is any one of multiple wildcard domains.

3. The method according to claim 2, characterized in that, The step of filtering the parsing records corresponding to multiple clustered addresses according to the first address table includes: Multiple cluster addresses are determined based on the first resolution table, and each cluster address corresponds to multiple domain names; When the first address is located in the first address table, filter the parsing records corresponding to the first address, where the first address is any one of the plurality of cluster addresses.

4. The method according to any one of claims 1-3, characterized in that, The process of obtaining the first parsing table includes: Upon receiving the first instruction information, a second instruction information is sent to the log collection device so that the log collection device can collect a full range of domain name resolution logs for a preset time period. The plurality of resolution records are determined based on the domain name and Internet Protocol address in each domain name resolution log; The first parsing table is determined based on the plurality of parsing records.

5. The method according to any one of claims 1-3, characterized in that, The step of filtering DNS records corresponding to multiple primary domains or wildcard domains according to the first domain name table, filtering DNS records corresponding to multiple invalid domains, and filtering DNS records corresponding to multiple clustered addresses according to the first address table includes: The second resolution table is obtained by filtering multiple resolution records in the first resolution table based on the first domain name table; Multiple resolution records in the second resolution table are filtered based on the first address table.

6. The method according to any one of claims 1-3, characterized in that, The step of filtering DNS records corresponding to multiple primary domains or wildcard domains according to the first domain name table, filtering DNS records corresponding to multiple invalid domains, and filtering DNS records corresponding to multiple clustered addresses according to the first address table further includes: Based on the first address table, multiple parsing records of the first parsing table are filtered to obtain the third parsing table; Multiple DNS records in the third DNS table are filtered based on the first domain name table.

7. The method according to any one of claims 1-3, characterized in that, The step of filtering DNS records corresponding to multiple primary domains or wildcard domains according to the first domain name table, filtering DNS records corresponding to multiple invalid domains, and filtering DNS records corresponding to multiple clustered addresses according to the first address table further includes: The second resolution table is obtained by filtering multiple resolution records in the first resolution table based on the first domain name table; Based on the first address table, multiple parsing records of the first parsing table are filtered to obtain the third parsing table; A fourth parsing table is determined based on the second parsing table and the third parsing table. The fourth parsing table includes multiple parsing records that are common to both the second parsing table and the third parsing table.

8. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a first resolution table, which includes multiple resolution records. The resolution record includes: a domain name, the Internet Protocol address corresponding to the domain name, and the number of domain name resolution logs corresponding to the domain name. The determination module is used to determine multiple primary domains based on multiple domains in the first resolution table, and sort the multiple primary domains in descending order according to the number of domain name resolution logs corresponding to each primary domain to obtain a sorting result; extract a preset number of whitelisted primary domains in order according to the sorting result, determine a second domain table based on the preset number of whitelisted primary domains; and send the second domain table to the address information database so that the address information database can update the preset number of whitelisted primary domains to the pre-stored whitelist table to obtain a first domain table. The determining module is also used to determine multiple cluster addresses based on the first parsing table, and each cluster address corresponds to multiple domain names; The mapping count and threshold count of a second address are determined, where the second address is any one of the plurality of clustered addresses; when the mapping count is greater than the threshold count, the second address is determined to be a content delivery network address; a second address table is determined based on the plurality of content delivery network addresses and the plurality of domain names corresponding to each content delivery network address; the second address table is sent to an address information database so that the address information database updates the plurality of content delivery network addresses and the plurality of domain names corresponding to each content delivery network address to a pre-stored content delivery network address table, thereby obtaining a first address table; The filtering module is used to filter the resolution records corresponding to multiple main domains or wildcard domains according to the first domain name table, filter the resolution records corresponding to multiple invalid domains, and filter the resolution records corresponding to multiple clustered addresses according to the first address table. Among them, the multiple domain names corresponding to the second address include a first number of fourth domain names and a second number of fifth domain names, each fourth domain name corresponds to a domain name resolution log, and each fifth domain name corresponds to multiple domain name resolution logs; The determining module is specifically used to determine the number of times the second address is mapped based on the first quantity and the second quantity, wherein the number of mappings is equal to the sum of the first quantity and the second quantity; The threshold number of times for the second address is determined using the following formula. : Among them, the The median of the number of DNS resolution logs corresponding to the second number of fifth domain names, the The average number of domain name resolution logs corresponding to the second number of fifth domain names, the For the first quantity, the For the second quantity, the The preset adjustment coefficient, the For the first The number of domain name resolution logs corresponding to each fifth domain name, the The above The above and stated The value is a positive integer. Not greater than the stated positive integers, the For positive numbers less than 1, the It is a positive integer greater than 1.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement a domain name resolution log filtering method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement a domain name resolution log filtering method as described in any one of claims 1 to 7.

Images