Domain name resolution log query method and device, electronic equipment and storage medium

Abstract

The invention relates to the field of computers, and provides a domain name resolution log query method and device, electronic equipment and a storage medium. The method comprises the steps that a log query request is obtained, the log query request comprises query conditions, and the query conditions at least comprise at least one domain name to be queried and a query time range; determining time in the query time range and a target warehouse corresponding to the to-be-queried domain name according to a corresponding relation between the log warehouse and the domain name and between the log warehouse and the log time; if the target warehouse comprises a cold log warehouse, determining a first block where the domain name resolution log meeting the first time range and the first domain name is located in the cold log warehouse; querying the domain name resolution log meeting the query condition in the first block; according to the method, the query result of the log query request is generated according to the domain name resolution log queried from the first block, so that the duration of querying the domain name resolution log in the cold log warehouse is greatly shortened, the query efficiency is improved, and the query time consumption is reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, electronic device, and storage medium for querying domain name resolution logs. Background Technology

[0002] Domain name resolution logs contain a wealth of valuable information, such as user access patterns, the source of domain name resolution requests, and response times. This information is crucial for identifying potential network attacks, optimizing network performance, handling abnormal resolutions, and improving user experience.

[0003] In related technologies, when querying domain name resolution logs, it is necessary to traverse and match all stored domain name resolution logs, which is extremely time-consuming, has high query complexity, and low query efficiency. Summary of the Invention

[0004] In view of this, embodiments of this application propose a method, apparatus, electronic device, and storage medium for querying domain name resolution logs to improve the above-mentioned problems.

[0005] The embodiments of this application are implemented using the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a method for querying domain name resolution logs, including:

[0007] Obtain a log query request. The log query request includes query conditions, which must include at least one domain name to be queried and a query time range.

[0008] Based on the correspondence between log warehouses, domain names, and log times, determine the target warehouse corresponding to the time within the query time range and the domain name to be queried. The log warehouses include cold log warehouses and hot log warehouses.

[0009] If the target repository includes a cold log repository, determine the first block in the cold log repository where the domain name resolution log that satisfies the first time range and the first domain name is located. Here, the domain name resolution logs stored in different blocks in the cold log repository involve at least one different domain name and log time. The first time range refers to the log time corresponding to the cold log repository in the query time range. The first domain name refers to the domain name corresponding to the cold log repository among at least one domain name to be queried.

[0010] In the first block, query the domain name resolution logs that meet the query conditions;

[0011] Based on the domain name resolution logs retrieved from the first block, generate the query results of the log query request.

[0012] Secondly, embodiments of this application provide a domain name resolution log query device, including:

[0013] The request retrieval module is used to retrieve log query requests. Log query requests include query conditions, which must include at least one domain name to be queried and a query time range.

[0014] The target log warehouse determination module is used to determine the target warehouse corresponding to the time within the query time range and the domain name to be queried, based on the correspondence between the log warehouse and the domain name and the log time. The log warehouse includes cold log warehouses and hot log warehouses.

[0015] The first block determination module is used to determine, if the target repository includes a cold log repository, the first block in the cold log repository containing the domain name resolution logs that satisfy the first time range and the first domain name, wherein the domain name resolution logs stored in different blocks in the cold log repository involve at least one different domain name and log time; the first time range refers to the log time corresponding to the cold log repository in the query time range; the first domain name refers to the domain name corresponding to the cold log repository among at least one domain name to be queried;

[0016] The query module is used to query domain name resolution logs that meet the query conditions in the first block;

[0017] The query result generation module is used to generate query results for log query requests based on the domain name resolution logs retrieved from the first block.

[0018] Optionally, the cold log repository includes multiple sets of blocks organized according to statistical periods, with one statistical period corresponding to one set of blocks; the first block determination module includes:

[0019] The target block set determination unit is used to determine the target block set corresponding to the target statistical period from multiple block sets organized according to the statistical period, based on the first time range; the target statistical period refers to the statistical period in which the time in the first time range is located.

[0020] The block index information acquisition unit is used to acquire block index information, which indicates the domain names involved in the domain name resolution logs stored in each block of the cold log repository;

[0021] The first block determination unit is used to locate the block corresponding to the first domain name in the target block set based on the block index information and the first domain name, and use it as the first block.

[0022] Optionally, the block index information includes the numerical range corresponding to each block. The numerical range is determined by sorting the numerical values ​​mapped to multiple domain names according to their size and then dividing the numerical values ​​mapped to multiple domain names into ranges. The first block determination unit is also used to map the first domain name to obtain the first numerical value mapped to the first domain name; determine the target numerical range where the first numerical value is located in the numerical range corresponding to multiple blocks; and locate the block corresponding to the target numerical range in the target block set according to the numerical range corresponding to each block, and use it as the first block.

[0023] Optionally, the target repository may also include a hot log repository; the domain name resolution log query device may further include: a first query module, used to query domain name resolution logs that meet the query conditions in the hot log repository; and a query result generation module 950, used to merge the domain name resolution logs queried from the first block and the domain name resolution logs queried from the hot log repository to obtain the query result for the log query request.

[0024] Optionally, the domain name resolution log query device also includes:

[0025] The log acquisition module is used to acquire multiple candidate domain name resolution logs to be stored.

[0026] The log grouping module is used to determine the first log set to be stored in the hot log warehouse and the second log set to be stored in the cold log warehouse from multiple candidate domain name resolution logs, based on the correspondence between the log warehouse and the domain name and the log time, as well as the log time and domain name in each candidate domain name resolution log.

[0027] The hot log storage module is used to create a log index for each candidate domain name resolution log in the first log set, and store the log index and the corresponding candidate domain name resolution log in the hot log repository;

[0028] The candidate block determination module is used to determine the candidate blocks to which each candidate domain name resolution log in the second log set should be stored in the cold log warehouse based on the block index information.

[0029] The cold log storage module is used to store each candidate domain name resolution log in the second log set into the corresponding candidate block in the cold log warehouse.

[0030] Optionally, the block index information includes the numerical range corresponding to each block; the cold log repository includes multiple block sets organized according to statistical periods, with one statistical period corresponding to one block set; the domain name resolution log query device also includes:

[0031] The candidate statistical period determination module is used to determine the candidate statistical period in which the log timestamps of multiple candidate domain name resolution logs belong.

[0032] The candidate block set determination module is used to create a candidate block set corresponding to the candidate statistical period in the cold log repository;

[0033] The candidate block determination module includes a processing module, which performs the following processing on each candidate domain name resolution log in the second log set: mapping the domain names in the candidate domain name resolution log to obtain candidate values; determining the candidate value range within the value range involved in the block index information; and determining the blocks corresponding to the candidate value range from the candidate block set corresponding to the candidate statistical period, as candidate blocks to be stored in the cold log warehouse for the candidate domain name resolution log.

[0034] Optionally, multiple candidate domain name resolution logs are generated within the same candidate statistical period; the domain name resolution log query device also includes:

[0035] The DNS resolution count acquisition module is used to count the number of times each domain name is resolved during the candidate statistical period based on multiple candidate domain name resolution logs;

[0036] The domain name set determination module is used to determine the first set of domain names whose number of resolutions exceeds the threshold and the second set of domain names whose number of resolutions does not exceed the threshold, based on the number of resolutions of each domain name in the candidate statistical period.

[0037] The correspondence determination module is used to determine that the log warehouse corresponding to each domain in the first domain set during the candidate statistical period is a hot log warehouse, and to determine that the log warehouse corresponding to each domain in the second domain set during the candidate statistical period is a cold log warehouse, and to update the correspondence between the log warehouse, the domain name, and the log time.

[0038] Optionally, the log acquisition module includes:

[0039] The log acquisition unit is used to retrieve multiple original candidate domain name resolution logs from the domain name resolution server;

[0040] The field extraction unit is used to extract the field values ​​of each key field in the key field list from each original candidate domain name resolution log based on a preset key field list; the key fields include at least a first field representing the domain name and a second field representing the log time;

[0041] The key field integration unit is used to integrate the field values ​​of multiple key fields extracted from the same original candidate domain name resolution log with the corresponding key fields to obtain the candidate domain name resolution log to be stored.

[0042] Optionally, the domain name resolution log query device also includes:

[0043] The filtering module is used to filter invalid logs from multiple original candidate domain name resolution logs, so as to extract field values ​​from the remaining original candidate domain name resolution logs after filtering.

[0044] Thirdly, embodiments of this application provide an electronic device, including: a processor; a memory, on which computer-readable instructions are stored, wherein when the computer-readable instructions are executed by the processor, a method for querying domain name resolution logs is implemented.

[0045] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-readable instructions thereon, which, when executed by a processor, implement a method for querying domain name resolution logs.

[0046] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement a method for querying domain name resolution logs.

[0047] In this application, domain name resolution logs are stored using a hot and cold separation method, with some logs stored in a cold log repository and some in a hot log repository. Furthermore, the log repository for a domain name resolution log is located based on the domain name and log time. The domain name and log time are two key pieces of information in the domain name resolution log, allowing for accurate location of the desired log repository. In addition, within the cold log repository, domain name resolution logs are stored in blocks, with different blocks distinguished by domain name and log time. Thus, when querying domain name resolution logs in the cold log repository, the process can first locate the first block containing the domain name resolution logs that match the first domain name and first time range involved in the query, and then specifically query the first block for domain name resolution logs that meet the query criteria. By first locating the first block containing the domain name resolution log to be queried based on the domain name and log time, the scope for querying and matching with the query conditions is narrowed. Compared with querying and matching the log indexes of all domain name resolution logs in the cold log repository based on the query conditions, the method of this application can significantly reduce the time spent querying domain name resolution logs in the cold log repository, improve query efficiency, and reduce query time.

[0048] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A schematic diagram illustrating an application scenario of the solution according to an embodiment of this application is shown.

[0051] Figure 2 A flowchart illustrating a method for querying domain name resolution logs according to an embodiment of this application is shown.

[0052] Figure 3 A flowchart illustrating step S130 according to an embodiment of this application is shown.

[0053] Figure 4 A flowchart illustrating the steps preceding step S120 according to an embodiment of this application is shown.

[0054] Figure 5 A flowchart illustrating step S310 according to an embodiment of this application is shown.

[0055] Figure 6 A flowchart illustrating the steps preceding step S320 according to another embodiment of this application is shown.

[0056] Figure 7 This illustration shows a schematic diagram of storing candidate domain name resolution logs in a cold log warehouse and a hot log warehouse in an embodiment of this application.

[0057] Figure 8 The diagram illustrates a storage process for a domain name resolution log and a query process for the domain name resolution log, according to an embodiment of this application.

[0058] Figure 9 A block diagram of a domain name resolution log query apparatus according to an embodiment of this application is shown.

[0059] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0060] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0061] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0062] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0063] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0064] Before providing a detailed description of the solution in this application, the terms used in this application are explained as follows:

[0065] A domain name is a human-readable text identifier used on the Internet to identify and locate computer or network resources (such as websites, servers, etc.). It is a string of characters used to create a URL, usually separated by dots, for example: example.com. Domain names are structured hierarchically, with the rightmost part representing the top-level domain (e.g., .com, .org, .net) and the leftmost part representing second-level domains, third-level domains, etc. All second-level domains are designated as primary domains, and all non-primary domains are called subdomains.

[0066] The Domain Name System (DNS) is used to translate domain names into corresponding IP addresses, enabling access to resources on the Internet through domain names. The DNS uses domain name resolution servers to resolve domain names.

[0067] Domain name resolution logs contain a wealth of valuable information, such as user access patterns, the source of domain name resolution requests, and response times. This information is crucial for identifying potential network attacks, optimizing network performance, addressing abnormal resolutions, and improving user experience. However, current techniques for querying domain name resolution logs require traversing and matching all stored logs, which is extremely time-consuming, complex, and inefficient.

[0068] Based on this, this application provides a method for querying domain name resolution logs. The method includes: obtaining a log query request, the log query request including query conditions, the query conditions including at least one domain name to be queried and a query time range; determining a target warehouse corresponding to the time in the query time range and the domain name to be queried based on the correspondence between log warehouses, domain names, and log times, the target warehouse including at least one of cold log warehouses and hot log warehouses; if the target warehouse includes a cold log warehouse, determining a first block in the cold log warehouse where the domain name resolution logs satisfying the first time range and the first domain name are located, wherein the domain name resolution logs stored in different blocks in the cold log warehouse involve at least one different domain name and log time; the first time range refers to the log time corresponding to the cold log warehouse in the query time range; the first domain name refers to the domain name corresponding to the cold log warehouse among at least one domain name to be queried; querying the domain name resolution logs satisfying the query conditions in the first block; and generating a query result of the log query request based on the domain name resolution logs queried from the first block, so as to reduce the time spent querying domain name resolution logs in the cold log warehouse, improve query efficiency, and reduce query time consumption.

[0069] Figure 1 A schematic diagram illustrating an application scenario where the technical solutions of the embodiments of this application can be applied. For example... Figure 1 As shown, this application scenario includes server 101 and terminal 102.

[0070] Server 101 can be a standalone physical server, a physical device in a server cluster or distributed system, or a device that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0071] Terminal 102 can be a smartphone, tablet computer, laptop computer, smart voice interaction device, smart home device, vehicle terminal, smart speaker, smart TV, etc. Figure 2 The example shows a terminal 102. Terminal 102 can run a client, and users can interact with server 101 by logging into the client and through the client's interactive interface or API interface.

[0072] The server 101 and the terminal 102 can communicate with each other via wired or wireless means.

[0073] like Figure 1As shown, terminal 102 can send a log query request to server 101. The log query request includes query conditions, which include at least one domain name to be queried and a query time range. In response to the received log query request, server 101 determines the target warehouse corresponding to the time in the query time range and the domain name to be queried, based on the correspondence between log warehouses, domain names, and log times. The target warehouse includes at least one of cold log warehouses and hot log warehouses. Then, if the target warehouse includes a cold log warehouse, server 101 determines the first block in the cold log warehouse where the domain name resolution logs that meet the first time range and the first domain name are located. The domain name resolution logs stored in different blocks in the cold log warehouse involve at least one different domain name and log time. The first time range refers to the log time corresponding to the cold log warehouse in the query time range. The first domain name refers to the domain name corresponding to the cold log warehouse among at least one domain name to be queried. Then, the server queries the domain name resolution logs that meet the query conditions in the first block. Based on the domain name resolution logs queried from the first block, the server generates the query result of the log query request and finally sends the generated query result to terminal 102.

[0074] In some embodiments, after receiving a log query request, the server 101 can verify the log query request, such as permission verification (e.g., verifying whether the terminal 102 is allowed to perform log queries), logic verification (verifying whether the log query request is a responsive request, verifying whether the query conditions conform to the query logic, etc.), and data type verification (verifying whether the data type of the data requested in the log query request is a processable type, etc.), to ensure the legality and validity of this query operation.

[0075] In some embodiments, the cold log repository and the hot log repository can both be deployed on server 101, or they can be deployed on other devices besides server 101. The cold log repository and the hot log repository can be deployed on different devices.

[0076] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0077] Figure 2 A flowchart illustrating a domain name resolution log query method according to an embodiment of this application is shown. This method can be executed by an electronic device with processing capabilities, the electronic device being... Figure 1 The specific equipment used on the server side is not limited here. (Refer to...) Figure 2 As shown, the method includes at least steps S110 to S150, which are described in detail below:

[0078] S110, Obtain log query request.

[0079] The log query request includes query conditions, which must include at least one domain name to be queried and a query time range.

[0080] A log query request is used to request domain name resolution logs that meet the query criteria. These domain name resolution logs are generated by the Domain Name System (DNS) during the domain name resolution process. In some embodiments, the DNS-generated DNS logs may be cleaned and processed (e.g., format conversion, key field extraction) before storage, serving as the objects for subsequent querying and analysis. The DNS logs must include at least the resolved domain name and the log's generation time. The query time range in the query criteria limits the generation time of the requested DNS logs; that is, the requested DNS logs must be those whose generation time falls within the query time range.

[0081] In this application, the domain name specified in the query conditions is referred to as the domain name to be queried. That is to say, the domain name resolution log requested for querying is the domain name resolution log generated for the domain name to be queried. The domain name to be queried can be a main domain or a subdomain.

[0082] In some embodiments, the query conditions may further limit one or more of the following: the source IP address corresponding to the requested domain name resolution log, the record type of the requested domain name resolution log, the IP address of the DNS server from which the requested domain name resolution log originated (i.e., the IP address of the server responding to the domain name resolution request), and the domain name resolution response status of the requested domain name resolution log (e.g., successful response, the requested domain name does not exist, the DNS server encountered an error, etc.). The record type of the domain name resolution log may include, for example, an A record, an AAAA record, a TXT record, a URL forwarding record, etc.

[0083] S120. Based on the correspondence between log warehouses, domain names, and log times, determine the time within the query time range and the target warehouse corresponding to the domain name to be queried.

[0084] In this application, the log warehouse storing domain name resolution logs includes a cold log warehouse and a hot log warehouse. The cold and hot log warehouses differ in terms of query response performance, storage cost, and storage media. Relatively speaking, the query response performance of the cold log warehouse is lower than that of the hot log warehouse; that is, the query response time of the cold log warehouse is longer, generally supporting slow queries, while the query response time of the hot log warehouse is shorter, supporting fast queries. The storage cost of the hot log warehouse is higher, typically using high-performance storage media, while the storage cost of the cold log warehouse is lower, typically using low-cost, low-performance storage media such as disks.

[0085] Cold log repositories, such as Cloud Object Storage (COS) and Object Storage Service (OSS), can be set up on cloud or edge servers (such as Content Delivery Networks (CDNs)). Hot log repositories, such as relational databases and key-value databases, are also available.

[0086] The correspondence between log repositories, domain names, and log times is constructed based on the domain name resolution logs stored in each log repository. If a domain name (let's say domain A) and a log time (let's say log time T) correspond to a log repository, then that log repository stores the domain name resolution logs for the resolved domain name A with the log generation time being log time T. In other words, the domain name and corresponding log time associated with a log repository define the domain name and corresponding log generation time (i.e., log time) of the domain name resolution logs stored in that log repository. That is, based on the domain name and log time, we can locate the log repository where the domain name resolution logs generated for that domain name at that log time are located. The correspondence between log repositories, domain names, and log times can also be called the correspondence between log repositories and combinations of domain names and log times. Table 1 below exemplifies the correspondence between log repositories and combinations of domain names and log times.

[0087] Table 1

[0088]

[0089] As shown in Table 1, the combinations of domain names and log times corresponding to hot log repositories include: (domain A, 12:00-13:00) and (domain B, 10:00-11:00); the combinations of domain names and log times corresponding to cold log repositories include: (domain A, 9:00-10:00), (domain B, 15:00-16:00), and (domain C, 22:00-23:00). The domain name and corresponding log time (i.e., the combination of the domain name and log time) corresponding to a log repository can be used together as the index for that log repository.

[0090] Of course, a log repository can correspond to multiple domains or just one domain, and the log timestamps for a log repository can be a time range or a single point in time. Understandably, DNS resolution logs generated for a single domain at multiple points in time may be stored in the same log repository or in two different log repositories.

[0091] In some embodiments, index pairs for a log repository can be determined based on the domain names involved in the domain name resolution logs stored in each log repository and the log time (i.e., the log generation time) of the domain name resolution logs. An index pair is a domain-time pair formed by combining a domain name and a time period. If the log time and the domain name involved in a domain name resolution log both reside in an index pair within a log repository (assuming it's a cold log repository), it can be determined that the domain name resolution log is stored in the cold log repository. The index pairs established for the log repository represent the correspondence between the log repository, the domain name, and the log time mentioned above. It is understood that a log repository can have multiple index pairs. At least one of the domain names and the corresponding time periods in different index pairs must be different.

[0092] In some embodiments, the frequency of domain name resolution (which can be the number of domain name resolutions or the domain name resolution frequency) for each domain name in multiple time periods can be statistically analyzed. Then, based on the determined resolution frequency, the log repository where the domain name resolution logs corresponding to a domain name in a given time period will be stored can be determined. It is understood that the higher the domain name resolution frequency of a domain name (let's say domain A) in a time period (let's say time period T), the higher the probability that the domain name resolution logs generated for domain A in time period T will be used for subsequent analysis; in other words, the higher the probability that the domain name resolution logs generated for domain A in time period T will be queried. Based on this, according to the statistically analyzed domain name resolution frequency, if the domain name resolution frequency of a domain name (let's say domain A) in a time period (let's say time period T) exceeds a first frequency threshold, then the domain name resolution logs generated for domain A in time period T will be stored in a hot log repository, and a corresponding correspondence will be established between the hot log repository, domain A, and time period T (the time point in time period T is the log time mentioned above).

[0093] Conversely, if the frequency of domain name resolution for a domain name (let's say domain name A) does not exceed the first frequency threshold in a time period (let's say time period T), then the domain name resolution logs generated for domain name A in that time period T will be stored in the cold log warehouse, and a corresponding relationship will be established between the cold log warehouse, domain name A, and time period T.

[0094] In some embodiments, the historical query frequency of domain name resolution logs for each domain name in each time period within multiple time periods of a historical statistical period can be obtained. The duration of a statistical period can be set as needed, such as one day, one week, etc., without specific limitations. Generally speaking, the query frequency of domain name resolution logs for a domain name in the same time period within multiple statistical periods is related. For example, if the query frequency of domain name resolution logs for a domain name in a time period (let's say time period T) within a statistical period is high, the query frequency of domain name resolution logs for that domain name in time period T within another statistical period may also be high. Based on this principle, according to the historical query frequency of domain name resolution logs for each domain name in each time period within multiple time periods of a historical statistical period, if the historical query frequency of domain name resolution logs for a domain name (let's say domain name A) in a time period (let's say time period T) exceeds a second frequency threshold, then the domain name resolution logs generated for domain name A in time period T in subsequent statistical periods will be stored in the hot log warehouse, and a corresponding correspondence will be established between the hot log warehouse, domain name A, and time period T (the time point in time period T is the log time mentioned above). Conversely, if the historical query frequency of a domain name (let's say domain name A) in a time period (let's say time period T) does not exceed the second frequency threshold, then in subsequent statistical periods, the domain name resolution logs generated for domain name A in that time period T will be stored in the cold log warehouse, and a corresponding correspondence will be established between the cold log warehouse, domain name A, and time period T.

[0095] The log repository corresponding to the time within the query time range and the domain name to be queried is called the target repository. Since the query time range may be limited to multiple time periods and there may be multiple domain names to be queried, the determined target repository may be a hot log repository, a cold log repository, or both.

[0096] It is understandable that once the time within the query time range and the target repository corresponding to the domain name to be queried are determined, it indicates that the domain name resolution log requested by the log query request is stored in the target repository.

[0097] S130. If the target repository includes a cold log repository, determine the first block in the cold log repository where the domain name resolution log that satisfies the first time range and the first domain name is located.

[0098] Among them, the domain name resolution logs stored in different blocks in the cold log warehouse involve at least one different domain name and log time; the first time range refers to the log time corresponding to the cold log warehouse in the query time range; the first domain name refers to the domain name corresponding to the cold log warehouse in at least one of the domain names to be queried.

[0099] In this application, the domain name resolution logs are stored in blocks in the cold log repository, and the domain name resolution logs stored in different blocks involve at least one of the domain names and log times.

[0100] In some embodiments, if the correspondence between log warehouses and domain names and log times is expressed as a combination of log warehouses, domain names, and log times, then for ease of description, the combination of domain names and log times is referred to as a domain name-time combination. Based on the correspondence between log warehouses and domain name-time combinations, and at least one domain name to be queried and a query time range, at least one target domain name-time combination corresponding to a cold log warehouse can be determined. Here, a target domain name-time combination refers to a domain name-time combination in the domain name-time combination corresponding to a cold log warehouse where the domain name is the domain name to be queried and the log time intersects with the query time range. Then, the domain name in the target domain name-time combination (which can be understood as the domain name to be queried) can be used as the first domain name, and the log time (or log time period) in the target domain name-time combination can be used as the first time range; or, the domain name intersecting the domain name in the target domain name-time combination with at least one domain name to be queried can be used as the first domain name, and the time intersecting the log time in the target domain name-time combination with the query time range can be used as the first time range.

[0101] In some embodiments, the domain name and log time involved in the domain name resolution logs stored in a block can be used as the index of that block. Based on this, if a time period (let's say time period T) and a domain name (let's say domain name A) match the index of a block (let's say block P), that is, the combination of time period T and domain name A is located in the index of block P, it can be determined that the domain name resolution logs generated for domain name A during time period T are stored in block P. Based on this, the block containing the domain name resolution logs satisfying the first time period and the first domain name can be determined according to the first time range, the first domain name, and the indexes of each block in the cold log repository; this is the first block. The determined first block can be one or more. Domain name resolution logs satisfying the first time range and the first domain name refer to the domain name resolution logs generated for the first domain name during the first time range.

[0102] S140. Query the domain name resolution logs that meet the query conditions in the first block.

[0103] In a cold log repository, log indexes can be built for each stored domain name resolution log. These indexes include fields representing the domain name and the log generation time, as well as other fields such as the record type, the IP address of the resolving server, and the source IP address, as mentioned above. It's understandable that the domain name resolution log index provides more detailed information than the information used to locate the log repository (i.e., the correspondence between the log repository, domain names, and log times) and the block indexes, including more indexed fields.

[0104] Based on this, the log indexes of each domain name resolution log in the first block can be matched with the query conditions to retrieve the domain name resolution logs that meet the query conditions from the first block. If there are multiple first blocks, queries can be performed in each first block according to the query conditions to obtain the domain name resolution logs that meet the query conditions.

[0105] In some embodiments, a query engine can be invoked to perform domain name resolution log queries in the first block. The query engine is not limited and can be deployed according to actual needs.

[0106] S150. Based on the domain name resolution log retrieved from the first block, generate the query result of the log query request.

[0107] The query result of the log query request includes at least the domain name resolution logs retrieved from the first block. In some embodiments, if the determined target repository is only a cold log repository, in S150, the domain name resolution logs retrieved from the first block can be used as the query result of the log query request.

[0108] In some embodiments, if the target repository further includes a hot log repository, after step S120, the method further includes: querying the hot log repository for domain name resolution logs that meet the query conditions. Correspondingly, S150 includes: merging the domain name resolution logs queried from the first block and the domain name resolution logs queried from the hot log repository to obtain the query result for the log query request.

[0109] In some embodiments, since the format of the domain name resolution logs stored in the cold log repository or the format of the domain name resolution logs stored in the hot log repository may differ from the log format that the requesting end of the log query request can adapt to display or accurately parse, directly sending the queried domain name resolution logs to the requesting end of the log query request may result in the requesting end being unable to accurately display or accurately parse the queried domain name resolution logs. Therefore, in this case, after step S150, the method further includes: obtaining log format adaptation information of the requesting end of the log query request, wherein the log format adaptation information indicates the log format that the requesting end of the log query request supports for parsing; if it is determined that the format of the domain name resolution logs in the query result of the log query request is different from the log format indicated by the log format adaptation information, the domain name resolution logs in the query result are format-converted according to the log format indicated by the log format adaptation information to obtain domain name resolution logs with the log format indicated by the log format adaptation information. Afterwards, the format-converted domain name resolution logs are sent to the requesting end of the log query request to ensure that the requesting end of the log query request can accurately parse and identify the received domain name resolution logs, so as to facilitate subsequent display or other processing.

[0110] The log format adaptation information of the requesting end of the log query request can be obtained by requesting the requesting end of the log query request, or it can be obtained from the basic information of the requesting end of the log query request. The basic information of the requesting end can include the file formats supported by each program running by the requesting end.

[0111] The application employs a hot-cold separation storage method for DNS resolution logs, storing a portion in a cold log repository and a portion in a hot log repository. Furthermore, the log repository for a DNS resolution log is located based on the domain name and log time. These two key pieces of information in the DNS resolution log are used to pinpoint the specific log repository containing the desired log. In addition, within the cold log repository, DNS resolution logs are stored in blocks, differentiated by domain name and log time. This allows for a more efficient query process within the cold log repository. First, based on the first domain name and time range involved in the query, the first block containing DNS resolution logs matching the first time range and first domain name can be located. Then, the query can be specifically performed within the first block to find DNS resolution logs that meet the query criteria. By first locating the first block containing the domain name resolution log to be queried based on the domain name and log time, the scope for querying and matching with the query conditions is narrowed. Compared with querying and matching the log indexes of all domain name resolution logs in the cold log repository based on the query conditions, the method of this application can significantly reduce the time spent querying domain name resolution logs in the cold log repository, improve query efficiency, and reduce query time.

[0112] In some embodiments, the log query request further includes processing requirements for the query results. These processing requirements indicate the necessary processing of the query results. For example, the processing requirements could be to count the number of DNS resolutions for a specified domain name within a specified time period based on the query results; or, for instance, to count the N domain names with the highest number of DNS resolutions within a specified time period based on the query results, where N is a positive integer. In this case, after obtaining the query results, the DNS resolution logs in the query results are processed according to the processing requirements in the log query request to obtain the processing results. Then, the processing results and the query results can be sent to the requesting end of the log query request.

[0113] In one embodiment, the cold log repository includes multiple block sets organized according to statistical periods, with one statistical period corresponding to one block set; such as Figure 3 As shown, S130 includes:

[0114] S210. If the target warehouse includes a cold log warehouse, determine the target block set corresponding to the target statistical period from multiple block sets organized according to the statistical period, based on the first time range.

[0115] The target statistical period refers to the statistical period within the first time range. The duration of the statistical period can be set according to requirements, such as 1 day, 2 days, etc. The number of blocks in the block set corresponding to different statistical periods can be the same or different.

[0116] The block set corresponding to a statistical period is the set of blocks of domain name resolution logs generated within that statistical period.

[0117] In some embodiments, the statistical period in which the time in the first time range is located can be determined first, i.e., the target statistical period. Then, based on the statistical period corresponding to each block set, the block set corresponding to the target statistical period can be located, i.e., the target block set.

[0118] In some embodiments, the first time range may span multiple statistical periods, and correspondingly, there are multiple target statistical periods.

[0119] S220, Obtain block index information.

[0120] The block index information indicates the domain names involved in the domain name resolution logs stored in each block of the cold log repository.

[0121] In this embodiment, multiple block sets organized according to statistical periods in the cold log repository share the same block index information. That is, block sets corresponding to different statistical periods can all use this block index information to locate the block where the domain name resolution logs under each domain are located.

[0122] For ease of description, the domain name involved in the DNS resolution logs stored in a block within the cold log repository indicated by the block index information is called the index domain name of that block. That is, the block can be located through the block's index domain name. It can be understood that a block can store DNS resolution logs for one or more domain names; therefore, the index domain name of a block in the block index information can be one or more, depending on the DNS resolution logs stored in that block.

[0123] In some embodiments, the block index information may include the index domain name of each block in the cold log repository; that is, the block index information includes the index domain name stored in association with the block identifier of each block. The blocks in the cold log repository include all blocks in the block set corresponding to each statistical period in the cold log repository.

[0124] In some embodiments, the block index information can be presented in the form of an index list, which associates the block identifier of each block with the domain identifier that indicates the domain name involved in the domain name resolution log stored in each block. The domain identifier can be the main domain name, subdomain name, domain name hash value, etc., and can be selected according to requirements.

[0125] In some embodiments, the blocks in the block set corresponding to each statistical period can be sorted to determine the sequence number of each block in the sorted block set. The block sets corresponding to different statistical periods are sorted according to the same sorting rule, such as sorting by creation time. The block index information uses the block sequence number to represent the block. Thus, domain name resolution logs generated for the same domain name in different statistical periods are stored on blocks with the same sequence number in different statistical periods. Correspondingly, the block where the domain name resolution log for a domain name is located in each statistical period can be located based on the block index information. In this way, for blocks with the same sequence number in multiple statistical periods, only one set of index information (i.e., the correspondence between block sequence number and indexed domain name) needs to be maintained, which can significantly reduce the data volume of the block index information.

[0126] S230. Based on the block index information and the first domain name, locate the block corresponding to the first domain name in the target block set and use it as the first block.

[0127] The chunk corresponding to the first domain name is the chunk whose index domain name includes the first domain name. Based on the chunk thumbnail information, the chunk whose index domain name includes the first domain name can be determined in the target chunk set and designated as the first chunk.

[0128] In one embodiment, the block index information includes the numerical range corresponding to each block. The numerical range is determined by sorting the numerical values ​​mapped to multiple domain names according to their size and then dividing the numerical values ​​mapped to multiple domain names into ranges. S230 includes A1 to A3, as follows:

[0129] A1. Map the first domain name to a numerical value to obtain the first numerical value mapped to the first domain name.

[0130] In some embodiments, mapping a domain name to a numerical value can be done using ASCII values ​​or by hashing the domain name and using the resulting hash value as the numerical value mapped to that domain name. It is worth noting that the numerical mapping method used for different domain names is the same.

[0131] In some embodiments, the main domain and subdomains in a domain name can be numerically mapped separately. Then, the final mapped value of the domain name can be determined based on the numerical values ​​mapped to the main domain and subdomains. For example, the numerical values ​​mapped to the main domain and subdomains can be concatenated to form the final mapped value; alternatively, the numerical values ​​mapped to the main domain and subdomains can be added together, and the sum can be used as the final mapped value. The methods for numerically mapping the main domain and subdomains can be the same or different, depending on the requirements. For example, in the domain name aaaa.bbbbb.com, the main domain is bbbbb.com and the subdomain is aaaa. Therefore, the main domain bbbbb.com and the subdomain aaaa can be numerically mapped separately to determine the final mapped value of the main domain and the subdomain.

[0132] In some embodiments, when the values ​​mapped by the main domain and the subdomain are concatenated and combined to form the value mapped by the domain, for example, if the value mapped by the main domain bbbbb.com is w1 and the value mapped by the subdomain aaaa is w2, the value mapped by the domain aaaa.bbbbb.com can be w1_w2. In this case, the values ​​mapped by multiple domains can be sorted according to the value mapped by the main domain in the domain to determine the value range corresponding to each block.

[0133] In some embodiments, before mapping the domain name to a value, the characters in the domain name can be reversed in order, for example, image.example.com can be converted to moc.elpmaxe.egami and blog.example.com can be converted to moc.elpmaxe.golb. Then, the reversed domain name can be mapped to a value to obtain the value mapped to the domain name.

[0134] In some embodiments, after determining the numerical values ​​of each domain name mapping as described above, the numerical values ​​of multiple domain name mappings are sorted in descending or ascending order to obtain a numerical sort. Then, according to the numerical sort, the numerical values ​​are sequentially grouped to obtain multiple numerical groups, where each numerical group includes one or more consecutive numerical values ​​in the numerical sort. Subsequently, for each numerical group, based on the maximum and minimum numerical values ​​within the group, the range defined by the minimum and maximum numerical values ​​of that group is taken as a numerical range, and a corresponding block is allocated to this numerical range. This numerical range corresponds to the numerical range of the allocated block. It can be understood that the maximum and minimum numerical values ​​within a numerical group serve as the boundaries of the numerical range corresponding to that numerical group.

[0135] The number of values ​​included in the numerical sorting within different numerical groups can be the same or different, depending on actual needs. The required number of numerical groups can be determined based on the number of chunks created in the cold log repository; for example, the number of numerical groups equals the number of chunks. Alternatively, the number of numerical groups can be determined based on the number of chunks in the chunk set corresponding to a statistical period in the cold log repository; for example, the number of numerical groups equals the number of chunks in the chunk set corresponding to a statistical period in the cold log repository.

[0136] In other embodiments, a subset of domain names can be selected from multiple domain names as reference domain names. Then, the multiple reference domain names are numerically mapped to obtain the values ​​mapped to each reference domain name. The values ​​mapped to the multiple reference domain names are then sorted in descending or ascending order to obtain a reference value sort. Next, the values ​​in the reference value sort are grouped to obtain multiple reference value groups, and a value range is determined based on the values ​​in each reference value group. Similarly, a reference value group includes multiple values ​​that are consecutively ordered in the reference value sort. The grouping method is similar to the previous embodiment and will not be repeated here.

[0137] In the above embodiment, since a portion of the domain names are selected for numerical mapping and sorting, the difference between two adjacent reference numerical sorting values ​​may be significant. Therefore, in this case, for each reference numerical group, the maximum and minimum values ​​in that group are determined. Then, the absolute value of the difference between the maximum value and the first reference value corresponding to the maximum value is determined as the first difference. The first reference value corresponding to the maximum value refers to the value that is closest to the maximum value in the reference numerical sorting and belongs to a different reference numerical group than the maximum value. The absolute value of the difference between the minimum value and the second reference value corresponding to the minimum value is determined as the second difference. The second reference value corresponding to the minimum value refers to the value that is closest to the minimum value in the reference numerical sorting and belongs to a different reference numerical group than the minimum value.

[0138] Then, based on the first difference and the maximum value in the reference value group, the maximum boundary value is determined, and based on the second difference and the minimum value in the reference value group, the minimum boundary value is determined. The range defined by the minimum boundary value and the maximum boundary value is taken as a numerical range, wherein the maximum boundary value is greater than the maximum value in the reference value group, and the minimum boundary value is less than the minimum value in the reference value group.

[0139] In some embodiments, the first coefficient can be multiplied by the first difference to obtain the first increment, and the first increment can be added to the maximum value in the reference value group to obtain the maximum boundary value; the second coefficient can be multiplied by the second difference to obtain the second increment, and the second increment can be added to the minimum value in the reference value group to obtain the minimum boundary value. Both the first and second coefficients are positive numbers greater than 0 and less than 1, and the first and second coefficients can be set according to actual needs, for example, both the first and second coefficients can be 0.5.

[0140] In the above embodiments, since it is not necessary to sort all the values ​​mapped to the domain names to determine the value range, the efficiency of determining the value range can be improved, especially when there are a large number of domain names involved.

[0141] Since the aforementioned numerical ranges are ordered (sorted from largest to smallest), the value in the preceding range is greater than the value in the following range. Therefore, in subsequent processes, the first value mapped to the first domain name can be compared with the maximum and minimum values ​​of each range to determine the numerical range in which the first value belongs. When querying domain name resolution logs in the cold log repository later, it is only necessary to determine the maximum value 'a' and the minimum value 'b' of a numerical range, as well as the number of blocks N corresponding to one statistical period. The number of queries is then log(N) + log(ab).

[0142] Since the numerical range is determined by sorting the numerical values ​​mapped to multiple domain names according to their size and then dividing the numerical values ​​mapped to multiple domain names into ranges, the numerical range corresponding to a block can be used to determine the domain name corresponding to the domain name resolution log stored in that block.

[0143] A2. Among the numerical ranges corresponding to multiple blocks, determine the target numerical range where the first value is located.

[0144] In a specific embodiment, the first value can be compared with the maximum and minimum values ​​in each value range. If the first value is not less than the minimum value in a value range and not greater than the maximum value in that value range, then the value range can be determined to be the target value range in which the first value is located.

[0145] A3. Based on the numerical range corresponding to each block, locate the block corresponding to the target numerical range in the target block set and use it as the first block.

[0146] In the above embodiments, the first domain name was numerically mapped and converted into a numerical value, making the subsequent process of determining the first block a numerical query process, thus reducing the computational resources required for the query process. In this embodiment, the first block is located through the block index information, improving query efficiency. Moreover, in this embodiment, since the numerical values ​​mapped to multiple domain names are sorted by size and then the numerical values ​​mapped to multiple domain names are divided into ranges, blocks in the block set under different statistical periods can share the same numerical range. That is, multiple different blocks can correspond to the same numerical range, which reduces the amount of index information indexed to the blocks and improves the query efficiency of the domain name resolution log.

[0147] In one embodiment, such as Figure 4 As shown, before S120, the method further includes:

[0148] S310: Obtain multiple candidate domain name resolution logs to be stored.

[0149] In some embodiments, the multiple candidate domain name resolution logs to be stored may be obtained periodically, and the multiple candidate domain name resolution logs in S310 may be domain name resolution logs generated in the most recent statistical period.

[0150] In some embodiments, the multiple candidate domain name resolution logs mentioned above can be obtained from a domain name resolution server. These multiple candidate domain name resolution logs can originate from a single domain name resolution server, or from multiple domain name resolution servers. The domain name resolution logs retrieved from the domain name resolution server are used as candidate domain name resolution logs. In some embodiments, the multiple candidate domain name resolution logs can be domain name resolution logs obtained by further processing the domain name resolution logs retrieved from the domain name resolution server; this further processing may include format conversion, cleaning, etc.

[0151] S320. Based on the correspondence between the log warehouse and the domain name and the log time, as well as the log time and domain name in each candidate domain name resolution log, determine the first log set to be stored in the hot log warehouse and the second log set to be stored in the cold log warehouse from multiple candidate domain name resolution logs.

[0152] In S320, according to the correspondence between the log warehouse and the domain name and the log time, as well as the log time and domain name in each candidate domain name resolution log, multiple candidate domain name resolution logs are divided to determine the set of candidate domain name resolution logs to be stored in the hot log warehouse (i.e., the first log set) and the set of candidate domain name resolution logs to be stored in the cold log warehouse (i.e., the second log set).

[0153] As described above, the correspondence between log warehouses, domain names, and log times defines the domain names and log times involved in the domain name resolution logs that need to be stored in the hot log warehouse, as well as the domain names and log times involved in the domain name resolution logs that need to be stored in the cold log warehouse. Based on this, for each candidate domain name resolution log, if the domain name and log time involved in the candidate domain name resolution log are in the domain name and time combination corresponding to the hot log warehouse, it is determined that the candidate domain name resolution log should be stored in the hot log warehouse; if the domain name and log time involved in the candidate domain name resolution log are in the domain name and time combination corresponding to the cold log warehouse, it is determined that the candidate domain name resolution log should be stored in the cold log warehouse.

[0154] S330. Establish log indexes for each candidate domain name resolution log in the first log set, and store the log indexes and the corresponding candidate domain name resolution logs in the hot log repository.

[0155] The log index of the candidate domain name resolution log can be used to retrieve the candidate domain name resolution log from the hot log repository. The log index can include the field values ​​of multiple fields in the candidate domain name resolution log. These fields are called index fields. For example, index fields may include a first field representing the domain name and a second field representing the log time. More fields can also be included, depending on the specific query requirements.

[0156] S340. Based on the block index information, determine the candidate blocks to which each candidate domain name resolution log in the second log set should be stored in the cold log warehouse.

[0157] As described above, the block index information indicates the domain name corresponding to the domain name resolution log to be stored in each block. Therefore, based on the domain name involved in each candidate domain name resolution log in the second log set, the block corresponding to the domain name involved in each candidate domain name resolution log in the second log set can be determined as the candidate block to be stored in the cold log warehouse for that candidate domain name resolution log.

[0158] In some embodiments, when multiple blocks in the cold log warehouse are organized according to statistical periods, the candidate statistical period in which the log time of each candidate domain name resolution log in the second log set is located can be determined based on the log time of each candidate domain name resolution log in the second log set. Then, based on the block index information, the candidate block to which each candidate domain name resolution log in the second log set is to be stored can be determined in the block set corresponding to the candidate statistical period.

[0159] In one embodiment, prior to S340, the method further includes B1 to B2, as follows:

[0160] B1. Determine the candidate statistical period in which the log times in multiple candidate domain name resolution logs belong.

[0161] B2. Create a set of candidate blocks corresponding to the candidate statistical period in the cold log repository.

[0162] The number of blocks included in the candidate block set can be set according to requirements, or it can be the same as the number of blocks in the block set corresponding to other statistical periods.

[0163] Correspondingly, in S340, the following processing (B3-B6) can be performed on each candidate domain name resolution log in the second log set:

[0164] B3. Map the domain names in the candidate domain name resolution logs to numerical values ​​to obtain candidate values.

[0165] The method for numerical mapping is described above and will not be repeated here.

[0166] B4. Within the range of values ​​involved in the block index information, determine the range of candidate values ​​where the candidate values ​​are located.

[0167] B5. From the candidate block set corresponding to the candidate statistical period, determine the block corresponding to the candidate value range, and use it as the candidate block to be stored in the cold log warehouse for the candidate domain name resolution log.

[0168] S350. Store each candidate domain name resolution log in the second log set into the corresponding candidate block in the cold log warehouse.

[0169] In addition, after maintaining the log index of each domain name resolution log in the block, the log index of each candidate domain name resolution log in the second log set is added to the log index set corresponding to the candidate block, so that the candidate domain name resolution log can be queried from the candidate block according to the log index of the candidate domain name resolution log in the future.

[0170] In the above embodiments, candidate domain name resolution logs to be stored are stored in the corresponding log warehouses according to the domain names involved in the candidate domain name resolution logs and the log time.

[0171] In one embodiment, such as Figure 5 As shown, S310 includes:

[0172] S3110: Obtain multiple original candidate domain name resolution logs from the domain name resolution server.

[0173] For ease of distinction, the domain name resolution logs retrieved by the domain name resolution server are referred to as the original candidate domain name resolution logs.

[0174] S3120. Based on the preset list of key fields, extract the field values ​​of each key field from each original candidate domain name resolution log.

[0175] The key fields include at least a first field representing the domain name and a second field representing the log time. In some embodiments, the list of key fields includes multiple fields, which can be determined according to business needs and subsequent analysis purposes. For example, fields representing key information in the domain name resolution log can be selected as key fields based on the needs of subsequent domain name resolution log analysis or query. In addition to the first and second fields mentioned above, key fields may also include fields indicating the record type of the domain name resolution log, fields indicating the IP address of the resolution server from which the domain name resolution log originated, fields indicating the domain name resolution response status of the domain name resolution log, and so on.

[0176] The value of a key field is the value of the attribute that the key field represents. For example, if the first field is a field representing the domain name (attribute), the value of the first field is the specific domain name, such as the domain name aaaa.bbbbb.com.

[0177] In some embodiments, S3120 includes: obtaining the log format (e.g., plain text, JSON, XML, etc.) of the original candidate domain name resolution log, and extracting the field values ​​of each key field in the key field list from the original candidate domain name resolution log according to the field value extraction method matching the log format. For example, for the original candidate domain name resolution log in plain text format, the field values ​​of each key field can be extracted using regular expressions; for the original candidate domain name resolution log in JSON or XML format, the field values ​​of the key fields can be extracted using the corresponding parser.

[0178] S3130. The field values ​​of multiple key fields extracted from the same original candidate domain name resolution log are integrated with the corresponding key fields to obtain the candidate domain name resolution log to be stored.

[0179] In some embodiments, integrating the field values ​​of multiple key fields extracted from the same original candidate domain name resolution log with their corresponding key fields can be done by integrating them according to a preset log template to obtain the candidate domain name resolution log to be stored. The log template defines the arrangement of each key field and its corresponding field value.

[0180] In practice, the original candidate DNS resolution logs generated by the DNS server may contain a lot of information. However, some information in these logs may not be used in subsequent DNS resolution log analysis. Therefore, storing the complete original candidate DNS resolution logs in a log repository (cold log repository and hot log repository) would result in invalid information occupying a significant amount of storage space, leading to low storage space utilization. Thus, in the above embodiment, based on a preset key field list, the values ​​of each key field in the key field list are extracted from the original candidate DNS resolution logs. The values ​​of multiple key fields extracted from the same original candidate DNS resolution log are then integrated with the corresponding key fields to obtain the candidate DNS resolution logs to be stored. These candidate DNS resolution logs are stored subsequently, instead of the original candidate DNS resolution logs pulled from the DNS server. Since the candidate DNS resolution logs do not involve field values ​​other than those in the key field list, the data volume is smaller than that of the original candidate DNS resolution logs, reducing wasted storage space. Furthermore, the key field list can be set according to actual application needs to ensure that the obtained candidate DNS resolution logs meet the actual DNS resolution log analysis requirements.

[0181] In one embodiment, prior to S3130, the method further includes: filtering invalid logs from multiple original candidate domain name resolution logs to extract field values ​​from the remaining original candidate domain name resolution logs after filtering.

[0182] Invalid logs can be filtered according to invalid log filtering rules. These rules can be set according to business needs. For example, filtering original candidate domain name resolution logs that are missing at least one key field can be used. Then, multiple original candidate domain name resolution logs can be filtered according to the invalid log filtering rules.

[0183] In some embodiments, the field values ​​of non-critical fields in each original candidate domain name resolution log can be compressed using a compression algorithm. The compressed text can be stored in the cloud or edge server, and the storage address generated after storage is associated with the corresponding candidate domain name resolution log to be stored and stored in a cold log repository or a hot log repository.

[0184] In another embodiment, multiple candidate domain name resolution logs are generated within the same candidate statistical period; before S320, such as Figure 6 As shown, the method also includes:

[0185] S410. Based on multiple candidate domain name resolution logs, count the number of times each domain name is resolved during the candidate statistical period.

[0186] S420. Based on the number of times each domain name is resolved in the candidate statistical period, determine the first set of domain names whose number of resolutions exceeds the threshold and the second set of domain names whose number of resolutions does not exceed the threshold.

[0187] The number of times threshold can be set according to business needs, without specific limitations.

[0188] S430. Determine that the log warehouse corresponding to each domain in the first domain set during the candidate statistical period is a hot log warehouse, and determine that the log warehouse corresponding to each domain in the second domain set during the candidate statistical period is a cold log warehouse, and update the corresponding relationship between the log warehouse, domain name, and log time.

[0189] Using the above method, when the existing correspondence between log warehouses, domain names, and log times does not include the log times and domain names involved in the current candidate domain name resolution logs, the method described above can be used to determine whether to store the candidate domain name resolution logs in the cold log warehouse or the hot log warehouse based on the number of times each domain name resolves within the candidate statistical period. Furthermore, generally speaking, domain names with more resolution counts are more likely to have their domain name resolution logs analyzed subsequently, meaning they are more likely to be queried. Therefore, storing the candidate domain name resolution logs corresponding to domain names with higher resolution counts in the hot log warehouse can meet the needs of subsequent high-speed queries.

[0190] Figure 7 This diagram illustrates how candidate domain name resolution logs are stored in both a cold log repository and a hot log repository. Figure 7 In this process, the original candidate domain name resolution logs are aggregated for key fields to obtain candidate domain name resolution logs to be stored. Among multiple candidate domain name resolution logs, the first log set to be stored in the hot log warehouse and the second log set to be stored in the cold log warehouse are determined. The process of determining the first log set and the second log set is as described above and will not be repeated here.

[0191] The first log set is stored in the hot log warehouse. The second log set is grouped and sorted to obtain the numerical range corresponding to each block in the cold log warehouse, and block index information is established. The candidate domain name resolution logs of each block are then stored in each block of the cold log warehouse.

[0192] In this embodiment, the above method is used to determine a first log set to be stored in the hot log warehouse and a second log set to be stored in the cold log warehouse from multiple candidate domain name resolution logs; establish a log index for each candidate domain name resolution log in the first log set, and store the log index and the corresponding candidate domain name resolution log in the hot log warehouse; and determine the candidate blocks to be stored in the cold log warehouse for each candidate domain name resolution log in the second log set according to the block index information; and store each candidate domain name resolution log in the second log set in the corresponding candidate block in the cold log warehouse. This storage method improves efficiency in subsequent log querying.

[0193] Figure 8 The diagram illustrates a process for storing domain name resolution logs and a process for querying domain name resolution logs.

[0194] During the storage of domain name resolution logs, the original candidate domain name resolution logs are first retrieved from the domain name resolution server. The original candidate domain name resolution logs are then processed, including but not limited to filtering invalid logs, aggregating key fields, distinguishing between candidate domain name resolution logs to be stored in the cold log warehouse and the hot log warehouse, and determining the correspondence between the log warehouse, the domain name, and the log time. After that, the candidate domain name resolution logs to be stored in the cold log warehouse and the hot log warehouse are stored in the cold log warehouse and the hot log warehouse, respectively.

[0195] During the domain name resolution log query process, the system receives log query requests sent by the client, determines the target repository corresponding to the time within the query time range and the domain name to be queried based on the correspondence between the log repository, the domain name, and the log time, and then queries the target repository for domain name resolution logs that meet the query conditions through the query engine. The system then integrates the queried domain name resolution logs to obtain the query results and returns the query results to the client.

[0196] In practical applications, the domain name resolution log query method has been put into use in the form of a system, processing more than 900 log query requests, saving more than 400 hours of development time and reducing query time by hundreds of times. This not only saves a lot of time and manpower costs, but also provides strong support for the development and optimization of related businesses.

[0197] The following describes an apparatus embodiment of this application, which can be used to perform the methods described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments described in the above embodiments of this application.

[0198] Figure 9 This is a block diagram of a domain name resolution log query device according to an embodiment of this application, which can be applied to electronic devices, such as... Figure 9As shown, the domain name resolution log query device 1000 includes:

[0199] The request acquisition module 1010 is used to acquire log query requests. The log query requests include query conditions, which include at least one domain name to be queried and a query time range.

[0200] The target log warehouse determination module 1020 is used to determine the target warehouse corresponding to the time within the query time range and the domain name to be queried, based on the correspondence between the log warehouse and the domain name and the log time. The log warehouse includes cold log warehouses and hot log warehouses.

[0201] The first block determination module 1030 is used to determine, if the target repository includes a cold log repository, the first block in the cold log repository containing the domain name resolution logs that satisfy the first time range and the first domain name, wherein the domain name resolution logs stored in different blocks in the cold log repository involve at least one different domain name and log time; the first time range refers to the log time corresponding to the cold log repository in the query time range; the first domain name refers to the domain name corresponding to the cold log repository among at least one domain name to be queried;

[0202] The query module 1040 is used to query domain name resolution logs that meet the query conditions in the first block;

[0203] The query result generation module 1050 is used to generate the query results of the log query request based on the domain name resolution logs retrieved from the first block.

[0204] Optionally, the cold log repository includes multiple sets of blocks organized according to statistical periods, with one statistical period corresponding to one set of blocks; the first block determination module 1030 includes:

[0205] The target block set determination unit is used to determine the target block set corresponding to the target statistical period from multiple block sets organized according to the statistical period, based on the first time range; the target statistical period refers to the statistical period in which the time in the first time range is located.

[0206] The block index information acquisition unit is used to acquire block index information, which indicates the domain names involved in the domain name resolution logs stored in each block of the cold log repository;

[0207] The first block determination unit is used to locate the block corresponding to the first domain name in the target block set based on the block index information and the first domain name, and use it as the first block.

[0208] Optionally, the block index information includes the numerical range corresponding to each block. The numerical range is determined by sorting the numerical values ​​mapped to multiple domain names according to their size and then dividing the numerical values ​​mapped to multiple domain names into ranges. The first block determination unit is also used to map the first domain name to obtain the first numerical value mapped to the first domain name; determine the target numerical range where the first numerical value is located in the numerical range corresponding to multiple blocks; and locate the block corresponding to the target numerical range in the target block set according to the numerical range corresponding to each block, and use it as the first block.

[0209] Optionally, the target repository also includes a hot log repository; the domain name resolution log query device 1000 further includes: a first query module, used to query domain name resolution logs that meet the query conditions in the hot log repository; and a query result generation module 1050, used to merge the domain name resolution logs queried from the first block and the domain name resolution logs queried from the hot log repository to obtain the query result for the log query request.

[0210] Optionally, the domain name resolution log query device 1000 further includes:

[0211] The log acquisition module is used to acquire multiple candidate domain name resolution logs to be stored.

[0212] The log grouping module is used to determine the first log set to be stored in the hot log warehouse and the second log set to be stored in the cold log warehouse from multiple candidate domain name resolution logs, based on the correspondence between the log warehouse and the domain name and the log time, as well as the log time and domain name in each candidate domain name resolution log.

[0213] The hot log storage module is used to create a log index for each candidate domain name resolution log in the first log set, and store the log index and the corresponding candidate domain name resolution log in the hot log repository;

[0214] The candidate block determination module is used to determine the candidate blocks to which each candidate domain name resolution log in the second log set should be stored in the cold log warehouse based on the block index information.

[0215] The cold log storage module is used to store each candidate domain name resolution log in the second log set into the corresponding candidate block in the cold log warehouse.

[0216] Optionally, the block index information includes the numerical range corresponding to each block; the cold log warehouse includes multiple block sets organized according to statistical periods, with one statistical period corresponding to one block set; the domain name resolution log query device 1000 also includes:

[0217] The candidate statistical period determination module is used to determine the candidate statistical period in which the log timestamps of multiple candidate domain name resolution logs belong.

[0218] The candidate block set determination module is used to create a candidate block set corresponding to the candidate statistical period in the cold log repository;

[0219] The candidate block determination module includes a processing module, which performs the following processing on each candidate domain name resolution log in the second log set: mapping the domain names in the candidate domain name resolution log to obtain candidate values; determining the candidate value range within the value range involved in the block index information; and determining the blocks corresponding to the candidate value range from the candidate block set corresponding to the candidate statistical period, as candidate blocks to be stored in the cold log warehouse for the candidate domain name resolution log.

[0220] Optionally, multiple candidate domain name resolution logs are generated within the same candidate statistical period; the domain name resolution log query device 1000 also includes:

[0221] The DNS resolution count acquisition module is used to count the number of times each domain name is resolved during the candidate statistical period based on multiple candidate domain name resolution logs;

[0222] The domain name set determination module is used to determine the first set of domain names whose number of resolutions exceeds the threshold and the second set of domain names whose number of resolutions does not exceed the threshold, based on the number of resolutions of each domain name in the candidate statistical period.

[0223] The correspondence determination module is used to determine that the log warehouse corresponding to each domain in the first domain set during the candidate statistical period is a hot log warehouse, and to determine that the log warehouse corresponding to each domain in the second domain set during the candidate statistical period is a cold log warehouse, and to update the correspondence between the log warehouse, the domain name, and the log time.

[0224] Optionally, the log acquisition module includes:

[0225] The log acquisition unit is used to retrieve multiple original candidate domain name resolution logs from the domain name resolution server;

[0226] The field extraction unit is used to extract the field values ​​of each key field in the key field list from each original candidate domain name resolution log based on a preset key field list; the key fields include at least a first field representing the domain name and a second field representing the log time;

[0227] The key field integration unit is used to integrate the field values ​​of multiple key fields extracted from the same original candidate domain name resolution log with the corresponding key fields to obtain the candidate domain name resolution log to be stored.

[0228] Optionally, the domain name resolution log query device 1000 further includes:

[0229] The filtering module is used to filter invalid logs from multiple original candidate domain name resolution logs, so as to extract field values ​​from the remaining original candidate domain name resolution logs after filtering.

[0230] Figure 10 A schematic diagram of a computer system suitable for implementing an electronic device according to embodiments of this application is shown. The electronic device can be the terminal described above, used to implement the domain name resolution log query method provided in this application. It should be noted that... Figure 10 The computer system 1200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0231] like Figure 10 As shown, the computer system 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 1202 or programs loaded from storage portion 1208 into Random Access Memory (RAM) 1203. The RAM 1203 also stores various programs and data required for system operation. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An Input / Output (I / O) interface 1205 is also connected to the bus 1204.

[0232] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, microphone, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1212 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1212 as needed so that computer instructions read from it can be loaded into storage section 1208 as needed.

[0233] In particular, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising computer instructions. When these computer instructions are executed by the central processing unit (CPU) 1201, various functions defined in the system of this application are performed.

[0234] This application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the methods described in any of the above method embodiments.

[0235] It should be noted that the computer-readable storage medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0236] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0237] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0238] In the embodiments of this application, the terms "module" or "unit" refer to computer instructions or a portion of computer instructions that have a predetermined function and work together with other related parts to achieve a predetermined goal. These instructions can be implemented, wholly or partially, using software, hardware (e.g., processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that functions as a whole.

[0239] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer-readable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-readable instructions from the computer-readable storage medium, and the processor executes the computer-readable instructions, causing the electronic device to perform the methods of any of the above embodiments.

[0240] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0241] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although this application has disclosed preferred embodiments as above, it is not intended to limit this application. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this application. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A method for querying a domain name resolution log, characterized in that, The method comprises: obtaining a log query request, the log query request comprising a query condition, the query condition comprising at least one domain name to be queried and a query time range; determining, according to a correspondence between a log warehouse and a domain name and a log time, a target warehouse corresponding to the time in the query time range and the domain name to be queried, the log warehouse comprising a cold log warehouse and a hot log warehouse; if the target warehouse comprises the cold log warehouse, determining, in the cold log warehouse, a first block in which a domain name resolution log satisfying a first time range and a first domain name is located, wherein the domain name resolution logs stored in different blocks in the cold log warehouse differ in at least one of a domain name and a log time; the first time range refers to a log time corresponding to the cold log warehouse in the query time range; and the first domain name refers to a domain name corresponding to the cold log warehouse in the at least one domain name to be queried; querying, in the first block, the domain name resolution log satisfying the query condition; generating a query result of the log query request according to the domain name resolution log queried from the first block.

2. The method of claim 1, wherein, The cold log warehouse comprises a plurality of block sets organized according to statistical periods, and one statistical period corresponds to one block set; The method comprises: determining, according to the first time range, a target block set corresponding to a target statistical period in the plurality of block sets organized according to statistical periods; the target statistical period refers to a statistical period in which the time in the first time range is located; obtaining block index information, the block index information indicating domain names involved in the domain name resolution logs stored in each block in the cold log warehouse; locating, according to the block index information and the first domain name, a block corresponding to the first domain name in the target block set as the first block.

3. The method of claim 2, wherein, The block index information comprises a numerical range corresponding to each block, and the numerical range is determined by range division of numerical values mapped by a plurality of domain names after the numerical values are sorted according to size; The method comprises: performing numerical mapping on the first domain name to obtain a first numerical value mapped by the first domain name; determining a target numerical range in which the first numerical value is located in the numerical ranges corresponding to a plurality of blocks; locating, according to the numerical range corresponding to each block, a block corresponding to the target numerical range in the target block set as the first block.

4. The method of claim 1, wherein, The target warehouse further comprises a hot log warehouse; and after the target warehouse corresponding to the time in the query time range and the domain name to be queried is determined according to the correspondence between the log warehouse and the domain name and the log time, the method further comprises: querying, in the hot log warehouse, the domain name resolution log satisfying the query condition; and The query result of the log query request is generated according to the domain name resolution log queried from the first sub-block, and the query result includes: The domain name resolution log queried from the first sub-block and the domain name resolution log queried from the hot log warehouse are merged to obtain the query result of the log query request.

5. The method of any one of claims 1 to 4, wherein the target warehouse corresponding to the time in the query time range and the domain name to be queried is determined according to the correspondence between the log warehouse and the domain name and the log time, and the method further comprises: obtaining a plurality of candidate domain name resolution logs to be stored; determining, according to the correspondence between the log warehouse and the domain name and the log time, and the log time and the domain name in each of the candidate domain name resolution logs, a first log set to be stored in the hot log warehouse and a second log set to be stored in the cold log warehouse from the plurality of candidate domain name resolution logs; establishing a log index of each of the candidate domain name resolution logs in the first log set, and storing the log index and the corresponding candidate domain name resolution log to the hot log warehouse; determining, according to the sub-block index information, a candidate sub-block to which each of the candidate domain name resolution logs in the second log set is to be stored in the cold log warehouse; storing each of the candidate domain name resolution logs in the second log set to the corresponding candidate sub-block in the cold log warehouse.

6. The method of claim 5, wherein, The sub-block index information includes a numerical range corresponding to each sub-block; and the cold log warehouse includes a plurality of sub-block sets organized according to statistical periods, one statistical period corresponding to one sub-block set; Before determining, according to the sub-block index information, the candidate sub-block to which each of the candidate domain name resolution logs in the second log set is to be stored in the cold log warehouse, the method further comprises: determining a candidate statistical period in which the log time in the plurality of candidate domain name resolution logs is located; creating a candidate sub-block set corresponding to the candidate statistical period in the cold log warehouse; Before determining, according to the sub-block index information, the candidate sub-block to which each of the candidate domain name resolution logs in the second log set is to be stored in the cold log warehouse, the method further comprises: For each of the candidate domain name resolution logs in the second log set, the following processing is performed: numerically mapping the domain name in the candidate domain name resolution log to obtain a candidate numerical value; determining a candidate numerical range in which the candidate numerical value is located in the numerical range involved in the sub-block index information; determining, from the candidate sub-block set corresponding to the candidate statistical period, a sub-block corresponding to the candidate numerical range as the candidate sub-block to which the candidate domain name resolution log is to be stored in the cold log warehouse.

7. The method of claim 5, wherein, The plurality of candidate domain name resolution logs are generated in the same candidate statistical period; Before determining, according to the correspondence between the log warehouse and the domain name and the log time, and the log time and the domain name in each of the candidate domain name resolution logs, a first log set to be stored in the hot log warehouse and a second log set to be stored in the cold log warehouse from the plurality of candidate domain name resolution logs, the method further comprises: According to the plurality of candidate domain name resolution logs, the number of resolutions of each domain name in the candidate statistical period is counted; According to the number of resolutions of each domain name in the candidate statistical period, a first domain name set whose number of resolutions exceeds a number threshold and a second domain name set whose number of resolutions does not exceed the number threshold are determined; It is determined that each domain name in the first domain name set is a hot log warehouse in the log warehouse corresponding to the candidate statistical period, and each domain name in the second domain name set is a cold log warehouse in the log warehouse corresponding to the candidate statistical period, and the corresponding relationship between the log warehouse and the domain name and the log time is updated accordingly.

8. The method of claim 5, wherein, The method comprises the following steps: Obtain a plurality of original candidate domain name resolution logs from a domain name resolution server; According to a preset key field list, the field values of each key field in the key field list are extracted from each original candidate domain name resolution log; the key field at least includes a first field indicating a domain name and a second field indicating a log time; The field values of the plurality of key fields extracted from the same original candidate domain name resolution log are integrated with the corresponding key fields to obtain the candidate domain name resolution log to be stored.

9. The method of claim 8, wherein, Before the step of extracting the field values of each key field in the key field list from each original candidate domain name resolution log according to the preset key field list, the method further comprises the following steps: Invalid log filtering is performed on the plurality of original candidate domain name resolution logs to extract the field values of the original candidate domain name resolution logs remaining after filtering.

10. A domain name resolution log query apparatus characterized by comprising: It comprises: A request acquisition module is configured to acquire a log query request, wherein the log query request comprises a query condition, and the query condition at least comprises at least one domain name to be queried and a query time range; A target log warehouse determination module is configured to determine a target warehouse corresponding to a time in the query time range and the domain name to be queried according to the corresponding relationship between the log warehouse and the domain name and the log time, wherein the log warehouse comprises a cold log warehouse and a hot log warehouse; A first block determination module is configured to determine a first block in which the domain name resolution log satisfying a first time range and a first domain name is located in the cold log warehouse if the target warehouse comprises the cold log warehouse, wherein the domain names and log times involved in the domain name resolution logs stored in different blocks in the cold log warehouse are different; the first time range refers to the log time corresponding to the cold log warehouse in the query time range; and the first domain name refers to the domain name corresponding to the cold log warehouse in the at least one domain name to be queried; A query module is configured to query the domain name resolution log satisfying the query condition in the first block; A query result generation module is configured to generate a query result of the log query request according to the domain name resolution log queried from the first block.

11. An electronic device, comprising: It comprises: A processor; A memory, wherein the memory has computer readable instructions stored thereon, and the computer readable instructions are executed by the processor to implement the method of any one of claims 1-9.

12. A computer readable storage medium having stored thereon computer readable instructions which, when executed by a processor, implement the method of any one of claims 1-9.

13. A computer program product comprising computer readable instructions, characterized in that, The computer readable instructions, when executed by a processor, implement the method of any one of claims 1-9.

Images