Method for quickly loading domain name file in recursive resolution server

Abstract

The invention discloses a method for rapidly loading a domain name file in a recursive resolution server, and the method comprises the steps: introducing a lexical verification and resource record set classification mechanism in a domain name file loading process, completing the structure standardization processing before data storage, and reducing the data redundancy and repeated verification operation in a loading stage; by constructing a double-layer Hash storage structure and taking'domain name + record type 'as a Hash key, quick positioning and organization management of domain name resolution records in a memory are realized, massive domain name records are fragmented into a plurality of binary files, and a multi-thread parallel loading and mmap memory mapping technology is combined, so that the domain name resolution records can be quickly positioned and organized. The file data can be efficiently written into the memory, the copying frequency is reduced, and the overall loading speed of the system is increased. The processing complexity of a recursive resolution server during loading of large-scale supervision domain names can be remarkably reduced, the query and write-in performance of a storage structure is improved, and the system response capability in the file loading process is improved.

Description

Technical Field

[0001] This application relates to the technical field of fast loading of domain name files in a recursive DNS server, and in particular to a method for fast loading of domain name files in a recursive DNS server. Background Technology

[0002] With the rapid development of the Internet, the Domain Name System (DNS), as one of the core infrastructures supporting the normal operation of the Internet, has received widespread attention for its security and high availability. In the DNS system, the recursive resolver server is a key component connecting clients to the entire hierarchical domain name system, responsible for translating domain name resolution requests into IP addresses. Especially in the network systems of critical sectors such as government, finance, and energy, the stable operation of the recursive resolver server is crucial to the continuity and security of business services. Therefore, in the event of a network attack or sudden security incident, the recursive resolver server's ability to quickly load large-scale domain name records becomes a vital link in ensuring system recovery efficiency.

[0003] Existing recursive DNS servers typically employ single-threaded sequential resolution and a red-black tree-based storage structure when loading domain name files. In practice, the original text file is first read line by line and parsed lexically and syntactically to extract key information such as the domain name, resource record type, and TTL, which is then inserted into red-black tree nodes in memory. Since each domain name may have multiple record types and values, a linked list is needed to organize the corresponding resource record set. This method is not only complex, but also suffers from limited red-black tree search efficiency, making it difficult to meet the high-performance requirements of massive data scenarios in terms of overall loading speed and matching efficiency. Furthermore, existing solutions lack a resource record merging mechanism and cannot achieve atomic-level fast replacement during updates or incremental loading, severely restricting the DNS server's emergency response capabilities in the event of a security incident.

[0004] Therefore, in application scenarios where recursive DNS servers face large-scale loading and rapid updates of regulatory domain names, there are problems such as low data structure matching efficiency, complex processing flow, and long loading response time, which have become technical bottlenecks that urgently need to be solved. Summary of the Invention

[0005] This application provides a method for quickly loading domain name files in a recursive DNS server, aiming to solve the problems of low data structure matching efficiency, complex processing flow, and long loading response time in existing technologies when recursive DNS servers face large-scale regulatory domain name loading and rapid updates.

[0006] This application provides a method for quickly loading domain name files in a recursive DNS server, the method comprising:

[0007] A method for quickly loading domain name files in a recursive DNS server, characterized in that the method includes:

[0008] The original domain name resource record file is preprocessed, wherein the preprocessing includes: performing lexical verification on the Rdata field in the resource record, classifying multiple records according to "domain name + resource record type" as the key, and merging the Rdata fields of the classified records to form a resource record set RRSet;

[0009] Perform a double-level hash operation on each RRSet. Calculate the hash of "domain name + type" to obtain the first hash value HashValue. Take the modulo of HashValue with the number of buckets Num1 to obtain the first-level hash bucket index HashValue1. HashValue1 is hashed again to obtain the second-level hash bucket index HashValue2.

[0010] Based on the HashValue2, all RRSets are divided into multiple binary file fragments, each file containing one or more record sets corresponding to HashValue1;

[0011] Each binary file is segmented and a thread is allocated for parallel loading using a multi-threaded approach;

[0012] The loading process uses memory mapping (mmap) to read the file and deserialize the file content.

[0013] The deserialized RRSet is inserted into an in-memory data structure composed of a two-level hash table. The data structure includes: a first-level hash table containing multiple first-level hash buckets; each first-level hash bucket pointing to a second-level hash table; each second-level hash table containing multiple buckets, each bucket having linked list nodes; each linked list node storing an RRSet, arranged in order of HashValue1.

[0014] Optionally, in the above scheme, the lexical verification includes rule matching verification of the format, length, field order, and completeness of each subfield in the Rdata field.

[0015] Optionally, in the above scheme, the merging of RRSet uses "domain name + resource record type" as the key, and all records with the same key are grouped into the same set and arranged in a preset order.

[0016] In the above scheme, optionally, the number of buckets Num1 of the first-level hash table is a power of 2 that is not less than the total number of RRSets after merging, and the number of buckets Num2 of the second-level hash table is equal to the number of parallel loading threads.

[0017] In the above scheme, optionally, each binary file fragment includes file header information and RRSet data content, wherein the file header information includes an index identifier field for deserialization processing.

[0018] Optionally, in the above scheme, when loading files using the mmap method, each file segment is mapped to the virtual memory address space used by the thread, and the file content is structurally parsed.

[0019] Optionally, in the above scheme, the method further includes performing an incremental update step:

[0020] Compare the original domain name file with the updated domain name file to identify the changed records;

[0021] The changes are categorized and merged into RRSets, and multiple .diff binary fragment files are generated respectively according to the described two-level hashing method;

[0022] Multiple threads are started to load each .diff shard file and perform data structure modification operations on the corresponding second-level hash bucket replicas, specifically including: replacing the contents of RRSet nodes; deleting or inserting RRSet nodes in the linked list; and building the updated complete second-level hash bucket.

[0023] Modify the pointer address of the corresponding bucket in the first-level hash table so that it points to the constructed second-level hash bucket.

[0024] In the above scheme, optionally, each thread only operates on the second-level hash bucket copy associated with the loaded .diff file, and hash buckets that have not changed are not copied or replaced.

[0025] Optionally, in the above scheme, the construction and modification operations of the second-level hash bucket replica are performed in a private memory space, and the pointer address switching is performed uniformly after all modification operations are completed.

[0026] In the above scheme, optionally, the number of threads is a configurable parameter, and it supports automatically setting the number of threads and the corresponding number of buckets based on system resources, and synchronously adjusting the number of binary file shards.

[0027] Compared with the prior art, this application has at least the following beneficial effects:

[0028] Based on further analysis and research of existing technical problems, this application recognizes that existing technologies suffer from low data structure matching efficiency, complex processing procedures, and long loading response times in recursive DNS servers facing large-scale regulatory domain name loading and rapid updates. By introducing lexical verification and resource record set classification mechanisms during domain name file loading, structural standardization can be completed before data is entered into the database, reducing data redundancy and repetitive verification operations during the loading phase. By constructing a two-layer hash storage structure, using "domain name + record type" as the hash key, rapid location and organization of domain name resolution records in memory is achieved, overcoming the low query efficiency and complex insertion problems of traditional red-black tree structures. By fragmenting massive domain name records into multiple binary files and combining multi-threaded parallel loading and mmap memory mapping technology, file data can be efficiently written to memory, reducing the number of copies and accelerating the overall system loading speed. The synergistic combination of these technologies significantly reduces the processing complexity of recursive DNS servers when loading large-scale regulatory domain names, improves the query and write performance of the storage structure, and enhances the system responsiveness during file loading, thereby effectively solving the problems of complex loading procedures, low structure matching efficiency, and slow loading speed in existing technologies. Attached Figure Description

[0029] Figure 1 A flowchart illustrating a method for quickly loading domain name files in a recursive DNS server according to an embodiment of this application;

[0030] Figure 2 A schematic diagram of the prior art solution for storing domain name files in memory, provided as an embodiment of this application;

[0031] Figure 3 This is a schematic diagram illustrating fast domain name loading for a recursive DNS server provided in one embodiment of this application;

[0032] Figure 4 This is a schematic diagram of a two-layer hash fast domain name matching provided in one embodiment of this application;

[0033] Figure 5 A flowchart illustrating the atomic-level update process of the secondary hash bucket of the recursive parsing server provided in one embodiment of this application;

[0034] Figure 6 This is a schematic diagram illustrating incremental domain name updates as provided in one embodiment of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0036] In the description of this application: unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," "third," etc., in this application are intended to distinguish the objects referred to and do not have any special meaning in terms of technical connotation (e.g., they should not be construed as an emphasis on importance or order). Expressions such as "comprising," "including," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0037] In one embodiment, such as Figure 1 As shown, a method for quickly loading domain name files in a recursive DNS server is provided, including the following steps:

[0038] The original domain name resource record file is preprocessed, wherein the preprocessing includes: performing lexical verification on the Rdata field in the resource record, classifying multiple records according to "domain name + resource record type" as the key, and merging the Rdata fields of the classified records to form a resource record set RRSet;

[0039] Perform a double-level hash operation on each RRSet. Calculate the hash of "domain name + type" to obtain the first hash value HashValue. Take the modulo of HashValue with the number of buckets Num1 to obtain the first-level hash bucket index HashValue1. HashValue1 is hashed again to obtain the second-level hash bucket index HashValue2.

[0040] Based on the HashValue2, all RRSets are divided into multiple binary file fragments, each file containing one or more record sets corresponding to HashValue1;

[0041] Each binary file is segmented and a thread is allocated for parallel loading using a multi-threaded approach;

[0042] The loading process uses memory mapping (mmap) to read the file and deserialize the file content.

[0043] The deserialized RRSet is inserted into an in-memory data structure composed of a two-level hash table. The data structure includes: a first-level hash table containing multiple first-level hash buckets; each first-level hash bucket pointing to a second-level hash table; each second-level hash table containing multiple buckets, each bucket having linked list nodes; each linked list node storing an RRSet, arranged in order of HashValue1.

[0044] To address the problems of complex processing, low data structure matching efficiency, and insufficient loading efficiency of existing recursive DNS servers when loading massive amounts of DNS records, this implementation provides a method for quickly loading domain name files using a recursive DNS server based on a two-layer hash structure and a parallel loading mechanism. In this implementation, the original domain name resource record file is first preprocessed. The original file is typically in plain text format and contains a large number of DNS resource records. The record content includes the domain name, resource record type (e.g., A, AAAA, CNAME, MX), resource record category (e.g., IN), TTL value, and Rdata field. The first step of preprocessing is to perform lexical verification, using rules corresponding to the DNS resource record type to segment and verify the format of the Rdata field. Taking an A record as an example, Rdata must be a valid IPv4 address, 4 bytes in length, and in standard dotted decimal format. The verification process does not involve semantic resolution; it only verifies the format validity to ensure the correct structure and consistent format of the subsequently loaded data.

[0045] After lexical validation, all records are categorized using the key combination of "domain name + resource record type". This operation aims to integrate multiple Rdata records of the same type under the same domain into a single resource record set (RRSet), forming a logically merged structure. For example, when a domain has multiple A record Rdata values, the system aggregates them into the same data set for easier subsequent unified insertion, updates, and matching.

[0046] After classification and merging, a two-level hash operation is performed on each RRSet. The first step is to use the domain name and record type as hash input to generate a first hash value (HashValue) using a hash function. Then, this hash value is modulo the number of first-level hash buckets (Num1) to obtain the first-level hash bucket index (HashValue1). This HashValue1 is then processed by a second hash function to obtain the second-level hash bucket index (HashValue2). Through these two hash operations, each RRSet can be uniquely mapped to a specific second-level sub-table structure within a first-level hash bucket, thus forming a two-level storage space partition.

[0047] Based on the second-level hash bucket index HashValue2, all RRSets are divided into multiple file fragments, and each fragment is serialized into a binary format file. This fragmentation process fully utilizes the uniformity of hash distribution, ensuring a relatively balanced number of RRSets in each fragment. The binary file format includes necessary header structures, such as the number of records, offset information, and index identifiers, to support subsequent deserialization processing. Multiple loading threads are started, each responsible for loading an independent fragment file. Each thread uses the mmap (memorymapping) interface provided by the operating system to map its corresponding file into virtual memory space, avoiding multiple data copies in traditional I / O and improving loading efficiency. After mapping, the thread performs deserialization operations on the file content, restoring the structured RRSet data to in-memory structure objects.

[0048] After deserialization, the RRSet data is inserted into a two-level hash table structure in memory. This structure consists of a first-level hash table and multiple second-level hash tables. Each bucket in the first-level hash table corresponds to a second-level hash table. Each bucket in the second-level hash table uses a linked list structure to store multiple RRSet nodes in case of hash collisions. The nodes in the linked list are sorted according to the original first hash value HashValue1 to ensure the orderly execution of subsequent insertion and lookup operations.

[0049] This embodiment merges resource record lexical verification with RRSet during the preprocessing stage, avoiding repetitive validity checks and data organization during loading, simplifying the loading path, and improving execution determinism and modularity. Traditional red-black trees have high search complexity in multi-level nested structures and are not suitable for concurrent writes. This solution uses a two-level hash structure, which can quickly locate the target bucket position through hash index during insertion and search, and then perform ordered search through linked lists, avoiding frequent rotation and rebalancing operations in tree structures, thus improving overall access efficiency.

[0050] This embodiment splits large files into multiple fragments according to hash bucket logic and uses multi-threaded concurrent loading. Each thread independently operates on its own responsible file and memory area. Combined with the mmap mechanism, it reduces I / O bottlenecks and accelerates the entire domain name loading process.

[0051] This implementation method achieves efficient, well-structured domain name file loading by collaboratively implementing steps such as lexical verification and RRSet merging, a two-layer hash storage structure, multi-threaded parallel loading, and mmap memory mapping. It can meet the needs of recursive resolution servers for fast loading of a large number of regulatory domain names in the event of network anomalies or sudden security incidents, and has good system adaptability and engineering feasibility.

[0052] In this embodiment, the lexical verification includes rule matching verification of the format, length, field order, and completeness of each subfield in the Rdata field.

[0053] In this embodiment, the RRSet is merged using "domain name + resource record type" as the key, grouping all records with the same key into the same set and arranging them in a preset order.

[0054] In this embodiment, the number of buckets Num1 of the first-level hash table is a power of 2 that is not less than the total number of RRSets after merging, and the number of buckets Num2 of the second-level hash table is equal to the number of parallel loading threads.

[0055] In this embodiment, each binary file fragment includes file header information and RRSet data content, wherein the file header information includes an index identifier field for deserialization processing.

[0056] In this embodiment, when loading files using the mmap method, each file segment is mapped to the virtual memory address space used by the thread, and the file content is structurally parsed.

[0057] In this embodiment, the method further includes performing an incremental update step:

[0058] Compare the original domain name file with the updated domain name file to identify the changed records;

[0059] The changes are categorized and merged into RRSets, and multiple .diff binary fragment files are generated respectively according to the described two-level hashing method;

[0060] Multiple threads are started to load each .diff shard file and perform data structure modification operations on the corresponding second-level hash bucket replicas, specifically including: replacing the contents of RRSet nodes; deleting or inserting RRSet nodes in the linked list; and building the updated complete second-level hash bucket.

[0061] Modify the pointer address of the corresponding bucket in the first-level hash table so that it points to the constructed second-level hash bucket.

[0062] In this embodiment, each thread only operates on the second-level hash bucket copy associated with the loaded .diff file; hash buckets that have not changed are not copied or replaced.

[0063] In this embodiment, the construction and modification of the second-level hash bucket replicas are performed in a private memory space, and the pointer address switching is performed uniformly after all modification operations are completed.

[0064] In this embodiment, the number of threads is a configurable parameter, and it supports automatically setting the number of threads and the corresponding number of buckets based on system resources, and synchronously adjusting the number of binary file fragments.

[0065] This embodiment introduces lexical verification and resource record set classification mechanisms during the domain name file loading process, enabling structural standardization before data is entered into the database, reducing data redundancy and repetitive verification operations during the loading phase. By constructing a two-layer hash storage structure, using "domain name + record type" as the hash key, it achieves rapid location and organization of domain name resolution records in memory, overcoming the low query efficiency and complex insertion problems of traditional red-black tree structures. By fragmenting massive domain name records into multiple binary files and combining multi-threaded parallel loading with mmap memory mapping technology, file data can be efficiently written to memory, reducing the number of copies and accelerating the overall system loading speed. The synergistic effect of these technologies significantly reduces the processing complexity of recursive resolution servers when loading large-scale regulated domain names, improves the query and write performance of the storage structure, and enhances the system responsiveness during file loading, thereby effectively solving the problems of complex loading processes, low structure matching efficiency, and slow loading speeds in existing technologies.

[0066] In one embodiment, DNS (Domain Name System), as the "navigation system" of the Internet, is not only a tool for converting domain names to IP addresses, but also a core infrastructure that supports network performance, security, and scalability. It is one of the most core and critical infrastructures of the Internet.

[0067] The recursive DNS server is a core component of the Domain Name System (DNS). As the entry point for users to interact with the DNS hierarchy, it completes the complex domain name resolution process on behalf of the client. Upon receiving a query request from a client, the recursive DNS server first triggers an iterative query process, sending domain name resolution queries to the root servers, top-level domain name servers, and authoritative servers, ultimately converting the user's domain name query request into an IP address. Secondly, the recursive DNS server caches the resolution results returned by the iterative queries for a period of time to reduce duplicate queries; the specific caching time is determined by the Time To Live (TTL).

[0068] Cache poisoning, response tampering, and domain hijacking are high-risk threats during the operation of recursive DNS servers. Recursive DNS servers inject incorrect DNS records into their cache. When a user requests to resolve a poisoned domain, the server directly returns a malicious cached record, redirecting the user to a fake phishing website, a malicious server, or to obtain fake resources, thereby achieving traffic hijacking, data theft, or network attacks.

[0069] Deploying DNSSEC, enabling TCP transport protocols, or DNS overTLS / HTTPS technologies can improve the security of recursive DNS servers and reduce the probability of encountering risks. However, if a recursive DNS server has already suffered attacks such as cache poisoning, domain hijacking, and response tampering, causing abnormal resolution of official domain names in critical sectors such as government, finance, and energy, resulting in a major emergency cybersecurity incident, regulatory authorities will issue a domain name file to the operator. This file contains the resolution records of millions of regulated domain names, requiring the recursive DNS server to quickly load the file containing a large number of regulated domain name resolution records and quickly restore normal resolution responses.

[0070] The current mainstream DNS record file loading schemes are as follows:

[0071] Construct a red-black tree data structure in memory;

[0072] Open the domain name file, perform lexical analysis line by line, and extract key information such as domain name, resource record (RR) type, RR category, TTL, and record value;

[0073] The extracted data units are thoroughly verified to ensure their legality and accuracy.

[0074] Verified domain names are inserted into the previously created red-black tree, with each domain name occupying a node in the tree. The same domain name may have different record types such as A, AAAA, MX, etc., and the same record type may have multiple DNS record values. Ultimately, the different DNS records for the same domain name are added to a linked list. The node corresponding to a domain name in the red-black tree stores the linked list of record values ​​for different record types of that domain name.

[0075] Repeat steps two through four, iterating through and reading each line in the domain name file, storing all records in a red-black tree, and completing the loading of the domain name file into memory.

[0076] The recursive DNS server stores its data in memory as a multi-level red-black tree. Each label represents a level, and each level is a tree structure. The domain name file's memory storage structure is as follows: Figure 2 As shown.

[0077] The existing methods have the following obvious drawbacks:

[0078] The processing flow is highly complex: It lacks resource record set merging processing and fails to integrate records of different types from the same domain. This results in each record insertion requiring multiple lookups based on the domain name. Furthermore, during loading, each record's content needs lexical and syntactic validation to ensure its validity. The RDATA format depends on its TYPE and CLASS values ​​and is itself a variable-length string with a maximum length of 65535 bytes, making the validation logic extremely complex and diverse.

[0079] The data structure matching efficiency is low: the domain name file is loaded into memory and stored as a red-black tree data structure. The time complexity of the matching query is O(logn), and the search efficiency needs to be improved.

[0080] Single-threaded loading, which does not fully utilize multi-core hardware resources: The current mainstream domain name loading scheme starts a thread to read the original text file line by line, which is inefficient and has a long loading time.

[0081] The purpose of this embodiment is to provide a novel technical method for rapidly loading domain name files in a recursive DNS server. First, a preprocessing process for the regulated domain name files is introduced. This process is responsible for preprocessing the original text files, storing them using a two-layer hash data structure, assigning different domain names to their corresponding hash buckets, and ultimately outputting multiple binary files. Then, multi-threaded parallel loading technology is combined, with each thread loading a different binary file. Rapid loading of the domain name files is achieved through mmap (memory map) and deserialization, avoiding multiple memory copies required in traditional file reading. This technical solution enables the rapid loading of massive amounts of regulated domain name files. For scenarios involving partial updates to regulated domain name resolution records, the atomic update method of this patent's two-layer hash bucket can also achieve rapid online updates.

[0082] This embodiment provides a novel recursive DNS server solution for fast loading of domain name files. It mainly includes preprocessing massive amounts of regulated domain name files, storing them using a two-layer hash data structure, and splitting the original files into multiple binary files for parallel loading. The specific solution is described below, such as... Figure 3 As shown:

[0083] Lexical validation: Lexical validity checks are performed on all resource records in the original file. First, the Rdata fields are divided into subfields based on the DNSRdata type. Then, the validity of each subfield is validated using the corresponding validity rules. Finally, the entire Rdata is validated to ensure the validity and completeness of each subfield. For example, this includes checking if the data length of each field meets the requirements and checking for any missing fields. When the domain name file is loaded, the lexical validation of DNS Rdata only checks the validity of its format, not its semantics. This is because semantic validation of Rdata usually requires specific scenarios, such as verifying whether the mail server in the MX record actually exists.

[0084] Resource record merging processing: Organize by domain name + type, merge the Rdata of multiple resource records of the same domain name and type to obtain RRSet. For example, www.b.com has two A type resolution records: 1.1.1.1 and 2.2.2.2. These two A records need to be integrated together. When updating the domain name later, a single lookup can obtain the set of all resource records, which can quickly update the domain name resource records.

[0085] Double-level hash bucket data storage: This method replaces the traditional red-black tree structure with a double-level hash function to improve data processing efficiency. First, `Hash(name, type)` is used to hash `name` and `type`. The resulting hash value, `HashValue`, is then moduloed by `Num1` (where `Num1` is the number of domain names after merging RRSets, aligned upwards to a power of 2; for example, with 1 million domain name resolution records and an average of 4 A-type resolution records per domain, `Num1` is at most 2^18), to obtain `HashValue1`. Then, `HashValue1` is hashed a second time to obtain `HashValue2`, ranging from 0 to `Num2-1` (where `Num2` corresponds to the number of concurrent loading threads). This `HashValue2` is an index used to determine its position in the `HashTable1` array. The `Num2` elements of `HashTable1` each point to a second-level `HashTable2` head node. Each node in `HashTable2` has an index `HasValue1`, which points to a linked list storing multiple elements with the same `HashValue1`. These elements are sorted based on their original `HashValue` values. Figure 4 As shown;

[0086] Split into multiple binary files: The original text file is split into multiple smaller files based on the size of the first-level HashTable1 array, and finally formatted and converted to output multiple binary files;

[0087] Multi-threaded parallel loading of binary files: Multiple threads are started to load binary files in parallel, with each thread loading a different binary file. The domain name file is loaded quickly through mmap and deserialization, avoiding multiple memory copies during traditional file reading.

[0088] After the domain name file is successfully loaded, online updates are supported for scenarios requiring updates to some domain name resolution records. Updates are divided into full and incremental types. Full updates perform the same actions as loading the domain name file, while incremental updates utilize atomic update technology with a two-level hash bucket.

[0089] Incremental update conversion to binary files: Lexical verification is performed on the domain resource records in the updated text file, and they are organized according to domain name + type. The Rdata of multiple resource records of the same domain name and type are merged. Then, the domain resource records in memory are compared one by one according to domain name + type. For those domain names that have changed, a second hash is performed using domain name + type. After binary format conversion, multiple binary .diff files are finally output.

[0090] Atomic update of the second-level hash bucket: Multiple threads are restarted, each allocated memory space, and a memory copy is performed on the currently active specified second-level hash bucket. Then, the corresponding .diff file is mapped to memory using mmap. For domains that have changed, a hash is performed based on the domain name and its type. The copied hash bucket node is then matched. If the domain's Rdata set has changed, the value stored in the hash bucket node is directly updated. If a domain of that type has been deleted, the hash bucket node is removed from the singly linked list. If a new domain name is added, it is inserted into the singly linked list of that node's hash bucket. After processing all changed domains in the .diff file, the pointer of the corresponding node in HashTable1 within the process is switched to point to the updated second-level hash bucket based on the file ID, thus completing the atomic update of the second-level hash bucket. Figure 5 As shown.

[0091] The key technical point of this embodiment is: dual-level hash bucket storage: optimize the storage structure of domain names in memory, and save the domain name resolution record of the specified type to the corresponding node of the second-level hash bucket by performing two hash calculations on the name and type of the domain name, so that the domain name recursive resolution and update can be matched quickly in the future.

[0092] Multi-threaded parallel loading of shards: By using a two-layer hash bucket algorithm, the original file containing a large number of regulatory domains can be split into multiple binary files, and then loaded in parallel by multiple threads to enable the domains to be loaded and take effect quickly.

[0093] Second-level Hash Bucket Atomic Update: First, the changed domain name is formatted and converted into multiple binary files according to the double-layer hash algorithm. Then, multiple new threads are started to apply for temporary memory caching of the specified second-level hash buckets. Next, the binary file fragments are mapped using mmap. Then, the changed domain name resolution records in the cached second-level hash buckets are updated. Finally, the memory mapping pointer address being used is switched. The second-level hash bucket atomic update mechanism ensures that the service is not interrupted during the update process.

[0094] Rdata merging processing: Organizes data according to domain name + type, merging Rdata from multiple resource records of the same domain name and type. When updating the domain name, a single lookup can retrieve all Rdata sets, enabling rapid updates to domain resource records.

[0095] This embodiment employs a two-level hash bucket storage method for higher matching efficiency: First, the domain name and type are hashed initially, and then a second hash is calculated based on the number of threads concurrently loading the file. Different types of domain name resolution records are saved to designated nodes in the second-level hash bucket. Compared to tree-structured storage, the hash bucket storage method significantly improves matching efficiency. Multi-threaded parallel loading: By launching multiple threads to load the specified binary file in parallel, the effective time of the domain name file is significantly shortened compared to existing single-threaded processing methods. Atomic updates in the second-level hash bucket support rapid incremental changes to the domain name file: Multiple new threads are launched, using mmap to map incrementally changed domain name binary file fragments, and then the storage of the changed domain name in the cached second-level hash bucket is updated. Atomic updates in a single hash bucket, rather than a full update, improve incremental change efficiency and do not affect the resolution behavior of the recursive resolution server.

[0096] In one embodiment, the application scenario is as follows: Some operators' recursive DNS servers are attacked and their caches are polluted. When an operator's recursive DNS server suffers cache pollution, response tampering, and domain hijacking, the DNS resolution records in the cache are incorrect, causing many official domain name resolutions to be abnormal. At this time, the regulatory authorities issue a domain name file to the operator, which contains more than one million DNS resolution records of regulated domain names, requiring them to quickly restore the resolution results.

[0097] Without using the method of this embodiment, importing a domain name file containing a large number of regulatory domain name resolution records can only be done in a single-threaded manner because it cannot be fragmented. For millions of regulatory domain names, loading can take tens of seconds, and resolution errors during this period may cause serious network security incidents.

[0098] Using the method in this embodiment, the domain name file is first preprocessed and lexical checked. Then, multiple DNS records of the same type for the same domain name are merged. Next, the domain name + type is hashed for the first time. Since there are more than one million domain names, the hash value is modulo 218, and the modulo value is hashed a second time to obtain values ​​from 0 to 7. Eight domain name binary file fragments are then formatted and output. Finally, eight threads are started to load the binary file fragments using mmap. The entire loading process can be completed within a few seconds.

[0099] In one embodiment, after the recursive DNS server loads a file containing millions of domain names, some regulated domains are adjusted, or the DNS records of some regulated domains change, and the goal is to quickly perform incremental updates within the recursive DNS server.

[0100] The method described in this embodiment is not used. Instead, a file reconfiguration process is used, which reuses the domain name file loading method. Although this does not affect the resolution behavior of the recursive resolution service, it still takes a relatively long time.

[0101] Using the method in this embodiment, the original and updated domain name files are first preprocessed to obtain a list of domain names that have changed (e.g., 2560 domain names have changed their DNS records, 50 new regulatory domain names have been added, and 10 regulatory domain names have been deleted). The changed domain names are then hashed for the first time using the domain name + type. The hash value is then modulo 2^18 to obtain HashValue1. HashValue1 is then hashed a second time to obtain HashValue2 values ​​from 0 to 7. Specifically, 1856 domain names have a HashValue2 of 0, 712 domain names have a HashValue2 of 5, and 52 domain names have a HashValue2 of 7. Finally, three domain name binary file fragments are formatted and output instead of eight. Three threads are then started. Each thread first allocates memory, then copies the contents of a specified secondary hash bucket according to the binary file fragment ID, then loads the corresponding ID to atomically update the secondary hash bucket, and finally switches the memory mapping pointer address being used, completing the incremental update of the domain name file. Figure 6 As shown.

[0102] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for quickly loading domain name files in a recursive DNS server, characterized in that, The method includes: The original domain name resource record file is preprocessed, wherein the preprocessing includes: performing lexical verification on the Rdata field in the resource record, classifying multiple records according to "domain name + resource record type" as the key, and merging the Rdata fields of the classified records to form a resource record set RRSet; Perform a double-level hash operation on each RRSet. Calculate the hash value of "domain name + type" to obtain the first hash value HashValue. Take the modulo of HashValue with the number of buckets Num1 to obtain the first-level hash bucket index HashValue1. HashValue1 is hashed again to obtain the second-level hash bucket index HashValue2. Based on the HashValue2, all RRSets are divided into multiple binary file fragments, each file containing one or more record sets corresponding to HashValue1; Each binary file is segmented and a thread is allocated for parallel loading using a multi-threaded approach; The loading process uses memory mapping (mmap) to read the file and deserialize the file content. The deserialized RRSet is inserted into an in-memory data structure composed of a two-level hash table. The data structure includes: a first-level hash table containing multiple first-level hash buckets; each first-level hash bucket pointing to a second-level hash table; each second-level hash table containing multiple buckets, each bucket having linked list nodes; each linked list node storing an RRSet, arranged in order of HashValue1.

2. The method according to claim 1, characterized in that, The lexical verification includes rule matching verification of the format, length, field order, and completeness of each subfield in the Rdata field.

3. The method according to claim 1, characterized in that, The RRSet is merged using "domain name + resource record type" as the key, grouping all records with the same key into the same set and arranging them in a preset order.

4. The method according to claim 1, characterized in that, The number of buckets Num1 in the first-level hash table is a power of 2 that is not less than the total number of RRSets after merging, and the number of buckets Num2 in the second-level hash table is equal to the number of parallel loading threads.

5. The method according to claim 1, characterized in that, Each binary file fragment includes file header information and RRSet data content, wherein the file header information includes an index identifier field for deserialization processing.

6. The method according to claim 1, characterized in that, When loading a file using the mmap method, each file segment is mapped to the virtual memory address space used by the thread, and the file content is structurally parsed.

7. The method according to claim 1, characterized in that, The method also includes performing an incremental update step: Compare the original domain name file with the updated domain name file to identify the changed records; The changes are categorized and merged into RRSets, and multiple .diff binary fragment files are generated respectively according to the described two-level hashing method; Multiple threads are started to load each .diff shard file and perform data structure modification operations on the corresponding second-level hash bucket replicas, specifically including: replacing the contents of RRSet nodes; deleting or inserting RRSet nodes in the linked list; and building the updated complete second-level hash bucket. Modify the pointer address of the corresponding bucket in the first-level hash table so that it points to the constructed second-level hash bucket.

8. The method according to claim 7, characterized in that, Each thread operates only on the second-level hash bucket copy associated with the loaded .diff file; hash buckets that have not changed are not copied or replaced.

9. The method according to claim 7, characterized in that, The construction and modification of the second-level hash bucket replicas are performed in a private memory space, and the pointer address switching is performed uniformly after all modification operations are completed.

10. The method according to any one of claims 1 to 9, characterized in that, The number of threads is a configurable parameter, and it supports automatically setting the number of threads and corresponding buckets based on system resources, and synchronously adjusting the number of binary file shards.

Images