Dga domain name detection method and device based on semantic representation

Abstract

The application discloses a DGA domain name detection method and device based on semantic representation, and the method comprises the following steps: performing extension processing on a benign domain name list, and training a word segmentation model on an extension data set conforming to zipf's law; performing word segmentation processing on a domain name to be detected to obtain a plurality of subwords; calculating word embedding representation of each subword; and obtaining a DGA domain name detection result of the domain name to be detected based on the word embedding representation of all the subwords. The application realizes efficient identification of word DGA domain names.

Description

Technical Field

[0001] The invention relates to the field of DGA domain name detection technology, specifically a DGA domain name detection method and apparatus based on semantic representation. Background Technology

[0002] A botnet is a network of computers infected with malware (known as zombie hosts or zombie machines). These computers are remotely controlled by attackers, forming a centralized or distributed control infrastructure. Attackers control these infected computers through command and control (C&C) servers and coordinate them to carry out malicious activities. Botnets pose a serious threat to the secure operation of the internet.

[0003] In botnet C&C communication, attackers often use Domain Generation Algorithms (DGAs) to rapidly generate DGA domains using random characters, bypassing defender-set detection mechanisms to launch malicious attacks on target hosts. Because DGA domains are random and generated very quickly, network security systems often struggle to accurately detect and block communication with these domains, increasing the difficulty of defending against malware. Therefore, detecting DGA domains is a crucial issue in maintaining network security.

[0004] Daniel Plohmann et al. analyzed malware families and variants through DGA reverse engineering and categorized DGA families into four types based on the generation characteristics of DGA domains: Arithmetic-based DGAs, Hash-based DGAs, Permutation-based DGAs, and Wordlist-based DGAs. With the increasing attack challenges posed by the ever-evolving DGA domains, traditional blacklist filtering mechanisms are no longer sufficient for effectively detecting malicious domains. To address the problem of identifying malicious DGA domains, many researchers have applied machine learning and deep learning to the field of cybersecurity, classifying and detecting DGA domains based on their word features.

[0005] Early DGA domain detection methods primarily relied on manually written rules and patterns to identify DGA domains. These rules typically depended on specific features of the DGA algorithm, such as random characters and fixed length. However, rule-based detection methods had limited effectiveness when dealing with new DGA variants and complex algorithms. With the development of machine learning technology, researchers turned to machine learning-based DGA domain detection methods, training models to learn the differences between DGA domains and normal domains. To improve detection performance, feature engineering was incorporated into machine learning-based DGA domain detection. In addition to the domain's inherent characteristics, researchers considered other DGA-related features, such as DNS query patterns and network traffic characteristics. With the rise of deep learning technology, more and more research is attempting to use neural network models to detect DGA domains.

[0006] However, unlike traditional DGA domains generated by random characters or algorithms, dictionary-based DGA domains have character distribution characteristics and "readability" very similar to normal domains, making them more deceptive and covert, and harder to identify as malicious. Existing detection models often perform poorly on dictionary-based DGA domains. Therefore, to address the problems of high similarity between word-based DGA domains and benign domains, difficulty in feature extraction based on expert knowledge, and poor detection performance of existing methods, this invention proposes a deep DGA domain detection method and system based on semantic representation.

[0007] Furthermore, traditional methods for training DGA domain name detection models require a large amount of domain traffic data. On the one hand, there is a lack of publicly available large-scale domain traffic data; on the other hand, collecting domain traffic data is time-consuming, labor-intensive, and involves data privacy protection. Therefore, this invention proposes a semantic representation-based deep DGA domain name detection method and system. This method can generate simulated domain traffic data based on publicly available domain name ranking lists, helping the model automatically extract semantic features from the domain data and improving DGA domain name detection performance without requiring additional traffic collection. Summary of the Invention

[0008] To address the challenges of difficult domain traffic collection, the poor performance of domain segmentation models trained on general corpora, the high similarity between DGA domains and benign domains, the difficulty of feature extraction based on expert knowledge, and the poor detection performance of existing methods, this invention proposes a DGA domain detection method and apparatus based on semantic representation, achieving efficient recognition of DGA domains.

[0009] According to a first aspect of the present disclosure, a DGA domain name detection method based on semantic representation is provided, comprising:

[0010] The domain name to be tested is segmented into words to obtain several sub-words;

[0011] Calculate the word embedding representation for each subword;

[0012] Based on the word embedding representation of all sub-words, the DGA domain name detection result of the domain name to be detected is obtained.

[0013] Furthermore, the domain name to be detected is segmented to obtain several sub-words, including:

[0014] Obtain publicly available rankings of popular and benign domain names;

[0015] A constant C is defined, which is greater than the total number of domain sample domains included in the domain popularity ranking list;

[0016] Obtain the domain ranking r of each domain sample i in the domain popularity ranking list. i ;

[0017] Based on the constant C and the domain name ranking r i Calculate the number of times domain name sample i appears in the expanded sample dataset, f. i To obtain the expanded sample dataset; wherein, the domain name data distribution in the expanded sample dataset conforms to Zipf's law;

[0018] The BPE model was pre-trained based on the expanded sample dataset;

[0019] The pre-trained BPE model is used to segment the domain name to be detected, resulting in several sub-words.

[0020] Further, the calculation of the word embedding representation of each sub-word includes:

[0021] Word segmentation is performed on a dataset constructed from samples of benign domain names to obtain a corpus for training word embedding models;

[0022] The word embedding model is pre-trained based on the corpus.

[0023] Based on the pre-trained word embedding model, the word embedding representation of each sub-word is obtained.

[0024] Furthermore, the word embedding model includes either the CBOW model or the Skip-gram model.

[0025] Further, the DGA domain name detection result of the domain name to be detected, obtained based on the word embedding representation of all sub-words, includes:

[0026] The word embedding representations of all sub-words of the domain name to be detected are sequentially input into the classifier;

[0027] The word embedding representation of the domain name to be detected is classified using a classifier to obtain the DGA domain name detection result of the domain name to be detected.

[0028] Furthermore, the classifier includes a biLSTM model, which is trained on a corpus consisting of benign domain name samples and DGA domain name samples.

[0029] According to a second aspect of the present disclosure, a DGA domain name detection apparatus based on semantic representation is provided, comprising:

[0030] The word segmentation module is used to segment the domain name to be detected into several sub-words.

[0031] The word embedding module is used to compute the word embedding representation of each subword;

[0032] A classifier is used to obtain the DGA domain name detection result of the domain name to be detected based on the word embedding representation of all sub-words.

[0033] According to a third aspect of the present disclosure, a computer device is provided, the computer device comprising: a processor and a memory storing computer program instructions; wherein the processor, when executing the computer program instructions, implements the semantic representation-based DGA domain name detection method described above.

[0034] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having computer program instructions stored thereon, which, when executed by a processor, implement the semantic representation-based DGA domain name detection method described in any of the preceding claims.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] 1. This invention proposes a BPE domain name segmentation method based on extended corpus for domain name data preprocessing. Compared with traditional BPE segmentation methods, this method is data-oriented, requiring no traffic collection to train a high-quality BPE segmentation model. It effectively solves the problems of difficulty in obtaining large-scale domain name traffic data and high collection thresholds, as well as the poor performance of BPE segmentation based on small-scale traffic data.

[0037] 2. Based on the aforementioned BPE domain name segmentation method using extended corpus, this invention integrates skipgram feature representation for semantic information capture and contextual information acquisition, and employs biLSTM for automatic feature extraction to complete domain name detection. This better preserves the relationship between second-level domains and top-level domains. This combination effectively improves the recognition of DGA domains, especially word DGA domains, by the entire domain name detection model. The aforementioned BPE domain name segmentation method based on extended corpus can effectively help subsequent word embedding and classification models extract and retain the semantic features of benign domains, improving the overall model's ability to recognize DGA domains, especially word DGA domains.

[0038] 3. The DGA domain name detection performance of the method presented in this paper has reached the current leading level. It has fewer model parameters, a simpler model structure, and a shorter model training time than the world's leading benchmark model Bilbo, while achieving higher precision, recall, and F1 score, which are 3.83%, 6.84%, and 5.33% higher, respectively. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating a DGA domain name detection method based on semantic representation proposed in an embodiment of this application. Detailed Implementation

[0040] To make the objectives, solutions, and advantages of this invention clearer, the invention will be further described in detail using experiments conducted on real fish microfossils as an example. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0041] This invention addresses the difficulty in detecting word-based DGA domains by proposing a semantic representation-based DGA domain detection method. Figure 1 As shown, it mainly includes: a word segmentation module, a word embedding module, and a classifier. Compared with existing models, this invention uses a method of expanding the domain name corpus to train the BPE word segmentation algorithm, so as to better preserve the word structure and semantic features within the domain name. It also combines this with the Skip-gram model to perform word embedding on the domain name to capture contextual information and reduce data dimensionality. Finally, it uses biLSTM to extract deep sequence features of the correlation between word fragments in the domain name, achieving efficient recognition of DGA domain names. Specifically, it includes the following steps:

[0042] This invention collects benign domains and DGA domains as sample datasets in experiments. The benign domain samples are from the top 10,000 most popular websites in the massive-million ranking; the DGA domain samples are from the 2020-06-19 version of the DGA domain dataset provided by DGArchive (data up to 2019), from which 15 different types of DGA domains were selected, including the Worldlist-based DGA family with character distributions very similar to benign domains, and other DGA families with character distributions significantly different from benign domains. Then, the sample datasets are preprocessed, and divided in an 8:2 ratio, with 80% used as the training set and 20% as the test set.

[0043] The word segmentation module of this invention uses the Byte-Pair encoding (BPE) algorithm to replace the previous method of directly using domain names as input data for DGA domain name detection. This avoids the problems of an excessively large vocabulary and poor processing ability for unknown and rare domain names. In the BPE encoding process, each character is initially treated as a separate symbol. Then, based on the frequency statistics of the character sequence, the most frequent character pairs are merged into a new symbol, and the frequency statistics of the character sequence are updated. This process is repeated until a predetermined number of iterations is reached or a certain stopping condition is met. In this way, BPE can break down complex words and phrases in text data into smaller sub-word units as input to the model. As a statistical word segmentation method, BPE can extract a limited number of frequently occurring sub-words, and unlike n-gram sub-words whose length is not fixed, it can better preserve the semantic features within words. Furthermore, compared to n-gram, the size of the vocabulary grows linearly with the number of training iterations, rather than exponentially. The word segmentation method is as follows: First, the top 10,000 domain names in the massive-million dataset are used as the corpus to pre-train the BPE model. The corpus expansion method for the dataset containing k domain names (k = 10,000) is as follows: the relationship between the domain name ranking and its frequency of occurrence in the expanded corpus is set as shown in formula (1).

[0044]

[0045] Where, r i For domain ranking, f iA new corpus, corpus2, is generated based on the frequency of domain name occurrences (e.g., the top-ranked domain "google.com" has a domain ranking of r1 and a frequency of f1, where f1*r1 = C, and the constant C = 1,000,000 is set in this paper). The expanded domain name data distribution conforms to Zipf's law. Then, a pre-trained BPE model is used to segment the domain name data. For example, the segmentation result of the BPE model for the top-ranked domain "google.com" is: ['google', '.', 'com']. The segmentation result of the BPE model for the 10,000th-ranked domain "danielpiedrabuena.com" is: ['da', 'ni', 'el', 'pi', 'ed', 'ra', 'bu', 'en', 'a', '.', 'com']. Google, with its rich semantic content, was preserved as a whole, while lower-ranking domains were broken down into smaller sub-words to narrow the vocabulary and improve the processing capabilities for unknown and rare domains. Training the BPE model on a corpus generated by expanding the list of good domains using this method significantly improves the segmentation results, helping to retain more semantic information while narrowing the vocabulary and effectively enhancing the subsequent classification model's ability to recognize DGA domains.

[0046] The word embedding module is used to map words to a high-dimensional vector space for feature extraction. It captures the semantic and contextual relationships between words by mapping discrete text features (such as words) to a low-dimensional continuous feature space representation. The word embedding model needs to be trained on a large-scale corpus to generate a corresponding vector representation for each word. These vectors can serve as feature vectors for input data and are used to build machine learning or deep learning models. Therefore, compared to discrete encodings such as one-hot encoding, word embedding can generate denser feature vectors, achieving dimensionality reduction, which helps reduce the computational complexity of the model and improves its generalization ability. To improve the model's generalization ability, this invention employs the Skip-gram model and the CBOW model, respectively, and pre-trains the word embedding model on the full dataset of the massive-million dataset, based on the corpus after BPE word segmentation.

[0047] The classifier is used to classify input data into different categories or labels. The classifier can be trained based on known features and labels in the dataset and classify new unlabeled data based on this learned knowledge. LSTM (Long Short-Term Memory) is a variant of Recurrent Neural Network (RNN). By adding memory units, input gates, forget gates, and output gates, it solves the gradient vanishing problem faced by traditional RNNs, thus better learning long-term dependencies between sequences. In the analysis of DGA domain name data, this invention found a strong correlation between the top-level domain and the DGA domain name family, so the inverse dependency relationship of domain names is still important for the judgment of DGA domain names. Therefore, the classifier used in this invention is a Bidirectional Long Short-Term Memory (BiLSTM) network. biLSTM extends LSTM by considering both forward and backward context information of the sequence data, thus taking into account both forward and backward sequence dependencies of domain names. Specifically, it employs two independent LSTM layers: one processing the sequence forward and the other backward, capturing past and future context information respectively. Then, the forward and backward LSTM layers process the input sequence step-by-step from beginning to end and from end to beginning, respectively, outputting hidden state sequences in both directions. Finally, these two hidden states can be concatenated or merged for further task processing.

[0048] The baseline model for this experiment is the Bilbo model proposed by Highnam et al. This model, also designed for domain name detection using dictionary-based DGA variants, achieves state-of-the-art (SOTA) performance in DGA domain name detection using a dictionary generation method through the parallel application of CNN and LSTM models.

[0049] Bilbo BP-BiND Precision 0.9378 0.9761 Recall 0.9281 0.9965 F1_score 0.9327 0.9860

[0050] Table 1 Comparison of detection results between the present invention model (BP-BiND) and the Bilbo model

[0051] In summary, this invention proposes a domain name corpus expansion method. The frequency-rank distribution of the corpus conforms to Zipf's law, and BPE segmentation on the expanded corpus can better preserve the semantic features within the domain name words.

[0052] Furthermore, previous DGA domain name detection studies have encountered problems such as excessively large vocabularies and poor processing capabilities for unknown and rare domain names. This invention uses the BPE algorithm for domain name segmentation, which avoids this problem by splitting complex words and phrases in text data into smaller sub-word units.

[0053] Furthermore, to improve the generalization ability of the model, this paper selects Skip-gram and pre-trains the word embedding model on the full dataset of the majestic-million dataset based on the corpus after BPE word segmentation.

[0054] Finally, this invention found a strong correlation between top-level domains and the DGA domain family. The reverse dependency relationship of domains is still important for the judgment of DGA domains. Therefore, after using BPE segmentation and word embedding, in order to realize the forward and reverse sequence dependencies of domains, this paper uses BiLSMT to automatically extract and classify domain features.

[0055] The above description is merely one embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A DGA domain name detection method based on semantic representation, characterized in that, The method includes: The domain name to be tested is segmented into words to obtain several sub-words; Calculate the word embedding representation for each subword; Based on the word embedding representation of all sub-words, the DGA domain name detection result of the domain name to be detected is obtained; The process of segmenting the domain name to be detected to obtain several sub-words includes: Obtain publicly available rankings of popular and benign domain names; Set a constant The constant Greater than the total number of domain samples included in the domain popularity ranking list; Get samples for each domain Domain ranking in the aforementioned domain popularity ranking list ; Based on the constant and the domain name ranking Calculate domain name samples Number of domain names in the expanded sample dataset To obtain the expanded sample dataset; wherein, the domain name data distribution in the expanded sample dataset conforms to Zipf's law; The BPE model was pre-trained based on the expanded sample dataset; The pre-trained BPE model is used to segment the domain name to be detected, resulting in several sub-words.

2. The method as described in claim 1, characterized in that, The calculation of the word embedding representation for each sub-word includes: Word segmentation is performed on a dataset constructed from samples of benign domain names to obtain a corpus for training word embedding models; The word embedding model is pre-trained based on the corpus. Based on the pre-trained word embedding model, the word embedding representation of each sub-word is obtained.

3. The method as described in claim 2, characterized in that, The word embedding model includes either the CBOW model or the Skip-gram model.

4. The method as described in claim 1, characterized in that, The word embedding representation based on all sub-words yields the DGA domain detection result for the domain to be detected, including: The word embedding representations of all sub-words of the domain name to be detected are sequentially input into the classifier; The word embedding representation of the domain name to be detected is classified using a classifier to obtain the DGA domain name detection result of the domain name to be detected.

5. The method as described in claim 4, characterized in that, The classifier includes a biLSTM model, which is trained on a corpus consisting of benign domain name samples and DGA domain name samples.

6. A DGA domain name detection device based on semantic representation, characterized in that, The device includes: The word segmentation module is used to segment the domain name to be detected into several sub-words. The word embedding module is used to compute the word embedding representation of each subword; A classifier is used to obtain the DGA domain name detection result of the domain name to be detected based on the word embedding representation of all sub-words; The process of segmenting the domain name to be detected to obtain several sub-words includes: Obtain publicly available rankings of popular and benign domain names; Set a constant The constant Greater than the total number of domain samples included in the domain popularity ranking list; Get samples for each domain Domain ranking in the aforementioned domain popularity ranking list ; Based on the constant and the domain name ranking Calculate domain name samples Number of domain names in the expanded sample dataset To obtain the expanded sample dataset; wherein, the domain name data distribution in the expanded sample dataset conforms to Zipf's law; The BPE model was pre-trained based on the expanded sample dataset; The pre-trained BPE model is used to segment the domain name to be detected, resulting in several sub-words.

7. A computer device, characterized in that, The computer device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the DGA domain name detection method based on semantic representation as described in any one of claims 1-5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the semantic representation-based DGA domain name detection method as described in any one of claims 1 to 5.

Images