A DGA domain name detection method and system based on deep support vector data description

Abstract

The present invention discloses a DGA domain name detection method and system based on deep support vector data description, which relates to the field of network security. In order to solve the problem that the existing methods for detecting unknown DGA families are single and have a low detection rate, the present invention first obtains an unresolvable domain name from real DNS traffic as a domain name to be detected, then extracts a feature vector for the unresolvable domain name, and finally inputs the feature vector into a deep support vector data description algorithm model to determine whether each unresolvable domain name is a DGA domain name.

Description

Technical Field

[0001] The present invention relates to the field of network security, and in particular to a DGA domain name detection method and system based on deep support vector data description. Background Art

[0002] Botnets are composed of zombie hosts infected with malware and remotely controlled by a botmaster. The botmaster sends instructions to the infected zombie hosts through the command and control (C&C) server to perform malicious activities. In order to evade detection, botmasters use fast-flux technology to quickly generate a large number of domain names. The domain generation algorithm (DGA) can automatically generate a large number of domain names, which is an important means to implement fast-flux technology. The domain name generated by the domain generation algorithm is called DGA domain name. DGA domain names have been used in many attack scenarios. As long as the command and control server and the infected machine use the same DGA and seed (the same DGA family), the same domain name list can be generated to establish a connection. The same domain generation algorithm can be regarded as the same DGA family. Usually, a large number of DGA domain names can be generated in a malicious behavior using DGA technology, but only a few can be resolved to the C&C server, so most DGA domain names are non-existent domain names (NXDomain). Analyzing unresolvable domain names in traffic is the main way to detect DGA activities and discover new DGA families.

[0003] DGA domain name detection methods can be divided into machine learning-based detection methods and deep learning-based detection methods. Since the DGA domain name string generated by the domain name generation algorithm is highly random, the machine learning-based detection method can achieve good results under the premise of good feature design. However, the machine learning-based detection method is less effective for some unknown DGA domain names. The deep learning-based detection method can automatically learn the potential semantic features in the domain name string without manual feature extraction, and has achieved good results in detecting DGA family domain names.

[0004] 1. Detection method based on machine learning:

[0005] Schüppen et al. (see: Schüppen S, Teubert D, et al. FANCI: feature-based automated NXDomain classification and intelligence. 2018.) identified DGA domain names by monitoring unresolvable domain names (NXDomain) in DNS traffic. They extracted structural, linguistic and statistical features of domain names from domain name strings and used support vector machines (SVM) and random forests (RF) to build classifiers. Finally, they evaluated the method on university campus networks and large corporate internal networks, verifying that the method can have high classification accuracy at a low false positive rate. Antonakakis et al. (see: Antonakakis M, Perdisci R, Nadji Y, et al. From Throw-Away Traffic to Bots: Detecting the Rise of DGA-Based Malware [C] / / Usenix Conference on Security Symposium. 2012.) designed the Pleiades system, including a DGA discovery module and a DGA classification module. The purpose of the DGA discovery module is to discover unknown DGA families. They first cluster domain names based on their similarities and the users who query them; then use a classifier based on an alternating decision tree to classify each cluster. The DGA classification module receives traffic from active domain names and is used to detect active DGA domain names and active C&C servers. Drichel et al. (see: Drichel A, Faerber N, Meyer U. First Step Towards EXPLAINable DGA Multiclass Classification [C] / / ARES 2021: The 16th International Conference on Availability, Reliability and Security. 2021.) proposed a multi-class classifier for real-time detection of DGAs with high interpretability. They summarized 136 domain name features from previous work and used a variety of feature refinement algorithms for feature engineering, achieving good results while using fewer features.

[0006] (II) Detection method based on deep learning:

[0007] Anderson et al. (see: Anderson HS, Woodbridge J, Filar B. DeepDGA: Adversarially-Tuned Domain Generation and Detection [J]. ACM, 2016.) Considering that the training data sets of some DGA families are very limited and there are many DGA variants, they use generative adversarial networks to construct a deep learning-based DGA domain name detection method. During the learning process, the generator learns to generate domain names that are increasingly difficult to detect, and the detector model updates its parameters to compensate for unfavorable generated domain names. Ren et al. (see: Ren F, Jiang Z, Wang X, et al. A DGAdomain names detection modeling method based on integrating an attention mechanism and deep neural network [J]. Cyberspace Security Science and Technology (English), 2018, 1 (1): 13.) Considering that the existing methods are not sufficient to deal with the DGA threat based on the word list, Convolutional Neural Networks (CNN) and Bi-directional Long Short-term Memory (BiLSTM) neural network layers are used to extract the features of domain sequence information; secondly, the attention layer is used to assign the corresponding weights of the deep information extracted from the domain name; finally, the features with different weights in the domain name are put into the output layer to complete the detection and classification tasks. Ravi et al. (see: Ravi V, Alazab M, Srinivasan S, et al. Adversarial Defense: DGA-Based Botnets and DNS Homographs Detection Through Integrated Deep Learning [J]. IEEE Transactions on Engineering Management, 2021, PP (99): 1-18.) proposed a two-level DNS traffic analysis framework based on deep learning. The first level of the framework adopts character-level embedding and uses Siamese neural networks to detect the similarity between domain names; the second level takes different domain names as input of the previous level and adopts a cost-sensitive deep learning model to detect and classify DGA domain names.

[0008] Existing technologies have good detection effects on known DGA family domain names, but most existing works use DGA domain name detection technology based on supervised learning, which cannot detect when unknown DGA family domain names appear. Existing methods for detecting unknown DGA family domain names cluster unknown DGA family domain names and then filter them to obtain results. This method requires changing the threshold according to different actual network scenarios, and when the number of unknown DGA family domain names is small and insufficient to form a cluster, the detection rate will become very low. Summary of the invention

[0009] The purpose of the present invention is to provide a DGA domain name detection method and system based on deep support vector data description to solve the problem that the existing methods for detecting unknown DGA families are single and the detection rate is low. The present invention first obtains unresolvable domain names from real DNS traffic as domain names to be detected, then extracts feature vectors for unresolvable domain names, and finally inputs the feature vectors into the deep support vector data description algorithm model to determine whether each unresolvable domain name is a DGA domain name.

[0010] To achieve the above object, the present invention adopts the following technical solutions:

[0011] A DGA domain name detection method based on deep support vector data description includes the following steps:

[0012] Get the real DNS traffic and obtain the unresolvable domain names from the real DNS traffic;

[0013] Perform feature extraction on each unresolvable domain name, extract language features, structural features and statistical features, and form a feature vector for each unresolvable domain name;

[0014] Use the public domain name whitelist Alexa to mark the unresolvable domain names, obtain the known benign domain names among the unresolvable domain names, and use other domain names except the known benign domain names as the domain names to be detected;

[0015] A deep support vector data description classification model based on a convolutional neural mapping network is constructed. All known benign domain names are input as training data into the convolutional neural mapping network of the model for training. The convolutional neural mapping network maps all known benign domain names to a new feature space. The network parameters are adjusted through multiple rounds of training. When all known benign domain names fall within the hypersphere, the trained convolutional neural mapping network, as well as the center position and radius of the hypersphere are obtained.

[0016] The feature vector of each unresolvable domain name to be detected is input into the trained convolutional neural mapping network of the deep support vector data description classification model. The convolutional neural mapping network determines the distance between each unresolvable domain name and the center of the hypersphere. If it is within the radius of the hypersphere, it is a benign domain name, otherwise it is a DGA domain name.

[0017] Furthermore, the method for obtaining the unresolvable domain name is: querying the resource record of each domain name in the real DNS traffic, if there is no resource record, the domain name is an unresolvable domain name.

[0018] Furthermore, the language features include: whether numeric characters are included, the proportion of vowel characters, the proportion of numeric characters, the number of letter types, the proportion of repeated characters, the proportion of consecutive consonants, the proportion of consecutive numbers, and the length of the longest meaningful continuous substring.

[0019] Furthermore, the structural features include: domain name length, number of subdomains, average length of subdomains, whether there is a www prefix, whether the top-level domain name is valid, whether there is a single character as a subdomain, whether there is a top-level domain name string as a subdomain, the proportion of numbers as subdomains, the proportion of hexadecimal characters as subdomains, the proportion of underscore characters, and whether the IP is included.

[0020] Furthermore, the statistical features include 1-Gram statistics, 2-Gram statistics, 3-Gram statistics and character entropy values.

[0021] Furthermore, the step of training the deep support vector data description classification model includes:

[0022] Initialize all network parameters of the convolutional neural mapping network;

[0023] A convolutional neural mapping network is used to map all known benign domain names input into a new feature space, and the center position of the hypersphere is calculated according to the average center position of all known benign domain names in the new feature space;

[0024] All known benign domain names are shuffled and grouped to obtain multiple batches of training data;

[0025] The training data of each batch is passed through a convolutional neural mapping network to obtain a new mapping space representation, and the distance between each sample of each batch and the center of the hypersphere is calculated based on the new mapping space representation and the center position of the hypersphere.

[0026] Calculate the loss generated by the distance of each sample in each batch from the center of the hypersphere, and update all network parameters of the convolutional neural mapping network through the gradient back propagation of the neural network;

[0027] According to the updated network parameters, the center position of the hypersphere is updated and the radius of the hypersphere is calculated, and the trained convolutional neural mapping network, as well as the center position and radius of the hypersphere are output.

[0028] Furthermore, the structure of the convolutional neural mapping network is: input layer + fully connected layer + fully connected layer + reshape layer + convolution layer + normalization layer + maximum pooling layer + convolution layer + normalization layer + maximum pooling layer + reshape layer + fully connected layer.

[0029] A DGA domain name detection system based on deep support vector data description includes a memory and a processor. A computer program is stored in the memory, and the processor implements the steps of the above method when executing the program.

[0030] The technical effects achieved by the present invention are as follows:

[0031] 1. The method of the present invention does not require any known DGA domain name when detecting DGA domain names. It only needs to know a small number of benign domain names. Compared with other methods that require DGA domain names for supervised training, the requirements are lower and better detection effects can be achieved.

[0032] 2. When the unresolvable domain names in the traffic are filtered by known DGA families in advance, the DGA domain names detected again by the method of the present invention are unknown DGA family domain names, which can be used to discover unknown DGA families.

[0033] 3. The method of the present invention uses a method of clustering benign domain names to detect unknown DGA domain names. Since benign domain names are very easy to obtain in traffic, the method of the present invention is easier to implement than the detection method of clustering DGA domain names. And when the number of potential DGA domain names in traffic is small, the detection method of clustering DGA domain names will be greatly limited, and the method of the present invention can still achieve good performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of a DGA domain name detection method based on deep support vector data description of the present invention.

[0035] Figure 2 It is a training deep support vector data description algorithm model and model workflow diagram.

[0036] Figure 3 This is a performance data chart comparing the number of F1 known benign domain names. DETAILED DESCRIPTION

[0037] In order to make the above features and advantages of the present invention more obvious and easy to understand, embodiments are given below and described in detail with reference to the accompanying drawings.

[0038] The embodiment of the present invention provides a DGA domain name detection method based on deep support vector data description, and its workflow is as follows: Figure 1 The specific instructions are as follows.

[0039] 1) Obtain real DNS traffic: Set up a network probe in the network to obtain real DNS traffic data for several days and store it in a passive traffic database.

[0040] 2) Obtain unresolvable domain names from real DNS traffic: Query the resource records of each domain name in the real DNS traffic in 1). If no resource records exist, it is an unresolvable domain name.

[0041] 3) Feature extraction: Extract features for each unresolvable domain name. The features to be extracted are shown in Table 1. After the extraction is completed, for each unresolvable domain name d i , will have a feature vector v i ={f i1 ,f i2 ,…,f i42}.

[0042] Table 1 Extracted domain name features

[0043]

[0044] 4) Mark benign domain names: Use the public domain name whitelist Alexa (see: Alexa. https: / / aws.amazon.com / alexa-top-sites[M]. On-line Resources, 2021.) to mark the unresolvable domain names, obtain the known benign domain names among the unresolvable domain names, and use the others as domain names to be detected.

[0045] 5) Constructing a deep support vector data description classification model: This model uses a deep support vector data description algorithm, the purpose of which is to map all unresolvable domain names to a new feature space so that the volume of the hypersphere composed of all known benign domain names in the new feature space is as small as possible. The calculation process of the deep support vector data description algorithm is described as follows:

[0046] Algorithm input: number of iterations T, feature dimension of unresolvable domain name status n, convolutional neural mapping network Q used to map to the new feature space, number of domain name samples for batch training m, known benign domain name dataset D.

[0047] Algorithm output: convolutional neural mapping network Q, hypersphere center O, hypersphere radius R.

[0048] a) Randomly initialize all parameters w of the convolutional neural mapping network Q.

[0049] b) Use Q to map all benign domain names to the new feature space Q(w,D), and calculate the hypersphere center O=MEAN(Q(w,D)) according to their average center position, where the MEAN function is the mean function.

[0050] c) Initialize the training round episode = 1.

[0051] d) Shuffle the order of the data set D and divide it into d1, d2, …, d according to each group of m samples K There are K batches of training data in total.

[0052] e) Let i=1.

[0053] f) d i The batch data is passed through the convolutional neural mapping network Q to find the new mapping space representation nd i =Q(w,d i ), and find the distance p of each sample in the batch from the center of the hypersphere i =DIST(nd i ,O), where the DIST(x,y) function finds the distance between x and y.

[0054] g) Calculate loss loss = MEAN(p i ), and update all parameters w of the convolutional neural mapping network Q through the gradient back propagation of the neural network.

[0055] h) Let i=i+1, if i≤K go to step f).

[0056] i) Update the center of the hypersphere O = MEAN(Q(w,D)).

[0057] j) Let episode = episode + 1. If episode ≤ T, return to step d).

[0058] k) Calculate the radius of the hypersphere R = MAX(DIST(Q(w,D),O)). The function of the MAX function is to find the maximum value.

[0059] l) Output convolutional neural mapping network Q, hypersphere center O, hypersphere radius R.

[0060] During the training process, the convolutional neural mapping network Q in the present invention may cause all points to be mapped to one point with a radius of zero. This phenomenon is called hypersphere collapse. The causes and solutions to hypersphere collapse are as follows:

[0061] a) The convolutional neural mapping network Q causes the hypersphere to collapse due to the zero weight solution: the center of the hypersphere cannot be made a free variable unrelated to normal samples. According to experience, the center of the hypersphere can be set to the average position after benign domain name mapping.

[0062] b) The hidden layer bias term can learn a constant function mapping that causes the hypersphere to collapse: the hidden layer does not use the bias term.

[0063] c) Network units with bounded activation functions will simulate bias terms in subsequent layers, causing the hypersphere to collapse: unbounded activation functions such as ReLU (or activation functions that are only bounded by 0) should be used in preference.

[0064] The workflow for training a deep support vector data description classification model is as follows Figure 2 shown.

[0065] 6) Determine whether the unresolvable domain name is a DGA domain name: After the deep support vector data description classification model is constructed, for each unresolvable domain name to be detected, the feature vector only needs to be passed through the convolutional neural mapping network Q to determine its distance from the center of the hypersphere O. If it is within the radius R of the hypersphere, it is a benign domain name, otherwise it is a DGA domain name. Specifically, the feature vector of the domain name to be detected in 4) is input into the deep support vector data description classification model in 5) to determine whether each unresolvable domain name is a DGA domain name.

[0066] A specific example is listed below to illustrate the use of the DGA domain name detection method based on deep support vector data description proposed in the present invention to solve the problem that the existing methods for detecting unknown DGA families are single and have a low detection rate.

[0067] 1) Obtain real DNS traffic: Set up a network probe in the campus network to obtain real DNS traffic for about 7 days.

[0068] 2) Establish blacklists and whitelists: blacklists and whitelists are established through authoritative channels, such as authoritative security websites or security companies, and highly recognized public blacklists and whitelists on the Internet. Specifically, the Alexa website traffic world ranking list TOP100,000 is used to build the whitelist, because the higher the world ranking of traffic, the higher the exposure to the public, and the less likely the domain name is to engage in malicious behavior. The blacklist uses the 360dga public on the Internet to build a DGA domain name blacklist, which includes more than 50 DGA families and millions of DGA domain names, and is always updated.

[0069] 3) Obtain unresolvable domain names from real DNS traffic: For each domain name in the real DNS traffic in 1), determine whether a resource record can be found. If no resource record can be found, it is an unresolvable domain name. Then use the trie tree in 2) to filter the unresolvable domain names to filter out known benign domain names and DGA domain names.

[0070] 4) Feature extraction: All features in Table 1 are extracted from the unresolvable domain names obtained in 3). A 42-dimensional feature vector is obtained for each unresolvable domain name.

[0071] 5) Obtain training set and test set: Take 10% of benign domain names as training set, 90% of benign domain names and all DGA domain names as test set. Here, 10-fold cross validation is used to avoid the influence of different segmentation samples on the results. The training set is used to train the model, and the test set is used to simulate the domain names to be detected.

[0072] 6) Construct a deep support vector data description classification model: set the number of iterations T = 10, the state feature dimension n = 42, the number of samples for batch gradient descent m = 128, and the structure used by the convolutional neural mapping network Q is shown in Table 2.

[0073] Table 2. The convolutional neural mapping network Q structure in the deep support vector data description classification model

[0074]

[0075] 7) Determine whether the domain name in the test set has malicious behavior: F1 is used as the evaluation standard here. For the results of the ten-fold cross validation, the average value of F1 is taken as the final effect of the model.

[0076] Positive effects:

[0077] The following experiments illustrate the positive effects of the method of the present invention. In order to demonstrate the positive effects of the method of the present invention, the detection performance of different single-classification algorithms is first evaluated, then the performance of different DGA domain name detection methods is analyzed, and finally the detection performance of different single-classification algorithms under different numbers of known benign domain names is analyzed.

[0078] 1) Performance of different single classifiers

[0079] The method of the present invention is compared with the single classification algorithm based on machine learning and deep learning, and the results of each detection algorithm are shown in Table 3. It can be seen from Table 3 that the performance of deep support vector data description is the best in terms of the three evaluation indicators of F1, recall rate and precision. It is also found that setting another deep support vector data description algorithm with soft boundary as the learning target limits the shrinkage of the hypersphere to a certain extent.

[0080] Table 3 Performance of different single classifiers

[0081] algorithm F1 Recall Accuracy SVDD 0.9521 0.9690 0.9358 Isolation Forest 0.9520 0.9523 0.9517 DCAE 0.9754 0.9520 0.9999 DCGAN 0.9819 0.9645 0.9999 Deep SVDD (Soft-boundary) 0.9911 0.9829 0.9994 Deep SVDD (One-class) 0.9945 0.9891 0.9999

[0082] 2) Performance of different DGA domain name detection methods

[0083] The method of the present invention is compared with several existing DGA detection methods, and the experimental results are shown in Table 4. It can be seen from Table 4 that the deep support vector data description has the best performance in terms of the three evaluation indicators of F1, recall rate and precision, which are better than the existing mainstream methods for detecting DGA domain names. In addition, our method does not require any known DGA domain names during training, while the training process of other methods requires the inclusion of known DGA domain names.

[0084] Table 4 Performance of different DGA detection methods

[0085] algorithm F1 Recall Accuracy ATT-CNN-BiLSTM 0.8994 0.8902 0.9087 M-LSTM.MI 0.9875 0.9837 0.9913 SNN 0.9879 0.9768 0.9993 GAN 0.9901 0.9809 0.9996 FANCI 0.9913 0.9830 0.9998 Deep SVDD (Soft-boundary) 0.9911 0.9829 0.9994 Deep SVDD (One-class) 0.9945 0.9891 0.9999

[0086] 3) Performance of different numbers of known benign domain names

[0087] The method of the present invention studies the performance of different single-classification algorithms when the number of known benign domain names is reduced. This experiment adjusts the number of known benign domain names by adjusting the proportion of benign domain names in the training set. The experimental results are as follows: Figure 3 As shown in Figure 2, when the proportion of benign domain names in the training set is reduced to 1%, 1 / 1000, or 1 / 10,000, the performance of other algorithms will be severely reduced, while the deep support vector data description can still achieve good performance. This shows that the method of the present invention can still achieve good performance when the number of known benign domain names is small.

[0088] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions of the technical solutions of the present invention made by ordinary technicians in the field should all be included in the protection scope of the present invention. The protection scope of the present invention shall be based on what is defined in the claims.

Claims

1. A DGA domain name detection method based on deep support vector data description, characterized in that: The following steps are involved: Get the real DNS traffic and obtain the unresolvable domain names from the real DNS traffic; Perform feature extraction on each unresolvable domain name, extract language features, structural features and statistical features, and form a feature vector for each unresolvable domain name; Use the public domain name whitelist Alexa to mark the unresolvable domain names, obtain the known benign domain names among the unresolvable domain names, and use other domain names except the known benign domain names as the domain names to be detected; A deep support vector data description classification model based on a convolutional neural mapping network is constructed. All known benign domain names are input as training data into the convolutional neural mapping network of the model for training. The convolutional neural mapping network maps all known benign domain names to a new feature space. The network parameters are adjusted through multiple rounds of training. When all known benign domain names fall within the hypersphere, the trained convolutional neural mapping network, as well as the center position and radius of the hypersphere are obtained. The feature vector of each unresolvable domain name to be detected is input into the trained convolutional neural mapping network of the deep support vector data description classification model. The convolutional neural mapping network determines the distance between each unresolvable domain name and the center of the hypersphere. If it is within the radius of the hypersphere, it is a benign domain name, otherwise it is a DGA domain name.

2. The method according to claim 1, characterized in that The method for obtaining an unresolvable domain name is to query the resource record of each domain name in the real DNS traffic. If no resource record exists, the domain name is an unresolvable domain name.

3. The method according to claim 1, characterized in that The language features include: whether numeric characters are included, the proportion of vowel characters, the proportion of numeric characters, the number of letter types, the proportion of repeated characters, the proportion of consecutive consonants, the proportion of consecutive numbers, and the length of the longest meaningful consecutive substring.

4. The method according to claim 1, characterized in that Structural features include: domain name length, number of subdomains, average length of subdomains, whether there is a www prefix, whether the top-level domain name is valid, whether there is a single character as a subdomain, whether there is a top-level domain name string as a subdomain, the proportion of numbers as subdomains, the proportion of hexadecimal characters as subdomains, the proportion of underscore characters, and whether the IP is included.

5. The method according to claim 1, characterized in that Statistical features include 1-Gram statistics, 2-Gram statistics, 3-Gram statistics and character entropy values.

6. The method according to claim 1, characterized in that The steps for training a deep support vector data description classification model include: Initialize all network parameters of the convolutional neural mapping network; A convolutional neural mapping network is used to map all known benign domain names input into a new feature space, and the center position of the hypersphere is calculated according to the average center position of all known benign domain names in the new feature space; All known benign domain names are shuffled and grouped to obtain multiple batches of training data; The training data of each batch is passed through a convolutional neural mapping network to obtain a new mapping space representation, and the distance between each sample of each batch and the center of the hypersphere is calculated based on the new mapping space representation and the center position of the hypersphere. Calculate the loss generated by the distance of each sample in each batch from the center of the hypersphere, and update all network parameters of the convolutional neural mapping network through the gradient back propagation of the neural network; According to the updated network parameters, the center position of the hypersphere is updated and the radius of the hypersphere is calculated, and the trained convolutional neural mapping network, as well as the center position and radius of the hypersphere are output.

7. The method according to claim 1, characterized in that The structure of the convolutional neural mapping network is: input layer + fully connected layer + fully connected layer + reshape layer + convolution layer + normalization layer + maximum pooling layer + convolution layer + normalization layer + maximum pooling layer + reshape layer + fully connected layer.

8. A DGA domain name detection system based on deep support vector data description, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images