Algorithm generation domain name detection method based on flow analysis

Abstract

The method comprises the following steps: firstly, collecting flow information of an algorithm generated domain name (DGA domain name) and a benign domain name of a known classification label, wherein the flow information comprises an analysis IP address and Flow data of the domain name; performing primary processing on the collected flow data, counting the occurrence frequency of each resolution IP of a domain name, and screening out information such as a source address, a destination address and time from the Flow data; calculating eight characteristic values of the domain name according to the flow information, wherein the eight characteristic values comprise the user IP access number, the active frequency, the standard variance mean value ratio of the number of times of occurrence of each analysis IP and the like; putting the feature data into a C4.5 supervised classification model for training to obtain a classification model with classification capability; and finally, collecting flow information of to-be-classified domain names, calculating feature data, and putting the feature data into the trained classification model for classification. According to the method, the difference between the flow behavior expression and the benign domain name when the domain name is generated by an algorithm to carry out malicious activities is utilized, the two types of domain names are effectively distinguished, the antagonistic attack is resisted, and the security of the network is maintained.

Description

Technical Field

[0001] The present invention relates to a technology of traffic analysis, feature extraction and machine learning knowledge, which is based on the performance difference between algorithm-generated domain names and benign domain names in network traffic behavior, extracts key features with obvious differences from the traffic, and uses a machine learning model to realize domain name classification, belonging to the fields of traffic analysis, artificial intelligence and network security technology. Background Art

[0002] With the popularization of the Internet, network security issues have become increasingly prominent, among which network security issues related to the domain name system have become increasingly severe. The domain name system is an important infrastructure that is closely related to various network activities, such as email, website services, and instant messaging. By taking advantage of loopholes in the early design of the Internet, domain name resolution services have become a tool for various network security threats, such as attackers using DNS technology to locate C&C servers.

[0003] Malicious clients randomly generate a large number of candidate domain names through the domain generation algorithm (DGA) and query them. Attackers and malware run the same DGA algorithm to generate the same list of candidate domain names. When they need to launch an attack, they select a small number of them for registration, and then they can establish communication and carry out attacks. In addition, attackers can also apply IP-Flux technology to registered domain names to quickly change IP addresses, making traditional protection measures based on blacklists ineffective. Therefore, the use of algorithm-generated domain names (DGA domain names) makes attacks easy and defense difficult. In order to resist network attacks based on DGA domain names, it is very important to detect DGA domain names in a timely manner to prevent attackers from causing more security threats.

[0004] In recent years, many excellent solutions have been proposed for DGA domain name detection, among which there are many studies focusing on the name character features of the domain name itself. For example, the differences in language features and statistical features, pronunciation and spelling between DGA domain names and benign domain names are analyzed, and the technologies used include machine learning, neural networks, etc., which can quickly detect DGA domain names. However, most of these methods have a common shortcoming, that is, the detection of domain names is too focused on the name character information of the domain name, and the actual activity information of the domain name is not involved. If DGA classifiers rely solely on domain name strings to make decisions, they are very vulnerable to adversarial attacks. Therefore, the accuracy of methods that only target name character features may be slightly insufficient, and the malicious domain names detected may not actually carry out malicious activities.

[0005] In the present invention, based on the behavioral differences between DGA domain names and benign domain names in actual network traffic, we collect key information of domain names in DNS traffic and Flow traffic, calculate 8 features that can represent the differences between the two types of domain names, and put them into the C4.5 classification model for classification. The C4.5 classification model is a supervised machine learning classification model, so a certain amount of domain name data with known labels is required for training in the early stage. This scheme is based on the traffic performance of DGA domain names in the actual network, which is carried out in the process of implementing network attacks, so it is difficult to carry out adversarial attacks against this scheme. At the same time, the detected DGA domain names must be domain names that have carried out malicious activities, resulting in the appearance of corresponding traffic characteristics, which improves the accuracy of detection and effectively maintains the security of the network. Summary of the invention

[0006] The algorithm-generated domain name detection method based on traffic analysis proposed in the present invention is driven by the accuracy and long-term effectiveness of detection, extracts relevant features from network traffic, and puts them into the C4.5 classification model for detection and classification. It mainly solves the problem of effective detection of algorithm-generated domain names under the domain name generation algorithm and IP-Flux mechanism. The domain name generation algorithm can randomly generate a large number of domain names based on various rules such as alphabets, dictionaries, and phonetic syllables, and IP-Flux can make the resolved IP address corresponding to the domain name constantly change, so the traditional blacklist scheme and the detection scheme based on the domain name character have obvious shortcomings. The algorithm-generated domain name detection scheme based on traffic features can achieve high accuracy and effectiveness, bringing great benefits to network security. In the training model stage, a batch of benign domain names and algorithm-generated domain names with known classification labels are collected, and the domain names that have appeared in the detection network segment are screened out, and their DNS traffic and Flow traffic are collected. The resolved IP address of the domain name and the interactive information with the user IP address are extracted from them, and the values ​​of 8 features are calculated. Put them into the C4.5 model for training to obtain a classification model with classification ability. In the domain name detection phase, traffic with the detected domain name is collected from the detection network segment, and the feature value is calculated in the same way as the training set domain name. The feature value is put into the C4.5 classification model for detection, and the domain names determined to be algorithm-generated domain names are screened out.

[0007] The present invention makes full use of the difference in traffic behavior between algorithm-generated domain names and benign domain names in the network, making it difficult for attackers to carry out adversarial attacks on the scheme. At the same time, the domain names screened out are generally currently active domain names, which improves the accuracy and effectiveness of detection.

[0008] Technical solution: In order to achieve the above purpose, the technical solution of the present invention is as follows. The process of the algorithm-generated domain name detection method of the present invention is as follows:

[0009] Step 1) Collect algorithm-generated domain names (DGA domain names) and benign domain names with known classification labels as training sets for the model;

[0010] Step 2) Collect the traffic information of the training set domain name from the detection network segment, including the domain name's resolved IP address related information (from DNS traffic) and Flow information;

[0011] Step 3) Perform preliminary processing on the collected traffic information, including counting the number of occurrences of each resolved IP address of the domain name, and filtering out data such as the source IP address, destination IP address, specific resolution time, and resolution hour value from the flow information;

[0012] Step 4) Calculate and process the processed flow information to obtain 8 flow characteristic values;

[0013] Step 5) Put the feature values ​​and classification labels of all training set domain names into the C4.5 classification model for training to obtain a model with classification capabilities;

[0014] Step 6) extracting the traffic information of the domain name to be classified and calculating its characteristic value;

[0015] Step 7) Put the feature value of the domain name to be classified into the classification model for classification to obtain the classification result.

[0016] Among them, step 1) collects algorithm-generated domain names (DGA domain names) and benign domain names with known classification labels as the training set of the model. The DGA domain names come from the DGA domain name dataset provided by open source websites, including 360netlab, bambenek, etc. The bambenek database will update the recently detected DGA domain names every day, adding about 500,000 entries each time, and will also be accompanied by the time when the DGA domain name was first detected and the malware information to which it belongs. Benign domain names are domain names with the highest number of visits in the detection network segment. Many large networks will maintain a top 1 million domain name set, indicating that this is a domain name with the highest number of visits and relatively stable in the network segment, which can be regarded as a benign domain name. The specific steps are as follows:

[0017] Step 1-1) Collect the DGA domain name dataset provided by the open source website. To ensure its timeliness, the domain name must have appeared within 1 year.

[0018] Step 1-2) Obtain the top 1 million information in the detection network segment. If there is no information, count the visits of all domain names in the detection network segment. In order to ensure that the training set covers all types of benign domain names, take the top 1,000 domain names, the middle 1,000 domain names, and the last 1,000 domain names as the source of the benign domain name training set;

[0019] Step 1-3) performs a matching search in the detection network segment based on the DGA domain name and the benign domain name obtained in step 1-1) and step 1-2), and uses the domain name appearing in the detection network segment as a training set.

[0020] Step 2) Collect the traffic information of the training set domain name from the detection network segment, including the resolved IP address and flow information of the domain name, as follows:

[0021] Step 2-1) The DGA domain name training set and the benign domain name training set are respectively put into the DNS traffic of the detection network segment for matching and searching. The search period is two consecutive days (48 hours). The search method is to see whether the domain name corresponds to the domain name resolved by the DNS request. If they are the same, it means that the domain name appears in the network segment, and the resolved IP address of the domain name is taken from the resolution information;

[0022] Step 2-2) Put all resolved IP addresses of the DGA domain name training set and the benign domain name training set into the flow traffic of the detection network segment for matching and searching. The search period is two consecutive days (48 hours). The search method is to see whether the IP address is the same as the source IP address or the destination IP address in the flow traffic. If they are the same, it means that the IP address has interacted with the user's IP address and can be regarded as a valid interaction information.

[0023] Step 3) Perform preliminary processing on the collected traffic information, including counting the number of occurrences of each resolved IP address, and filtering out data such as source IP address, destination IP address, specific resolution time, and resolution hour value from the flow information, as follows:

[0024] Step 3-1) Based on the training set domain name resolution information obtained in step 2-1), all resolved IP addresses of each domain name and the number of occurrences of each resolved IP address in the two days before and after are counted;

[0025] Step 3-2) Based on the training set domain name flow information obtained in step 2-2), five fields including source IP address, destination IP address, first routing forwarding time, last routing forwarding time, and hour are filtered out from each piece of information.

[0026] Step 4) Calculate and process the processed flow information to obtain 8 flow characteristic values, as follows:

[0027] Step 4-1) Calculate the number of user IP address visits to the domain name based on the number of occurrences of each resolved IP address the next day. Generally speaking, since the botnet is generally large in scale and controls a large number of bot hosts, the C&C server using the DGA domain name has a large number of visits, resulting in a large number of user IP address visits to the DGA domain name. The specific calculation method is: client_IP_visit=∑IP i _resolved, where client_IP_visit indicates the number of times a user's IP address visits a domain name. i _resolved indicates the number of times the ith resolved IP address of the domain name is resolved in the DNS traffic. The number of times all resolved IP addresses of the domain name are resolved in the DNS traffic is used to represent the number of user IP address visits. The larger the client_IP_visit value is, the more times the user IP address has visited. It is taken as the feature value 1.

[0028] Step 4-2) Calculate the life span of the domain name based on the source IP address, destination IP address, first routing forwarding time, and last routing forwarding time in the flow information of the previous and next two days. Due to the existence of various defense mechanisms in computer networks, various malicious domain names including DGA domain names will eventually be discovered and blocked, so the life span of DGA domain names will be shorter than that of benign domain names. The specific calculation method is: life_span = last_seen_time-first_seen_time, where life_span represents the life span of the domain name, last_seen_time represents the last appearance time of the domain name in the detection network segment, and first_seen_time represents the first appearance time of the domain name in the detection network segment. Both last_seen_time and first_seen_time are in seconds, and the difference between the two is used to represent the life span of the domain name. The larger the life_span value, the longer the life span.

[0029] Take it as eigenvalue 2;

[0030] Step 4-3) Calculate the active frequency of the domain name based on the source IP address, destination IP address, first routing forwarding time, and last routing forwarding time in the flow information of the previous and next two days. Since the zombie host will centrally access the C&C server using the DGA domain name in a short period of time according to the instructions, and the life cycle of the DGA domain name itself is relatively short, its active frequency is higher than that of the benign domain name that is independently and intermittently accessed by the user for a long time. The specific calculation method is: Among them, live_freq represents the active frequency of the domain name, life_span represents the life cycle of the domain name, client_IP_visit(h) represents the number of visits to the domain name by the user IP address in the h time period, and the active frequency is represented by the proportion of the time period when the domain name is visited by the user IP address in the entire life cycle. The larger the live_freq value, the higher the active frequency, which is taken as the feature value 3;

[0031] Step 4-4) Calculate the periodicity of the domain name based on the source IP address, destination IP address, and hour information in the flow information of the previous and next two days. For zombie hosts, since commands are regularly issued through the C&C server, the behavior of zombie hosts has very strong synchronization, which is reflected in the domain name, that is, the access to the DGA domain name has a very strong periodicity. The specific calculation method is: Among them, periodic represents the periodicity value of the domain name, client_IP_visit(h) represents the number of visits to the domain name by the user IP address in the h time period, and the ratio of the Euler distance of the time series curve of the number of visits by the user IP address in the two time periods before and after the domain name to the relative distance of the curve mean is used to represent the periodicity. The smaller the periodic value, the stronger the periodicity, which is taken as the characteristic value 4;

[0032] Step 4-5) Calculate the user access burstiness of the domain name based on the source IP address, destination IP address, and hour information in the flow information of the next day. After the C&C server issues a command, a large number of zombie hosts controlled by it will synchronously access the DGA domain name, causing the DGA domain name to have a higher user access burstiness than benign domain names. The specific calculation method is: Where abrupt represents the user access burst value of the domain name, client_IP_visit(T, life_span) represents the number of visits by the user IP address in a fixed time period T during the life cycle of the domain name, Avg represents the mean, Max represents the maximum value, and the relative distance between the mean and the maximum value in the user IP address visit time series curve represents the user access burst. The larger the abrupt value, the higher the user access burst, which is taken as the feature value 5;

[0033] Step 4-6) Calculate the standard deviation mean ratio of the number of occurrences of each resolved IP address of the domain name based on the number of occurrences of each resolved IP address on the next day. DGA domain names will use IP-Flux technology to frequently change the resolved IP address. Since its Flux is a random polling mechanism, the probability of each resolved IP address being polled is equivalent, that is, the number of occurrences of each resolved IP address is equivalent. Benign domain names generally do not change the resolved IP address at will. For CDN domain names with load balancing requirements, the polling mechanism of the CDN nodes is the principle of proximity, so the probability of the server IP address close to the user group being polled will be greater, and the number of occurrences of each resolved IP address will vary, resulting in a larger variance. The specific calculation method is: Among them, IP_var represents the standard deviation mean ratio of the number of occurrences of each resolved IP address of the domain name, var(IP_appear_times) represents the variance of the number of occurrences of each resolved IP address of the domain name, and avg(IP_appear_times) represents the average number of occurrences of each resolved IP address of the domain name. Their ratio is used to represent the standard deviation mean ratio of the number of occurrences of each resolved IP address. The larger the IP_var value, the larger the standard deviation mean ratio of the number of occurrences of the resolved IP address, which is used as the eigenvalue 6;

[0034] Step 4-7) Calculate the total number of domain names reverse mapped by the resolved IP based on the corresponding relationship between the domain name and the resolved IP on the next day. Attackers usually use Domain-Flux technology to avoid the problem of single point failure of domain names. Therefore, the resolved IP address of the C&C server will be mapped to a set of DGA domain names. The mapping relationship between benign domain names and resolved IP addresses generally does not change, and the CDN node server in the CDN domain name may run multiple services on the same server to improve utilization, but the resolved IP address set of the CDN node server usually belongs to the same or a few secondary domain names. The specific calculation method is: IP_domain_num = |{d(IP)|IP∈resolvedIPset(W)}|, where IP_domain_num represents the total number of domain names reverse mapped by all resolved IP addresses of the domain name, d(IP) represents the domain name reverse mapped by the IP address, and resolvedIPset(W) represents the set of all resolved IP addresses of the domain name in the current time window. The larger the IP_domain_num value, the more total number of domain names reverse mapped by the resolved IP address, and it is used as the characteristic value 7;

[0035] Step 4-8) Calculate the difference between the resolved IP address set based on the correspondence between the domain name and the resolved IP two days ago. In order to avoid security defense mechanisms such as IP address blacklists, attackers usually use IP-Flux technology. The resolved IP addresses of botnet domain names will change frequently, resulting in a large difference between the resolved IP address sets in the two time windows. Although the CDN node servers in the benign domain names will also have several resolved IP addresses, these safe IP addresses will survive stably for a long time and will not change frequently. Therefore, the difference between the resolved IP address sets of benign domain names will not be too large. The specific calculation method is: Among them, IP_diff represents the difference of the resolved IP address set, resolvedIPset(W) represents the set of all resolved IP addresses of the domain name in the current time window, and resolvedIPset(W') represents the set of all resolved IP addresses of the domain name in the previous time window. The ratio of their intersection and union is used to calculate the difference of the resolved IP address set of the domain name. The larger the IP_diff value, the greater the difference of the resolved IP address set, and it is used as the feature value 8.

[0036] Step 5) Put the feature values ​​and classification labels of all training set domain names into the C4.5 classification model for training to obtain a model with classification capabilities. The C4.5 model is a relatively mature supervised machine learning classification model and can be used directly. The details are as follows:

[0037] Step 5-1) For each domain name in the training set, first retrieve its related resolved IP address and its occurrence times in the collected DNS traffic information database according to its domain name name, then use the resolved IP address to find the number of user IP addresses, interaction time and other information from the Flow traffic, and calculate the 8 features according to the calculation method described in step 5), and add the classification label of the domain name (using 1 and 0 to represent DGA domain names and benign domain names respectively), and merge them to obtain a 9-dimensional vector. The specific content of the vector is: {Feature 1 user IP address access times; Feature 2 life cycle; Feature 3 active frequency; Feature 4 periodicity; Feature 5 user access burst; Feature 6 standard deviation mean ratio of the number of occurrences of each resolved IP; Feature 7 total number of domain names reverse mapped by the resolved IP; Feature 8 resolved IP address set difference; classification label (0 or 1)},

[0038] Step 5-2) Put n (n is the number of domain names in the training set) 9-dimensional vectors into the C4.5 classification model for model training to obtain a classification model with classification capabilities. In order to make the classification results of the model have more reference value, a method for calculating confidence is added on the basis of the general model structure. The confidence can be measured by the proportion of labels in the leaf node. For example, if a leaf node contains 80% positive classes and 20% negative classes, the confidence is 0.8. Add a class_counts attribute to the Node class to record the label distribution in each leaf node. Then, calculate class_counts when creating a leaf node in the tree_grow_tree method, and return the predicted value and confidence in the predict_with_confidence method.

[0039] Step 6) extracts the traffic information of the domain name to be classified in the detection network segment and calculates its characteristic value. The specific data processing and calculation process are consistent with steps 3) to 6).

[0040] Step 7) Put the feature value of the domain name to be classified into the classification model for classification to obtain the classification result, which includes the classification prediction value and confidence.

[0041] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the algorithm-generated domain name detection method based on traffic analysis when executing the program.

[0042] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the algorithm-generated domain name detection method based on traffic analysis.

[0043] This scheme collects algorithm-generated domain names and benign domain names with known classification labels, collects their DNS traffic and Flow traffic, extracts key information from them, calculates the values ​​of 8 features with obvious differences, and puts them into the C4.5 model for model training. After the model training is completed, the traffic information of the domain name to be detected is collected, and its feature value is calculated according to the same steps. It is put into the C4.5 classification model for classification, and the domain names classified as algorithm-generated domain names are screened out. The 8 features are the number of user IP accesses, life cycle, active frequency, periodicity, user access burstiness, standard deviation mean ratio of the number of occurrences of each resolved IP, the total number of domain names reversely mapped by the resolved IP, and the difference of the resolved IP address set. These features are data that are obviously different from benign domain names due to malicious behavior generated by the algorithm-generated domain names, and can well distinguish between algorithm-generated domain names and benign domain names. This scheme detects based on the difference in traffic behavior performance between algorithm-generated domain names and benign domain names, which can resist anti-detection attacks, improve the accuracy and effectiveness of detection, and maintain network security.

[0044] Compared with the prior art, the advantages of the present invention are as follows:

[0045] 1) The present invention makes full use of the difference in traffic behavior between algorithm-generated domain names and benign domain names, and proposes 8 features based on traffic analysis, including {Feature 1: number of user IP address accesses; Feature 2: life cycle; Feature 3: active frequency; Feature 4: periodicity; Feature 5: user access burstiness; Feature 6: standard deviation mean ratio of the number of occurrences of each resolved IP; Feature 7: total number of domain names reversely mapped by the resolved IP; Feature 8: difference in the resolved IP address set}. Based on these features, algorithm-generated domain names and benign domain names can be well distinguished;

[0046] 2) The present invention designs an accurate and concise calculation method for each feature, which can be expressed by a specific mathematical formula. While facilitating the calculation of the feature into a specific numerical value, the value can correspond to the change of the feature, reflecting the difference between the algorithm-generated domain name and the benign domain name in each feature;

[0047] 3) The present invention fully considers the privacy and security of traffic information in the network, avoids using sensitive information as much as possible when designing the algorithm, obtains only the data required for the subsequent feature calculation process during the collection and processing, avoids collecting data such as the user's specific IP address and geographic location, and encapsulates the domain name and resolved IP address that need to be used in the algorithm, and only displays the feature value and classification result information in the final result;

[0048] 4) When selecting the machine learning classification model, the present invention takes into account that each feature has both continuous and discrete values, and that some features may have default values. Therefore, the C4.5 classification model that can solve these problems is selected. At the same time, a method for calculating confidence is independently added on the basis of the basic model, so that the classification result contains more effective information;

[0049] 5) The present invention is based on the traffic performance that is different from that of benign domain names when algorithm-generated domain names are used to carry out attack behaviors. This is what the algorithm-generated domain names will definitely have after carrying out malicious activities. Unlike their domain names, the characteristics based on traffic activities are difficult to mask against detection methods. Therefore, this method can ensure the long-term effectiveness of the classification model and resist anti-detection attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of the domain name detection method generated by the algorithm based on traffic analysis. DETAILED DESCRIPTION

[0051] In order to deepen the recognition and understanding of the present invention, the scheme is described in detail below with reference to the accompanying drawings and embodiments.

[0052] Example 1: See Figure 1 ,based on Figure 1 The method flow is described in detail, including the model training stage and the domain name classification stage to be detected.

[0053] A method for detecting domain names generated by an algorithm based on traffic analysis, the method comprising the following steps:

[0054] Step 1) Collect algorithm-generated domain names (DGA domain names) and benign domain names with known classification labels as the training set of the model.

[0055] Step 1-1) Collect the DGA domain name dataset provided by the open source website. The domain name must have appeared within 1 year of the current detection time. The open source website can be selected from 360netlab, bambenek and other websites. The bambenek database will update the recently detected DGA domain names every day, adding about 500,000 entries each time, and will also include the time when the DGA domain name was first detected, the malware it belongs to, and other information;

[0056] Step 1-2) Obtain the top 1 million information in the detection network segment. If there is no information, count the visits of all domain names in the detection network segment. In order to ensure that the training set covers all types of benign domain names, take the top 1,000 domain names, the middle 1,000 domain names, and the last 1,000 domain names as the source of the benign domain name training set;

[0057] Step 1-3) performs a matching search in the DNS traffic of the detection network segment based on the DGA domain name and the benign domain name obtained in step 1-1) and step 1-2), and uses the domain name appearing in the detection network segment as a training set.

[0058] Step 2) Collect the traffic information of the training set domain name from the detection network segment, including the resolved IP address and flow information of the domain name;

[0059] Step 2-1) The DGA domain name training set and the benign domain name training set are respectively put into the DNS traffic of the detection network segment for matching and searching. The search period is two consecutive days (48 hours). The search method is to see whether the domain name corresponds to the domain name resolved by the DNS request. If they are the same, it means that the domain name appears in the network segment, and the resolved IP address of the domain name is taken from the resolution information;

[0060] Step 2-2) Put all resolved IP addresses of the DGA domain name training set and the benign domain name training set into the flow traffic of the detection network segment for matching and searching. The search period is two consecutive days (48 hours). The search method is to see whether the IP address is the same as the source IP address or the destination IP address in the flow traffic. If they are the same, it means that the IP address has interacted with the user's IP address and can be regarded as a valid interaction information.

[0061] Step 3) Perform preliminary processing on the collected traffic information, including counting the number of occurrences of each resolved IP address of the domain name, filtering out data such as the source IP address, destination IP address, specific resolution time, and resolution hour value from the flow information,

[0062] Step 3-1) Based on the training set domain name resolution information obtained in step 2-1), all resolved IP addresses of each domain name and the number of occurrences of each resolved IP address in the two days before and after are counted;

[0063] Step 3-2) Based on the training set domain name flow information obtained in step 2-2), five fields including source IP address, destination IP address, first routing forwarding time, last routing forwarding time, and hour are filtered out from each piece of information.

[0064] Step 4) Calculate and process the processed flow information to obtain 8 flow characteristic values.

[0065] Step 4-1) Calculate the number of user IP address visits to the domain name based on the number of occurrences of each resolved IP address on the next day. The calculation formula is client_IP_visit = ∑IP i_resolved. In step 3-1), the number of occurrences of each resolved IP address of each domain name has been counted. The sum of the number of occurrences of all resolved IP addresses of each domain name is the number of user IP address visits to the domain name, which is used as feature 1;

[0066] Step 4-2) Calculate the life span of the domain name based on the source IP address, destination IP address, first routing forwarding time, and last routing forwarding time in the flow information of the two days before and after. The calculation formula is life_span = last_seen_time-first_seen_time. Use all the resolved IP addresses of the domain name to filter out all the first routing forwarding times and the last routing forwarding times from the flow information obtained in step 3-2), and calculate the difference between the latest time and the earliest time, which is the life span of the domain name, and use it as feature 2;

[0067] Step 4-3) Calculate the active frequency of the domain name based on the source IP address, destination IP address, first routing forwarding time, and last routing forwarding time in the flow information of the previous and next two days. The calculation formula is: After calculating the life cycle of the domain name in step 4-2), use all resolved IP addresses to filter out all the first routing forwarding times and the last routing forwarding times from the flow information, and calculate the sum of all time periods with an interval time of less than 5 minutes as the active time of the domain name. Then calculate the ratio of the active time to the life cycle as the active frequency of the domain name, which is used as feature 3;

[0068] Step 4-4) Calculate the periodicity of the domain name based on the source IP address, destination IP address, and hour information in the flow information of the previous and next two days. The calculation formula is: Use all resolved IP addresses of the domain name to filter out the hourly user IP address visits in the life cycle from the Flow information, and calculate the relative distance between the Euler distance of the hourly user IP address visits in the two days before and after and the mean of the curve (the value of the interval time period T can be adjusted according to the observation time) as the periodicity of the domain name, which is used as feature 4;

[0069] Step 4-5) Calculate the user access burst of the domain name based on the source IP address, destination IP address, and hour information in the flow information of the next day. The calculation formula is: Use all resolved IP addresses of the domain name to filter out the number of user IP address visits per hour in the life cycle from the flow information, calculate the average number of visits per hour, and select the maximum value. In order to make the change pattern of the abrupt value consistent with the change in the strength of the user access burstiness, use the difference between 1 and the ratio of the mean and the maximum value to represent abrupt as the user access burstiness of the domain name, and use it as feature 5;

[0070] Step 4-6) Calculate the standard deviation mean ratio of the number of occurrences of each resolved IP address of the domain name based on the number of occurrences of each resolved IP address on the next day. The calculation formula is: According to the number of occurrences of each resolved IP address of each domain name counted in step 3-1), calculate its variance and mean. In order to prevent the calculated values ​​of domain names of different magnitudes from differing too much, use the ratio of the variance to the mean to represent the difference in the number of occurrences of each resolved IP address of the domain name, and use it as feature 6;

[0071] Step 4-7) Calculate the total number of domain names mapped by the resolved IP reverse mapping based on the corresponding relationship between the domain name and the resolved IP the next day. The calculation formula is IP_domain_num = |{d(IP)|IP∈resolvedIPset(W)}|. After counting all the resolved IP address sets of all domain names, a dictionary with IP address as key and domain name as value is established. The total number of domain names mapped by all resolved IP addresses of the domain name is calculated from it and used as feature 7;

[0072] Step 4-8) Calculate the difference between the resolved IP address set based on the correspondence between the domain name and the resolved IP address two days before and after. The calculation formula is: For the time windows of two days before and after, all the resolved IP address sets of the domain name are established, and their intersection and union are calculated. Then, the ratio of the number of elements in the intersection and union is calculated. In order to make the change pattern of the size of the IP_diff value consistent with the change in the size of the difference of the resolved IP address set, the difference between 1 and the ratio of the number of elements in the intersection and union is used to represent the difference of the resolved IP address set of the domain name, which is used as feature 8.

[0073] Step 5) Put the feature values ​​and classification labels of all training set domain names into the C4.5 classification model for training to obtain a model with classification capabilities.

[0074] Step 5-1) For each domain name in the training set, the 8 features calculated in step 5) and the classification label of the domain name (using 1 and 0 to represent DGA domain names and benign domain names respectively) are combined to obtain a 9-dimensional vector, the specific content of which is: {Feature 1: number of visits by user IP addresses; Feature 2: life cycle; Feature 3: active frequency; Feature 4: periodicity; Feature 5: burstiness of user visits; Feature 6: standard deviation mean ratio of the number of occurrences of each resolved IP; Feature 7: total number of domain names reversely mapped by the resolved IP; Feature 8: difference in the resolved IP address set; classification label (0 or 1)};

[0075] Step 5-2) Put n (n is the number of domain names in the training set) 9-dimensional vectors into the C4.5 classification model for model training to obtain a classification model with classification ability. In order to make the classification results of the model have more reference value, a method for calculating confidence is added on the basis of the general model structure. The confidence can be measured by the proportion of labels in the leaf node. For example, if a leaf node contains 80% positive classes and 20% negative classes, the confidence is 0.8. Add a class_counts attribute to the Node class to record the label distribution in each leaf node. Then, calculate class_counts when creating a leaf node in the build_grow_tree method, and return the predicted value and confidence in the predict_with_confidence method.

[0076] Step 6) extract the traffic information of the domain name to be classified in the detection network segment and calculate its characteristic value.

[0077] Step 7) Put the feature value of the domain name to be classified into the trained classification model for classification to obtain the classification result, which includes the category prediction value and confidence.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions described in the above embodiments can still be modified or replaced by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A domain name detection method based on traffic analysis algorithm, characterized in that: The method comprises the following steps: Step 1) Collect algorithm-generated domain names and benign domain names with known classification labels as training sets for the model; Step 2) Collect the traffic information of the training set domain name from the detection network segment, including the domain name's resolved IP address related information and Flow information; Step 3) Perform preliminary processing on the collected traffic information, including counting the number of occurrences of each resolved IP address of the domain name, and filtering out the source IP address, destination IP address, specific resolution time, and resolution hourly value data from the flow information; Step 4) Calculate and process the processed flow information to obtain 8 flow characteristic values; Step 5) Put the feature values ​​and classification labels of all training set domain names into the C4.5 classification model for training to obtain a model with classification capabilities; Step 6) extracting the traffic information of the domain name to be classified and calculating its characteristic value; Step 7) Put the feature value of the domain name to be classified into the classification model for classification to obtain the classification result.

2. The method for detecting domain names generated by an algorithm based on traffic analysis according to claim 1, characterized in that: Step 1) Collect algorithm-generated domain names (DGA domain names) and benign domain names with known classification labels as the training set of the model. The DGA domain names come from the DGA domain name dataset provided by the open source website. The specific steps are as follows: Step 1-1) Collect the DGA domain name dataset provided by the open source website. To ensure its timeliness, the domain name must have appeared within 1 year. Step 1-2) Obtain the top 1 million information in the detection network segment. If there is no information, count the visits of all domain names in the detection network segment. In order to ensure that the training set covers all types of benign domain names, take the top 1,000 domain names, the middle 1,000 domain names, and the last 1,000 domain names as the source of the benign domain name training set; Step 1-3) performs a matching search in the detection network segment based on the DGA domain name and the benign domain name obtained in step 1-1) and step 1-2), and uses the domain name appearing in the detection network segment as a training set.

3. The method for detecting domain names generated by an algorithm based on traffic analysis according to claim 1, characterized in that: Step 2) Collect the traffic information of the training set domain name from the detection network segment, including the resolved IP address and flow information of the domain name, as follows: Step 2-1) The DGA domain name training set and the benign domain name training set are respectively put into the DNS traffic of the detection network segment for matching and searching. The search period is two consecutive days (48 hours). The search method is to see whether the domain name corresponds to the domain name resolved by the DNS request. If they are the same, it means that the domain name appears in the network segment, and the resolved IP address of the domain name is taken from the resolution information; Step 2-2) Put all resolved IP addresses of the DGA domain name training set and the benign domain name training set into the flow traffic of the detection segment for matching and searching. The search period is two consecutive days (48 hours). The search method is to see whether the IP address is the same as the source IP address or the destination IP address in the flow traffic. If they are the same, it means that the IP address has interacted with the user's IP address and is regarded as a valid interaction information.

4. The method for detecting domain names generated by an algorithm based on traffic analysis according to claim 1, characterized in that: Step 3) Perform preliminary processing on the collected traffic information, including counting the number of occurrences of each resolved IP address, filtering out the source IP address, destination IP address, specific resolution time, and resolution hourly value data from the Flow information, as follows: Step 3-1) Based on the training set domain name resolution information obtained in step 2-1), all resolved IP addresses of each domain name and the number of occurrences of each resolved IP address in the two days before and after are counted; Step 3-2) Based on the training set domain name flow information obtained in step 2-2), five fields including source IP address, destination IP address, first routing forwarding time, last routing forwarding time, and hour are filtered out from each piece of information.

5. The method for detecting domain names generated by an algorithm based on traffic analysis according to claim 1, characterized in that: Step 4) Calculate and process the processed flow information to obtain 8 flow characteristic values, as follows: Step 4-1) Calculate the number of user IP address visits to the domain name based on the number of occurrences of each resolved IP address the next day. The specific calculation method is: client_IP_visit = ∑IP i _resolved, where client_IP_visit indicates the number of times a user's IP address visits a domain name. i _resolved indicates the number of times the ith resolved IP address of the domain name is resolved in the DNS traffic. The number of times all resolved IP addresses of the domain name are resolved in the DNS traffic is used to represent the number of user IP address visits. The larger the client_IP_visit value is, the more times the user IP address has visited. It is taken as the feature value 1. Step 4-2) Calculate the life cycle of the domain name based on the source IP address, destination IP address, first routing forwarding time, and last routing forwarding time in the flow information of the previous and next two days. Due to the existence of various defense mechanisms in computer networks, various malicious domain names including DGA domain names will eventually be discovered and blocked, so the life cycle of DGA domain names will be shorter than that of benign domain names. The specific calculation method is: life_span = last_seen_time-first_seen_time, where life_span represents the life cycle of the domain name, last_seen_time represents the last time the domain name appears in the detection network segment, and first_seen_time represents the first time the domain name appears in the detection network segment. Both last_seen_time and first_seen_time are in seconds, and the difference between the two is used to represent the life cycle of the domain name. The larger the life_span value, the longer the life cycle, which is used as the feature value 2; Step 4-3) Calculate the active frequency of the domain name based on the source IP address, destination IP address, first routing forwarding time, and last routing forwarding time in the flow information of the previous and next two days. Since the zombie host will centrally access the C&C server using the DGA domain name in a short period of time according to the instructions, and the life cycle of the DGA domain name itself is relatively short, its active frequency is higher than that of the benign domain name that is independently and intermittently accessed by the user for a long time. The specific calculation method is: Among them, live_freq represents the active frequency of the domain name, life_span represents the life cycle of the domain name, client_IP_visit(h) represents the number of visits to the domain name by the user IP address in the h time period, and the active frequency is represented by the proportion of the time period when the domain name is visited by the user IP address in the entire life cycle. The larger the live_freq value, the higher the active frequency, which is taken as the feature value 3; Step 4-4) Calculate the periodicity of the domain name based on the source IP address, destination IP address, and hour information in the flow information of the previous and next two days. For zombie hosts, since commands are regularly issued through the C&C server, the behavior of zombie hosts has very strong synchronization, which is reflected in the domain name, that is, the access to the DGA domain name has a very strong periodicity. The specific calculation method is: Among them, periodic represents the periodicity value of the domain name, client_IP_visit(h) represents the number of visits to the domain name by the user IP address in the h time period, and the ratio of the Euler distance of the time series curve of the number of visits by the user IP address in the two time periods before and after the domain name to the relative distance of the curve mean is used to represent the periodicity. The smaller the periodic value, the stronger the periodicity, which is taken as the characteristic value 4; Step 4-5) Calculate the user access burstiness of the domain name based on the source IP address, destination IP address, and hour information in the flow information of the next day. After the C&C server issues a command, a large number of zombie hosts controlled by it will synchronously access the DGA domain name, causing the DGA domain name to have a higher user access burstiness than benign domain names. The specific calculation method is: Where abrupt represents the user access burst value of the domain name, client_IP_visit(T, life_span) represents the number of visits by the user IP address in a fixed time period T during the life cycle of the domain name, Avg represents the mean, Max represents the maximum value, and the relative distance between the mean and the maximum value in the user IP address visit time series curve represents the user access burst. The larger the abrupt value, the higher the user access burst, which is taken as the feature value 5; Step 4-6) Calculate the standard deviation mean ratio of the number of occurrences of each resolved IP address of the domain name based on the number of occurrences of each resolved IP address on the next day. The DGA domain name will use IP-Flux technology to frequently change the resolved IP address. The specific calculation method is: Among them, IP_var represents the standard deviation mean ratio of the number of occurrences of each resolved IP address of the domain name, var(IP_appear_times) represents the variance of the number of occurrences of each resolved IP address of the domain name, and avg(IP_appear_times) represents the average number of occurrences of each resolved IP address of the domain name. Their ratio is used to represent the standard deviation mean ratio of the number of occurrences of each resolved IP address. The larger the IP_var value, the larger the standard deviation mean ratio of the number of occurrences of the resolved IP address, which is used as the eigenvalue 6; Step 4-7) Calculate the total number of domain names that are reverse mapped to the resolved IP address based on the corresponding relationship between the domain name and the resolved IP address on the next day. The specific calculation method is: IP_domain_num = |{d(IP)|IP∈resolvedIPset(W)}|, where IP_domain_num represents the total number of domain names that are reverse mapped to all resolved IP addresses of the domain name, d(IP) represents the domain name that is reverse mapped to the IP address, and resolvedIPset(W) represents the set of all resolved IP addresses of the domain name in the current time window. The larger the IP_domain_num value is, the more domain names that are reverse mapped to the resolved IP address are, and it is used as the feature value 7; Step 4-8) Calculate the difference between the resolved IP address set based on the correspondence between the domain name and the resolved IP address two days before and after. The specific calculation method is: Among them, IP_diff represents the difference of the resolved IP address set, resolvedIPset(W) represents the set of all resolved IP addresses of the domain name in the current time window, and resolvedIPset(W') represents the set of all resolved IP addresses of the domain name in the previous time window. The ratio of their intersection and union is used to calculate the difference of the resolved IP address set of the domain name. The larger the IP_diff value, the greater the difference of the resolved IP address set, and it is used as the feature value 8.

6. The method for detecting domain names generated by an algorithm based on traffic analysis according to claim 1, characterized in that: Step 5) Put the feature values ​​and classification labels of all training set domain names into the C4.5 classification model for training to obtain a model with classification capabilities. The C4.5 model is a relatively mature supervised machine learning classification model and is used directly as follows: Step 5-1) For each domain name in the training set, first retrieve its related resolved IP address and its occurrence times in the collected DNS flow information database according to its domain name, then use the resolved IP address to find the number of user IP addresses and interaction time information from the Flow flow, and calculate the 8 features obtained by the calculation method described in step 5), and add the classification label of the domain name at the same time, and merge them to obtain a 9-dimensional vector. The specific content of the vector is: {Feature 1 Number of user IP address visits; Feature 2 Life cycle; Feature 3 Active frequency; Feature 4 Periodicity; Feature 5 User access burstiness; Feature 6 Standard deviation mean ratio of the number of occurrences of each resolved IP; Feature 7 The total number of domain names reversely mapped by the resolved IP; Feature 8 The difference of the resolved IP address set; Classification label (0 or 1)}, Step 5-2) Put n (n is the number of domain names in the training set) 9-dimensional vectors into the C4.5 classification model for model training to obtain a classification model with classification ability. Assume that a leaf node contains 80% positive classes and 20% negative classes, then the confidence is 0.

8. Add a class_counts attribute to the Node class to record the label distribution in each leaf node. Then, calculate class_counts when creating a leaf node in the build_grow_tree method, and return the predicted value and confidence in the predict_with_confidence method.

7. The method for detecting domain names generated by an algorithm based on traffic analysis according to claim 1, characterized in that: Step 6) extracts the traffic information of the domain name to be classified in the detection network segment and calculates its characteristic value. The specific data processing and calculation process are consistent with steps 3) to 6).

8. The method for detecting domain names generated by an algorithm based on traffic analysis according to claim 1, characterized in that: Step 7) Put the feature value of the domain name to be classified into the classification model for classification to obtain the classification result, which includes the classification prediction value and confidence.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for generating domain name detection based on an algorithm for traffic analysis as described in any one of claims 1 to 8 above is implemented.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by the processor, the method for generating domain name detection based on the algorithm of traffic analysis as described in any one of claims 1 to 8 is implemented.

Images