Abnormal domain name detection method, device, storage medium and equipment

Abstract

The present application provides a method, apparatus, storage medium, and device for detecting abnormal domain names. This method, based on the significant differences in character distribution between normal and abnormal domain names, uses the N-Gram method to segment the character string of the domain name to be detected. The probability distribution of each target substring set obtained by the segmentation is then calculated. The JS divergence between the probability distribution of each target substring set and the probability distribution of the substring set of the normal domain name is then calculated. Finally, the calculated JS divergence is compared with a preset divergence threshold to determine whether the domain name to be detected is an abnormal domain name. This effectively improves the accuracy of abnormal domain name detection.

Description

Technical Field

[0001] The present application relates to the field of network information security technology, and more specifically, to a method, apparatus, storage medium, and device for detecting abnormal domain names. Background Art

[0002] The DNS (Domain Name System) protocol is an essential network communication protocol for both the internet and corporate intranets. To access internet and intranet resources, the DNS protocol provides domain name resolution services, translating domain names into IP addresses. Generally, network devices and perimeter defense equipment rarely filter, analyze, or block DNS traffic. Therefore, hiding data or instructions within the DNS protocol for transmission is a stealthy and effective method. This is precisely why DNS has attracted widespread attention from attackers, and using the DNS protocol to hide data has become a key method for attackers to carry out spam, phishing, and click fraud.

[0003] Related technologies for detecting abnormal domain names primarily rely on domain blacklists. By determining whether the domain to be detected exists on the blacklist, the system determines whether to restrict user access, thereby ensuring network security. However, blacklists only contain verified abnormal domains, making the detection accuracy for unknown abnormal domains low, prone to false positives and omissions. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a method, apparatus, storage medium and device for detecting abnormal domain names, aiming to solve the problem of low detection accuracy and easy false positives or omissions in the detection methods for abnormal domain names in related technologies.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting abnormal domain names, including:

[0006] Perform N-Gram partitioning on the domain name to be detected to obtain at least two target substring sets; each target substring set includes multiple character segments; the lengths of the character segments included in different target substring sets are different;

[0007] For each target substring set, the probability distribution of the target substring set is calculated based on the frequency of occurrence of each character segment in the target substring set, and the JS divergence between the probability distribution of the target substring set and the probability distribution of the substring set corresponding to the same character segment length in the training set is calculated; the training set contains multiple normal domain names;

[0008] Based on the comparison result of the JS divergence and a preset divergence threshold, it is determined whether the domain name to be detected is an abnormal domain name; wherein the preset divergence threshold is obtained based on the JS divergence between the probability distribution of the subset set of the test set and the probability distribution of the subset set of the training set, and the test set contains multiple normal domain names.

[0009] In this implementation, based on the significant differences in character distribution between normal and abnormal domain names, the N-Gram method is used to segment the character strings of the domain name to be detected. The probability distribution of each target substring set obtained from the segmentation is then calculated. The JS divergence between the probability distribution of each target substring set and the probability distribution of the substring set of normal domain names is then calculated. Finally, the calculated JS divergence is compared with a preset divergence threshold to determine whether the domain name to be detected is an abnormal domain name. This effectively improves the accuracy of abnormal domain name detection.

[0010] Furthermore, in some embodiments, before performing N-Gram division on the domain name to be detected, the process includes:

[0011] Determine whether the domain name to be detected contains a target character; the target character is a character not contained in the collected multiple normal domain names;

[0012] If the judgment result is yes, it is determined that the domain name to be detected is an abnormal domain name.

[0013] In the above implementation process, when processing the domain name to be detected, the character judgment is first performed on the domain name to be detected, and the domain name containing non-common characters of the DNS protocol is directly determined as an abnormal domain name, thereby improving the efficiency of abnormal domain name detection to a certain extent.

[0014] Furthermore, in some embodiments, performing N-Gram partitioning on the domain name to be detected to obtain at least two target substring sets includes:

[0015] The domain name to be detected is segmented through a sliding window of size N to obtain multiple character fragments of length N, and a target substring set is formed based on the obtained character fragments; N takes at least two values ​​from 1 to 7, and different target substring sets correspond to different N values.

[0016] In the above implementation process, a specific method of performing N-Gram division on the domain name to be detected is provided, and the maximum value of N is limited to 7, which balances the detection accuracy and computational complexity and can achieve relatively good results.

[0017] Furthermore, in some embodiments, the preset divergence threshold is obtained based on the following method:

[0018] Divide the collected normal domain names into training sets and test sets according to the preset ratio;

[0019] Calculating the probability distribution of each subset of the training set and the probability distribution of each subset of the test set respectively; the subset of the training set and the subset of the test set are obtained by performing N-Gram partitioning on the training set and the test set respectively;

[0020] The JS divergence between the probability distribution of each subset of the training set and the probability distribution of the subset of the test set corresponding to the same character segment length is calculated, and a preset divergence threshold is obtained based on each JS divergence.

[0021] In the above implementation process, a specific method for obtaining a preset divergence threshold is provided.

[0022] Furthermore, in some embodiments, the probability distribution of each subset of the training set includes the probability distribution of the subset of each normal domain name included in the training set.

[0023] In the above implementation process, each normal domain name included in the training set is treated as a separate individual for statistical calculation, and multiple divergences corresponding to each gram can be obtained, making the preset divergence threshold set thereby more reasonable.

[0024] Furthermore, in some embodiments, before performing N-Gram on the training set and the test set respectively, the method includes:

[0025] The top-level domain name and the second-level domain name of each normal domain name in the training set are deleted, and the top-level domain name and the second-level domain name of each normal domain name in the test set are deleted.

[0026] In the above implementation process, the top-level domain name and the second-level domain name that do not affect the judgment of whether the entire domain name string is normal are deleted, thereby effectively reducing the computational complexity.

[0027] Furthermore, in some embodiments, judging whether the domain name to be detected is an abnormal domain name based on the comparison result of the JS divergence and a preset divergence threshold includes:

[0028] If, among the calculated JS divergences, the number of JS divergences exceeding the preset divergence threshold is greater than the number of JS divergences not exceeding the preset divergence threshold, it is determined that the domain name to be detected is an abnormal domain name.

[0029] In the above implementation process, it is determined whether the JS divergence of each gram is greater than the preset divergence threshold of the corresponding gram. If half or more of the JS divergences are greater than the preset divergence threshold of the corresponding gram, the domain name to be detected is determined to be an abnormal domain name. Otherwise, the domain name to be detected is determined to be a normal domain name. In this way, abnormal domain names can be accurately detected.

[0030] In a second aspect, an embodiment of the present application provides an abnormal domain name detection device, comprising:

[0031] A segmentation module is used to perform N-Gram segmentation on the domain name to be detected to obtain at least two target substring sets; each target substring set includes multiple character segments; the lengths of the character segments included in different target substring sets are different;

[0032] a calculation module configured to calculate, for each target substring set, a probability distribution of the target substring set based on the frequency of occurrence of each character segment in the target substring set, and calculate a JS divergence between the probability distribution of the target substring set and the probability distribution of a substring set corresponding to the same character segment length in a training set; the training set includes multiple normal domain names;

[0033] a judgment module, configured to judge whether the domain name to be detected is an abnormal domain name based on a comparison result of the JS divergence and a preset divergence threshold; wherein the preset divergence threshold is obtained based on the JS divergence between the probability distribution of a subset of a test set and the probability distribution of a subset of a training set, and the test set includes multiple normal domain names.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method as described in any one of the first aspects.

[0037] Other features and advantages disclosed in the present application will be described in the following description, or some features and advantages can be inferred or determined without doubt from the description, or can be learned by implementing the above-mentioned technology disclosed in the present application.

[0038] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A flowchart of a method for detecting abnormal domain names provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the workflow of a DNS abnormal domain name detection solution based on N-Gram and JS divergence provided in an embodiment of the present application;

[0042] Figure 3 A block diagram of an abnormal domain name detection device provided in an embodiment of the present application;

[0043] Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0045] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0046] As described in the background technology, the detection method for abnormal domain names in the related art has the problem of low detection accuracy and easy false positives or missed negatives. Based on this, the embodiment of the present application provides an abnormal domain name detection solution to solve the above problems.

[0047] Next, the embodiments of the present application are introduced:

[0048] like Figure 1 As shown, Figure 1This is a flowchart of an abnormal domain name detection method provided in an embodiment of the present application. The method can be applied to a terminal or server. The terminal can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers and desktop computers; the server can be a single server or a distributed server cluster composed of multiple servers. The terminal or server provides a data processing environment, which includes a software part and a hardware part. The software part mainly includes an operating system, such as Windows, Linux, etc., and the hardware part mainly includes computing resources, storage resources, etc., such as CPU (Central Processing Unit), memory, hard disk, etc. It should be noted that the terminal / server can also be implemented as multiple software or software modules, or can also be implemented as a single software or software module. This application does not limit this.

[0049] The method comprises:

[0050] In step 101, N-Gram partitioning is performed on the domain name to be detected to obtain at least two target substring sets; each target substring set includes multiple character segments; the lengths of the character segments included in different target substring sets are different;

[0051] A domain name, also known as a network domain, is the name of a computer or computer group on the Internet, consisting of a string of names separated by dots, and is used to locate and identify computers during data transmission. A normal domain name is usually a string of characters that is recognizable and readable by humans, that is, a string of characters that is meaningful, and the design of a normal domain name generally follows the language strategy of natural language, such as vowels following consonants. In addition, normal domain names are usually not encoded or encrypted. For abnormal domain names, in order to avoid hitting non-destination DNS servers and thus failing to reach the destination DNS server, attackers often encode or encrypt the domain name to impose certain special features, thereby ensuring that abnormal domain name queries can reach the destination DNS server. Therefore, the character string of the abnormal domain name does not have the characteristics of the character string of the normal domain name mentioned above. Based on this, this embodiment uses the N-Gram method to divide the domain names to be detected, laying the foundation for subsequent comparison and judgment.

[0052] N-Gram is an algorithm based on a statistical language model. Its basic idea is to perform a sliding window operation of size N on the byte content of a text, forming a sequence of byte segments of length N. There are significant differences in the character distribution between normal and abnormal domain names, and N-Gram is a character segmentation method that can effectively reflect these differences in character distribution.

[0053] In some embodiments, this step may include: segmenting the domain name to be detected using a sliding window of size N to obtain multiple character segments of length N, and forming a target subset set based on the obtained character segments; N is at least two values ​​between 1 and 7, and different target subset sets correspond to different N values. That is, the character string in the domain name to be detected is divided using a sliding window of size N. When N is 1, a character segment is a single character, referred to as a 1-gram, and the set of all 1-grams is a target subset set; when N is 2, a character segment is a string of two characters, referred to as a 2-gram, and the set of all 2-grams is a target subset set; and so on, when N is 7, the target subset set is a 7-gram set. Larger values ​​of N improve accuracy to a certain extent, but also increase computational complexity. Experiments have found that setting the maximum value of N to 7 can achieve relatively good results. Of course, in other embodiments, the value of N can also be set differently based on the needs of specific scenarios.

[0054] In addition, in some embodiments, before this step, the following steps may be included: determining whether the domain name to be detected contains target characters; the target characters are characters that are not contained in the multiple normal domain names collected; if the judgment result is yes, determining that the domain name to be detected is an abnormal domain name. The target characters here are special characters that basically do not exist in the normal DNS domain name data currently collected, that is, characters commonly used in non-DNS protocols, such as "%", "$", "&", etc. When processing the domain name to be detected, the domain name to be detected is first subjected to character judgment, and the domain name containing characters commonly used in non-DNS protocols is directly determined to be an abnormal domain name. For the domain name to be detected that passes the character judgment, the N-Gram method is used for division, and subsequent comparison and judgment steps are performed. In this way, the efficiency of abnormal domain name detection can be improved to a certain extent.

[0055] In step 102, for each target subset, a probability distribution of the target subset is calculated based on the frequency of occurrence of each character segment in the target subset, and the JS divergence between the probability distribution of the target subset and the probability distribution of a subset of the same character segment length in a training set is calculated; the training set includes multiple normal domain names;

[0056] After obtaining several target substring sets of the domain name to be detected, a probability distribution is calculated for each target substring set. Optionally, the probability distribution of the target substring set can be obtained by dividing the frequency of occurrence of each character segment in the target substring set by the total number of character segments contained in the target substring set. Specifically, there are 64 common characters in DNS. If the target substring set includes a 1-gram set, the frequency of each 64 1-grams can be counted separately, and the frequency of the 1-gram that does not appear is recorded as 0. Then, the frequency of each 1-gram is divided by the total number of characters contained in the 1-gram set, and the quotient obtained is used as the probability of each 1-gram. These probability values ​​constitute the probability distribution of the 1-gram set; similarly, if the target substring set includes a 2-gram set, the frequency of 4096 (i.e., 64 squared) 2grams can be counted separately, and the frequency of the 2-gram that does not appear is recorded as 0. Then, the frequency of each 2-gram is divided by the total number of characters contained in the 2-gram set, and the quotient obtained is used as the probability of each 2-gram. These probability values ​​constitute the probability distribution of the 2-gram set; and so on, for the target substring sets when N = 3, 4, 5, 6, and 7, similar methods are used to calculate them separately to obtain the corresponding probability distribution.

[0057] Since the strings of normal domain names and abnormal domain names follow significantly different probability distributions, the JS divergence between the probability distribution of the target substring set and the probability distribution of the substring set corresponding to the same character segment length in the training set is calculated to measure the similarity of the probability distribution of the string of the domain name to be detected and the string of the normal domain name, so as to determine whether the domain name to be detected is an abnormal domain name. JS (Jensen-Shannon) divergence is a metric used to measure the similarity of two probability distributions. Generally, JS divergence is symmetric and its value is between 0 and 1. The higher the similarity between the two probability distributions, the smaller the JS divergence value. Assuming P and Q are two probability distributions, the JS divergence between the two probability distributions is calculated based on the following formula:

[0058]

[0059] Since the JS divergence is symmetric, that is, when calculating the divergence values ​​of two probability distributions, the order of the probability distributions has no effect on the metric value, therefore, when calculating the JS divergence of any two domain names or two sets of domain names, you can directly use the above formula for calculation. There is no need to consider processing the domain name data according to a specific rule or order, which reduces the computational complexity.

[0060] The training set mentioned in this step includes multiple normal domain names, and these normal domain names can be legal domain names collected from search engine platforms. Considering that the training set includes multiple normal domain names, therefore, when calculating the probability distribution of the subset set of the training set, the subset sets corresponding to the same character fragment length in all the normal domain names included in the training set can be put together for statistics, for example, all 1-grams are put together for statistical frequency and the probability distribution of the 1-gram set is calculated, all 2-grams are put together for statistical frequency and the probability distribution of the 2-gram set is calculated, and so on. In this way, it is possible to effectively reduce the time consuming of calculation. Of course, in other embodiments, other calculation methods can also be taken according to the needs of specific scenarios, for example, for each normal domain name in the training set, the frequency of 1-gram is first counted, then the probability distribution of 1-gram set is calculated, and finally the probability distribution of all domain names corresponding to the 1-gram set is averaged and calculated, and the probability distribution of the subset set corresponding to N value of 1 in the training set is obtained, and then the probability distribution of the subset set corresponding to N value of other numerical values ​​in the training set is calculated by corresponding method.

[0061] The number of JS divergences calculated in this step is the same as the number of target substring sets. For example, when N = 3, 4, and 5, the JS divergences between the probability distribution of the 3-gram set of the domain name to be detected and the probability distribution of the 3-gram set of the training set, the JS divergence between the probability distribution of the 4-gram set of the domain name to be detected and the probability distribution of the 4-gram set of the training set, and the JS divergence between the probability distribution of the 5-gram set of the domain name to be detected and the probability distribution of the 5-gram set of the training set are calculated respectively.

[0062] In step 103, based on the comparison result of the JS divergence and a preset divergence threshold, it is determined whether the domain name to be detected is an abnormal domain name; wherein the preset divergence threshold is obtained based on the JS divergence between the probability distribution of the subset set of the test set and the probability distribution of the subset set of the training set, and the test set includes multiple normal domain names.

[0063] The preset divergence threshold mentioned in this step is used to compare with the calculated JS divergence to determine whether the domain name to be detected is an abnormal domain name. Similarly, the number of preset divergence thresholds is also the same as the number of target subsets. Continuing with the previous example, when N = 3, 4, and 5, the three calculated JS divergences are compared with the preset divergence thresholds with N value of 3, N value of 4, and N value of 5 according to the length of the character segment. The preset divergence threshold is obtained based on the JS divergence between the probability distribution of the subset of the test set and the probability distribution of the subset of the training set. The test set here, like the training set, also contains multiple normal domain names. These normal domain names can also be legitimate domain names collected from the search engine platform.

[0064] In some embodiments, the preset divergence threshold can be obtained based on the following method: dividing the collected multiple normal domain names into a training set and a test set according to a preset ratio; calculating the probability distribution of each subset of the training set and the probability distribution of each subset of the test set respectively; the subset of the training set and the subset of the test set are obtained by performing N-Gram partitioning on the training set and the test set respectively; calculating the JS divergence between the probability distribution of each subset of the training set and the probability distribution of the subset of the test set corresponding to the same character segment length, and obtaining the preset divergence threshold based on each JS divergence. The preset ratio here can be set according to the needs of the actual scenario, such as 80% for the training set and 20% for the test set. The domain names in the training and test sets are segmented using the N-Gram method. The frequency of each character segment is then counted, and the probability distribution of each subset is calculated. The JS divergence between the probability distribution of each subset in the training set and the probability distribution of the subset corresponding to the same character segment length in the test set is then calculated. Specifically, the JS divergence between the probability distribution of the 1-gram set in the training set and the probability distribution of the 1-gram set in the test set, and the JS divergence between the probability distribution of the 2-gram set in the training set and the probability distribution of the 2-gram set in the test set are calculated. Finally, a preset divergence threshold is obtained based on each calculated JS divergence. This preset divergence threshold can be used as a critical value to characterize the similarity of the probability distributions followed by the character strings of two normal domain names, and can therefore be used to determine whether the domain name to be detected is an abnormal domain name.

[0065] It's important to note that the multiple normal domain names collected and the domain name to be detected can be from the same scenario. In practice, different enterprises or networks may define abnormal DNS domain names differently. Therefore, judging the domain name to be detected in a particular scenario based on the normal DNS domain names collected in that scenario can effectively improve judgment accuracy. Furthermore, choosing the appropriate time to collect normal DNS domain names ensures that the normal DNS domain names collected are representative of the environment.

[0066] Furthermore, the aforementioned probability distribution of each subset of the training set can be the probability distribution of each subset of the normal domain name contained in the training set. That is, when calculating the probability distribution of each subset of the training set, each normal domain name contained in the training set can be treated as a separate individual for statistical calculation. For example, if the test set contains 10 domain names, then for each test set domain name, after N-Gram partitioning, the probability distribution of the 1-gram set is calculated, and then the JS divergence between the probability distribution of the 1-gram set of each of the 10 test set domain names and the probability distribution of the 1-gram set of the training set is calculated, obtaining 10 JS divergence values. Based on these 10 JS divergence values, a preset divergence threshold corresponding to an N value of 1 can be determined. It should be noted that when setting the preset divergence threshold, in order to avoid setting the threshold too extreme, the mean of the 10 JS divergence values ​​can be used as the preset divergence threshold, or the 10 JS divergence values ​​can be sorted from large to small and the JS divergence value ranked second can be selected as the preset divergence threshold, and so on. In this way, the preset divergence threshold set is more reasonable. Of course, in other embodiments, the aforementioned probability distribution of each subset of the training set can also be obtained by statistically calculating all normal domain names contained in the training set as a whole. Afterwards, the JS divergence between the probability distribution of each subset of the training set and the probability distribution of each subset of the test set can be directly set to a preset divergence threshold.

[0067] Furthermore, before performing N-Gram division on the training set and the test set respectively, the following steps may be included: deleting the top-level domain name and the second-level domain name of each normal domain name in the training set, and deleting the top-level domain name and the second-level domain name of each normal domain name in the test set. Experiments have found that the top-level domain name and the second-level domain name usually do not affect the judgment of whether the entire domain name string is normal. Therefore, deleting these two parts can effectively reduce the computational complexity. It should be noted that before performing N-Gram division on the domain name to be detected, the top-level domain name and the second-level domain name of the domain name to be detected can also be deleted to further improve the detection efficiency.

[0068] In addition, the determination of whether the domain name to be detected is an abnormal domain name based on the comparison result of the JS divergence and the preset divergence threshold mentioned in this step may include: if the number of JS divergences exceeding the preset divergence threshold in the calculated JS divergence is greater than the number of JS divergences not exceeding the preset divergence threshold, determining that the domain name to be detected is an abnormal domain name. In other words, it is determined whether the JS divergence of each gram is greater than the preset divergence threshold of the corresponding gram. If half or more of the JS divergences are greater than the preset divergence threshold of the corresponding gram, the domain name to be detected is determined to be an abnormal domain name; otherwise, the domain name to be detected is determined to be a normal domain name. In this way, abnormal domain names can be accurately detected. Of course, in other embodiments, different settings can also be adopted according to the needs of specific scenarios. For example, in some scenarios with a low tolerance for abnormal domain names, it can also be set that if any JS divergence in the calculated JS divergence exceeds the corresponding preset divergence threshold, the domain name to be detected is determined to be an abnormal domain name. This application does not limit this.

[0069] In this embodiment of the application, based on the significant difference in character distribution between normal and abnormal domain names, the N-Gram method is used to segment the character string of the domain name to be detected. The probability distribution of each target substring set obtained by the segmentation is then calculated. The JS divergence between the probability distribution of each target substring set and the probability distribution of the substring set of the normal domain name is then calculated. Finally, the calculated JS divergence is compared with a preset divergence threshold to determine whether the domain name to be detected is an abnormal domain name. In this way, the accuracy of abnormal domain name detection is effectively improved.

[0070] In order to explain the solution of this application in more detail, a specific embodiment is introduced below:

[0071] Abnormal domain name detection is an important means of ensuring network security. Experiments have found significant differences in character distribution between normal and abnormal DNS domain names. Based on this, this embodiment provides a DNS abnormal domain name detection solution based on N-Gram and JS divergence to accurately detect abnormal DNS domain names.

[0072] The workflow of this embodiment is as follows Figure 2 Shown, including:

[0073] S201. Collect normal DNS domain name data;

[0074] S202, dividing the collected normal DNS domain name data into a training set and a test set;

[0075] Specifically, we first delete the top-level domain name and the second-level domain name from the domain name, and then divide the normal DNS domain name data into two parts with a ratio of 8:2. The larger part is used as the training set, and the other is used as the test set.

[0076] S203, performing N-Gram partitioning on the training set and the test set respectively;

[0077] Specifically, the domain names in the training set are divided into N-Gram substrings. The substring subsets obtained with different N values ​​are stored separately, that is, 1-gram substrings are stored in 1-gram sets, 2-gram substrings are stored in 2-gram sets, and so on. The N-Gram division of the test set adopts the same method; in this embodiment, N = 1, 2, 3, 4, 5, 6, 7;

[0078] S204, calculating the probability distribution of each set of the training set, and the probability distribution of each set of the training set;

[0079] Specifically, for the 1-gram set, the frequencies of 64 1-gram subcharacters (i.e., the 64 common characters in DNS) are counted respectively, the frequencies of 1-gram subcharacters that do not appear are recorded as 0, and then the frequencies of each 1-gram subcharacter are divided by the total number of 1-gram subcharacters. The obtained values ​​are used as the probabilities of each 1-gram subcharacter, and these probability values ​​constitute the probability distribution of the 1-gram set; for 2-gram, the frequencies of 4096 2-gram substrings are counted respectively, the frequencies of 2-gram substrings that do not appear are recorded as 0, and then the frequencies of each 2-gram substring are divided by the total number of 2-gram substrings. The obtained values ​​are used as the probabilities of each 2-gram substring, and these probability values ​​constitute the probability distribution of the 2-gram set, and so on. For the domain name data in the training set, all domain names are statistically calculated as a whole. That is, all 1-grams are put together to count the frequency and calculate the probability distribution of the 1-gram set. All 2-grams are put together to count the frequency and calculate the probability distribution of the 2-gram set, and so on. For the domain name data in the test set, each domain name is statistically calculated as an individual. That is, for each domain name, the frequency of 1-gram to 7-gram is counted separately, and then the probability distribution of the 1-gram set to 7-gram set is calculated separately.

[0080] S205. Determine a JS divergence threshold based on the JS divergence between the probability distribution of each set in the training set and the probability distribution of each set in the test set;

[0081] Specifically, take a domain name in the test set, calculate the JS divergence between the probability distribution of the 1-gram set of the domain name and the probability distribution of the 1-gram set of the training set, then take the next domain name in the test set, and similarly calculate the JS divergence between the probability distribution of the 1-gram set of the domain name and the probability distribution of the 1-gram set of the training set. In this way, multiple 1-gram JS divergence values ​​are obtained, and the 1-gram JS divergence threshold is determined using these 1-gram JS divergence values. The JS divergence threshold can be the minimum of these 1-gram JS divergence values. Large values ​​or values ​​exceeding a certain proportion. For example, the JS divergence values ​​of the probability distributions of the 10 test set domain names and the training set 1-gram are 0.5824, 0.5595, 0.5317, 0.5485, 0.6018, 0.6184, 0.5578, 0.5673, 0.4921, and 0.5527, respectively. To avoid setting the threshold too extreme, the JS divergence threshold can be set to 0.6 based on these 10 JS divergence values. Correspondingly, the JS divergence thresholds for 2-gram to 7-gram are calculated in the same way.

[0082] S206: Obtain the domain name to be detected;

[0083] Specifically, the domain name to be detected corresponds to the same environment as the collected normal DNS domain name data. For example, the domain name to be detected and the normal DNS domain name data mentioned above are domain names used to access the same website. In addition, the execution order of S206 is not limited to after S205. The execution result of S205 is applied to S210.

[0084] S207, determine whether the domain name to be detected contains non-DNS protocol common characters, if so, execute S212, if not, execute S208;

[0085] S208, after dividing the domain name to be detected into N-Gram groups, calculate the probability distribution of each group;

[0086] Specifically, the method in S203 and S204 is used to calculate the probability distribution of the 1-gram set to the 7-gram set of the domain name to be detected;

[0087] S209, calculating the JS divergence between the probability distribution of each set of domain names to be detected and the probability distribution of each set of training sets;

[0088] Specifically, the JS divergence value between the probability distribution of the 1-gram set of the domain name to be detected and the probability distribution of the 1-gram set of the training set, the JS divergence value between the probability distribution of the 2-gram set of the domain name to be detected and the probability distribution of the 2-gram set of the training set, and the JS divergence values ​​corresponding to 3-gram, 4-gram, 5-gram, 6-gram, and 7-gram are calculated respectively;

[0089] S210: Determine whether half or more of the JS divergence values ​​are greater than the JS divergence threshold. If so, execute S212; if not, execute S211.

[0090] Specifically, the JS divergence value of the 1-gram (and 2-gram, 3-gram, ...) obtained in S209 is compared with the JS divergence threshold of the 1-gram (and 2-gram, 3-gram, ...) obtained in S205 to determine whether the JS divergence value of each gram is greater than the corresponding JS divergence threshold;

[0091] S211, determining that the domain name to be detected is a normal domain name;

[0092] S212: Determine that the domain name to be detected is an abnormal domain name.

[0093] The solution of the embodiment of the present application is based on the fact that there is a large difference in the character distribution of normal and abnormal DNS domain names. It uses the N-gram method to divide the domain name character string, then calculates the probability distribution of the N-gram string, and finally uses the fact that there is a large difference in the probability distribution of the N-gram of normal and abnormal domain names, which leads to a large JS divergence value of the probability distribution of the two, to detect abnormal domain names. Therefore, it has a high detection accuracy and significantly improves the detection effect.

[0094] Corresponding to the embodiments of the aforementioned method, the present application also provides embodiments of an abnormal domain name detection device and a terminal using the same:

[0095] like Figure 3 As shown, Figure 3 This is a block diagram of an abnormal domain name detection device provided by an embodiment of the present application, the device comprising:

[0096] The segmentation module 31 is used to perform N-Gram segmentation on the domain name to be detected to obtain at least two target substring sets; each target substring set includes multiple character segments; the lengths of the character segments included in different target substring sets are different;

[0097] a calculation module 32 for calculating, for each target substring set, a probability distribution of the target substring set based on the frequency of occurrence of each character segment in the target substring set, and calculating a JS divergence between the probability distribution of the target substring set and the probability distribution of a substring set corresponding to the same character segment length in a training set; the training set includes multiple normal domain names;

[0098] A judgment module 33 is used to judge whether the domain name to be detected is an abnormal domain name based on the comparison result of the JS divergence and a preset divergence threshold; wherein the preset divergence threshold is obtained based on the JS divergence between the probability distribution of the subset set of the test set and the probability distribution of the subset set of the training set, and the test set contains multiple normal domain names.

[0099] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0100] This application also provides an electronic device, see Figure 4 , Figure 4 This is a block diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include a processor 410, a communication interface 420, a memory 430, and at least one communication bus 440. The communication bus 440 is used to enable direct communication between these components. The communication interface 420 of the electronic device in this embodiment of the present application is used to communicate signaling or data with other node devices. The processor 410 may be an integrated circuit chip with signal processing capabilities.

[0101] The processor 410 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The processor 410 may be a microprocessor, or the processor 410 may be any conventional processor.

[0102] The memory 430 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 430 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 410, the electronic device can perform the above-mentioned operations. Figure 1 The various steps involved in the method embodiment.

[0103] Optionally, the electronic device may further include a storage controller and an input / output unit.

[0104] The memory 430, storage controller, processor 410, peripheral interface, and input / output units are electrically connected to each other directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses 440. The processor 410 is used to execute executable modules stored in the memory 430, such as software function modules or computer programs included in the electronic device.

[0105] The input and output unit is used to provide users with the ability to create tasks and to create optional time periods or preset execution times for the tasks to enable interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and a keyboard.

[0106] I understand. Figure 4 The structure shown is only for illustration, and the electronic device may also include Figure 4 More or fewer components than shown, or with Figure 4 Different configurations shown. Figure 4 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0107] An embodiment of the present application further provides a storage medium having instructions stored thereon. When the instructions are run on a computer, the computer program is executed by a processor to implement the method described in the method embodiment. To avoid repetition, details are not given here.

[0108] The present application also provides a computer program product, which, when running on a computer, enables the computer to execute the method described in the method embodiment.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0110] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0111] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0112] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0113] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0114] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A method for detecting abnormal domain names, characterized in that: include: Perform N-Gram partitioning on the domain name to be detected to obtain at least two target substring sets; each target substring set includes multiple character segments; the lengths of the character segments included in different target substring sets are different; For each target substring set, the probability distribution of the target substring set is calculated based on the frequency of occurrence of each character segment in the target substring set, and the JS divergence between the probability distribution of the target substring set and the probability distribution of the substring set corresponding to the same character segment length in the training set is calculated; the training set contains multiple normal domain names; Based on a comparison result of the JS divergence and a preset divergence threshold, determining whether the domain name to be detected is an abnormal domain name; wherein the preset divergence threshold is obtained based on the JS divergence between the probability distribution of a subset of strings in a test set and the probability distribution of a subset of strings in a training set, the test set including multiple normal domain names; The determining whether the domain name to be detected is an abnormal domain name based on the comparison result of the JS divergence and the preset divergence threshold includes: If, among the calculated JS divergences, the number of JS divergences exceeding the preset divergence threshold is greater than the number of JS divergences not exceeding the preset divergence threshold, it is determined that the domain name to be detected is an abnormal domain name.

2. The method according to claim 1, characterized in that Before performing N-Gram division on the domain name to be detected, the following steps are included: Determine whether the domain name to be detected contains a target character; the target character is a character not contained in the collected multiple normal domain names; If the judgment result is yes, it is determined that the domain name to be detected is an abnormal domain name.

3. The method according to claim 1, characterized in that The domain name to be detected is divided into N-Grams to obtain at least two target substring sets, including: The domain name to be detected is segmented through a sliding window of size N to obtain multiple character fragments of length N, and a target substring set is formed based on the obtained character fragments; N takes at least two values ​​from 1 to 7, and different target substring sets correspond to different N values.

4. The method according to claim 1, wherein The preset divergence threshold is obtained based on the following method: Divide the collected normal domain names into training sets and test sets according to the preset ratio; Calculating the probability distribution of each subset of the training set and the probability distribution of each subset of the test set respectively; The subset of the training set and the subset of the test set are obtained by performing N-Gram partitioning on the training set and the test set respectively; The JS divergence between the probability distribution of each subset of the training set and the probability distribution of the subset of the test set corresponding to the same character segment length is calculated, and a preset divergence threshold is obtained based on each JS divergence.

5. The method according to claim 4, characterized in that The probability distribution of each subset of the training set includes the probability distribution of the subset of each normal domain name included in the training set.

6. The method according to claim 4, characterized in that Before performing N-Gram on the training set and the test set respectively, the method includes: The top-level domain name and the second-level domain name of each normal domain name in the training set are deleted, and the top-level domain name and the second-level domain name of each normal domain name in the test set are deleted.

7. An abnormal domain name detection device, characterized in that: include: A segmentation module is used to perform N-Gram segmentation on the domain name to be detected to obtain at least two target substring sets; each target substring set includes multiple character segments; the lengths of the character segments included in different target substring sets are different; a calculation module configured to calculate, for each target substring set, a probability distribution of the target substring set based on the frequency of occurrence of each character segment in the target substring set, and calculate a JS divergence between the probability distribution of the target substring set and the probability distribution of a substring set corresponding to the same character segment length in a training set; the training set includes multiple normal domain names; a determination module, configured to determine whether the domain name to be detected is an abnormal domain name based on a comparison result of the JS divergence with a preset divergence threshold; wherein the preset divergence threshold is obtained based on the JS divergence between the probability distribution of a subset of a test set and the probability distribution of a subset of a training set, wherein the test set includes multiple normal domain names; The judgment module is specifically used for: If, among the calculated JS divergences, the number of JS divergences exceeding the preset divergence threshold is greater than the number of JS divergences not exceeding the preset divergence threshold, it is determined that the domain name to be detected is an abnormal domain name.

8. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

Images