Malicious domain name detection method, device, electronic device and storage medium

Abstract

The present application provides a malicious domain name detection method, device, electronic device and storage medium, which relate to the field of security technology. The method obtains multiple word segments of a domain name to be detected, inputs the multiple word segments into a fastText model, detects whether the domain name to be detected is a malicious domain name through the fastText model, and obtains a detection result. Since the fastText model can realize the text classification function using a shallow neural network, in the text classification task, the shallow network can often achieve an accuracy comparable to that of a deep network, so the fastText model is used to detect malicious domain names, which can achieve the detection accuracy that can be achieved by a deep network. Compared with the existing detection method based on regular expressions or black and white lists, the detection accuracy of this solution is higher.

Description

Technical Field

[0001] The present application relates to the field of security technology, and in particular to a malicious domain name detection method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of Internet technology, network security issues have become increasingly prominent. At present, a large number of DGA domain names can be quickly generated through the Domain Generation Algorithm (DGA), and a botnet with good robustness can be built through the DGA domain name. Attackers can use botnets to launch network attacks on devices in the network.

[0003] In order to improve network security, it is necessary to detect DGA domain names. Currently, the commonly used detection technology is based on regular expressions or black and white lists. However, since DGA domain names are easy to generate and change quickly, the false positive rate of using pre-set regular expressions or black and white lists to detect DGA domain names is high. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a malicious domain name detection method, device, electronic device and storage medium to improve the problems of high false alarm rate and low accuracy of domain name detection methods in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting malicious domain names, the method comprising:

[0006] Get the domain name to be tested;

[0007] Segmenting the domain name to be detected to obtain multiple segmented words;

[0008] The multiple word segments are input into the fastText model, and the fastText model is used to detect whether the domain name to be detected is a malicious domain name to obtain a detection result.

[0009] In the above implementation process, the method obtains multiple word segments of the domain name to be detected, inputs the multiple word segments into the fastText model, detects whether the domain name to be detected is a malicious domain name through the fastText model, and obtains the detection result. Since the fastText model can realize the text classification function using a shallow neural network, in the text classification task, the shallow network can often achieve the accuracy comparable to that of the deep network. Therefore, the fastText model is used to detect malicious domain names, which can achieve the detection accuracy that can be achieved by the deep network. Compared with the existing detection method based on regular expressions or black and white lists, the detection accuracy of this scheme is higher.

[0010] Optionally, the fastText model includes an input layer, a hidden layer and an output layer, the multiple word segments are used as inputs of the input layer, the input layer is used to convert the multiple word segments into corresponding word vectors, the hidden layer is used to perform superposition and average processing on the word vectors, and the output layer is used to output the detection result of the domain name to be detected based on the processing result of the hidden layer. The fastText model combines the idea of ​​natural language processing and the hierarchical idea of ​​neural networks, so it can better extract the correlation between each word segment and achieve the prediction accuracy of the neural network.

[0011] Optionally, the segmenting the domain name to be detected to obtain a plurality of segmented words includes:

[0012] The N-grama model is used to segment the domain name to be detected to obtain multiple segmented words. In this way, the domain name to be detected can be segmented better, so that the fastText model can more accurately analyze the rationality of the order of each segmented word and the constraint information on the next segmented word, with greater discrimination, thereby improving the detection accuracy of malicious domain names.

[0013] Optionally, the segmenting the domain name to be detected to obtain a plurality of segmented words includes:

[0014] Using an N-grama model to segment the domain name to be detected to obtain multiple initial segmentations;

[0015] The data association analysis method is used to analyze the association relationship between each initial participle;

[0016] According to the correlation between each initial participle, multiple participles with strong correlation are selected.

[0017] In the above implementation process, the data association analysis method is used to mine the associations between the initial word segmentations and screen out the word segmentations with strong associations. This can further improve the fastText model to extract the associations between the word segmentations with strong associations and improve the detection accuracy.

[0018] Optionally, the segmenting the domain name to be detected to obtain a plurality of segmented words includes:

[0019] Using an N-grama model to segment the domain name to be detected to obtain multiple initial segmentations;

[0020] Combining the multiple initial participles according to a set combination method to obtain multiple combined participles;

[0021] The input data of the fastText model includes the multiple initial word segmentations and the multiple combined word segmentations.

[0022] In the above implementation process, multiple initial word segmentations are combined, which can increase the amount of data input into the fastText model, so that the fastText model can use more data to detect malicious domain names and improve detection accuracy.

[0023] Optionally, after obtaining the domain name to be detected, segmenting the domain name to be detected, and before obtaining multiple segmented words, the method further includes:

[0024] Calculating similarity between the domain name to be detected and multiple pre-stored malicious domain names to obtain similarity between the domain name to be detected and each malicious domain name;

[0025] If the similarity is greater than the set similarity by more than a set ratio, the domain name to be detected is determined to be a suspicious domain name, and the step of segmenting the domain name to be detected is executed.

[0026] In the above implementation process, the similarity between the domain name to be detected and multiple malicious domain names is calculated in advance, and subsequent detection is performed only when the domain name to be detected is a suspicious domain name. This can reduce the detection amount of the subsequent fastText model. When there are a large number of domain names to be detected in the network, this can improve the detection efficiency.

[0027] Optionally, the calculating similarity between the domain name to be detected and a plurality of pre-stored malicious domain names to obtain the similarity between the domain name to be detected and each malicious domain name includes:

[0028] Calculating the fuzzy hash value of the domain name to be detected, and calculating the fuzzy hash value of each pre-stored malicious domain name;

[0029] The fuzzy hash value of the domain name to be detected is compared with the fuzzy hash values ​​of each malicious domain name for similarity, so as to obtain the similarity between the domain name to be detected and each malicious domain name.

[0030] In the above implementation process, the fuzzy hash value can more accurately compare the similarity between two strings, because the addition, modification, deletion and other changes of data in a certain shard of the domain name will only affect the hash of the shard, and have little impact on the global situation, so the impact on the final similarity is not significant. Even if several consecutive characters are changed, or multiple changes are made, the fuzzy hash algorithm can still make an effective judgment, and then by comparing the fuzzy hash values ​​between the domain names, the similarity between the two domain names can be more accurately judged.

[0031] In a second aspect, an embodiment of the present application provides a malicious domain name detection device, the device comprising:

[0032] A domain name acquisition module is used to obtain the domain name to be detected;

[0033] A word segmentation module, used to segment the domain name to be detected to obtain multiple word segments;

[0034] The detection module is used to input the multiple word segments into the fastText model, detect whether the domain name to be detected is a malicious domain name through the fastText model, and obtain a detection result.

[0035] Optionally, the fastText model includes an input layer, a hidden layer and an output layer, the multiple word segmentations are used as inputs of the input layer, the input layer is used to convert the multiple word segmentations into corresponding word vectors, the hidden layer is used to perform superposition and averaging processing on the word vectors, and the output layer is used to output the detection result of the domain name to be detected based on the processing result of the hidden layer.

[0036] Optionally, the word segmentation module is used to segment the domain name to be detected using an N-grama model to obtain multiple word segments.

[0037] Optionally, the word segmentation module is used to segment the domain name to be detected using an N-grama model to obtain multiple initial word segmentations; use a data association analysis method to analyze the association relationship between the initial word segmentations; and screen out multiple word segmentations with strong associations based on the association relationship between the initial word segmentations.

[0038] Optionally, the word segmentation module is used to use an N-grama model to segment the domain name to be detected to obtain multiple initial word segmentations; combine the multiple initial word segmentations according to a set combination method to obtain multiple combined word segmentations; wherein the input data of the fastText model includes the multiple initial word segmentations and the multiple combined word segmentations.

[0039] Optionally, the device further comprises:

[0040] The pre-detection module is used to calculate the similarity between the domain name to be detected and multiple malicious domain names stored in advance to obtain the similarity between the domain name to be detected and each malicious domain name; if the similarity is greater than the set similarity by more than a set ratio, the domain name to be detected is determined to be a suspicious domain name.

[0041] Optionally, the pre-detection module is used to calculate the fuzzy hash value of the domain name to be detected, and calculate the fuzzy hash value of each malicious domain name stored in advance; compare the fuzzy hash value of the domain name to be detected with the fuzzy hash value of each malicious domain name for similarity, and obtain the similarity between the domain name to be detected and each malicious domain name.

[0042] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are performed.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first aspect are performed.

[0044] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A flowchart of a malicious domain name detection method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the structure of a fastText model provided in an embodiment of the present application;

[0048] Figure 3 A structural block diagram of a malicious domain name detection device provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of an electronic device for executing a malicious domain name detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application.

[0051] It should be noted that the terms "system" and "network" in the embodiments of the present invention can be used interchangeably. "Multiple" means two or more. In view of this, in the embodiments of the present invention, "multiple" can also be understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the previously associated objects are in an "or" relationship.

[0052] The embodiment of the present application provides a malicious domain name detection method, which obtains multiple word segments of the domain name to be detected, inputs the multiple word segments into the fastText model, detects whether the domain name to be detected is a malicious domain name through the fastText model, and obtains the detection result. Since the fastText model can realize the text classification function using a shallow neural network, in the text classification task, the shallow network can often achieve the same accuracy as the deep network. Compared with the detection method based on regular expressions or black and white lists in the existing method, the detection accuracy of this solution is higher. In addition, the depth of the fastText model is not as deep as the deep learning model, and the detection process is relatively simple and the time is shorter. Therefore, compared with the deep learning model, the fastText model is used in this application to detect malicious domain names, which can shorten the detection time while ensuring accuracy.

[0053] Please refer to Figure 1 , Figure 1 A flowchart of a malicious domain name detection method provided in an embodiment of the present application, the method comprising the following steps:

[0054] Step S110: Obtain the domain name to be detected.

[0055] Among them, the domain name to be detected can refer to any domain name that needs to be detected for malicious domain names. For example, any domain name received by the security protection device can be used as the domain name to be detected, and then the malicious domain name detection is performed according to the subsequent steps to identify the received malicious domain name, thereby avoiding the situation in which malicious domain names pose a threat to network security in advance, such as intercepting malicious domain names, or taking other protective measures to deal with malicious domain names.

[0056] Step S120: segmenting the domain name to be detected to obtain multiple segmented words.

[0057] The method for segmenting the domain name to be detected may adopt any segmentation method, such as word2vec segmentation method, or adopt N-grama model to segment the domain name to be detected to obtain multiple segmentations.

[0058] Since the difference between normal domain names and malicious domain names is mainly reflected in the character composition of the domain names, the N-grama model can be used in the embodiment of the present application to segment the domain names to be detected. When segmenting, the N-gram features of word granularity and character granularity can be divided according to the different granularities. N-gram is an algorithm based on a statistical language model. Its basic idea is to perform a sliding window operation of size n on the content of the text according to bytes to form a sequence of byte fragments of length n. Each byte fragment is called a gram, forming a gram list, that is, the vector feature space of the text, and each gram in the list is a feature vector dimension. In the embodiment of the present application, word segmentation can be performed using the character granularity unigram (unary word segmentation, each character from the beginning to the end of the domain name to be detected is composed of a word), bigram (binary word segmentation, every two characters from the beginning to the end of the domain name to be detected are composed of a word) and trigram (ternary word segmentation, every three characters from the beginning to the end of the domain name to be detected are composed of a word), and the characters after the word segmentation of the domain name to be detected are regarded as independent words, and all the character words after the word segmentation of the domain name to be detected are regarded as a sentence, which is used to analyze the rationality of the arrangement order of the characters of the domain name to be detected, and has more constraint information on the appearance of the next word and greater discrimination.

[0059] For example, the domain name to be detected is google.cn, and then the unigram features, bigram features and trigram features of the domain name are extracted, and divided according to the division windows n=1, n=2 and n=3, the word segmentation set ["g", "o", "o", "g", "l", "e", "$g", "go", "oo", "og", "gl", "le", "e$", "$go", "goo", "oog", "ogl", "gle", "le$"] can be obtained, where the $ character is used to mark the beginning and end of the domain name.

[0060] Step S130: input the multiple word segments into the fastText model, and use the fastText model to detect whether the domain name to be detected is a malicious domain name to obtain a detection result.

[0061] After the domain name to be detected is segmented according to the corresponding segmentation method, the obtained multiple segmentations are input into the fastText model, so that the domain name to be detected is detected as a malicious domain name based on the multiple segmentations by the fastText model. In some embodiments, Figure 2 As shown in the figure, the fastText model includes an input layer, a hidden layer, and an output layer. iRepresents a segmented word) can be used as the input of the input layer, the input layer is used to convert multiple segmented words into corresponding word vectors, the hidden layer is used to perform superposition and average processing on the word vectors, and the output layer is used to output the detection result of the domain name to be detected based on the processing result of the hidden layer. The detection result can be the category to which the domain name to be detected belongs, such as malicious domain name and normal domain name.

[0062] In some text classification tasks, there are many categories and the computational complexity of linear classifiers is high. To improve the running time, the fastText model uses a hierarchical softmax technique. The hierarchical softmax technique is based on Huffman coding and encodes the labels, which can greatly reduce the number of model predictions. The fastText model also takes advantage of the fact that the classes are imbalanced (some categories appear more often than others) by using the Huffman method to build a tree structure for representing the categories. Therefore, the depth of the tree structure for frequently occurring categories is smaller than the depth of the tree structure for infrequently occurring categories, which further makes the model more computationally efficient.

[0063] The following is a brief introduction to the training process of the fastText model.

[0064] During the training process, a large number of malicious domain names and normal domain names can be obtained from the Internet as training samples. For example, the top 1 million domain names can be selected from the Alexa website. This part of the data is marked as positive samples. Another part of the data is the DGA seed file obtained from the security laboratory, which contains DGA data of about 40 malicious domain name families, nearly 1.2 million data, and this part of the data is marked as negative samples. These positive and negative samples can be processed by the above-mentioned N-grama model for word segmentation. The multiple word segments obtained can be divided into training sets and test sets in a ratio of 6:4. The division of negative samples also needs to consider the domain name family information to ensure that the characteristics of all types of domain names appear in the training set. Construct the input data according to the requirements of the fastText model, and save the training set and test set divided in the previous step in the fasttext_train.txt and fasttext_test.txt files in the format of "__label__+label\t+list of fragments after word segmentation". For example, some data in the fasttext_train.txt file are as follows:

[0065] __lable__abnormal chkrfmwsihaitlj$c ch hk kr rf fm mwws si ih ha ai it tl lj j$$ch chk hkr krf rfm fmw mws wsi sih iha hai ait itltlj lj$;

[0066] __lable__normal google$g go oo og gl le e$$go goo oog ogl gle le$;

[0067] Among them, abnormal is the label of negative samples, and normal is the label of positive samples.

[0068] Then you can use the model training function train_supervised provided by fastText to train the model, where the model training function is as follows:

[0069] classifier=fasttext.train_supervised("fasttext_train.txt");

[0070] The input parameter fasttext_train.txt is the text file containing the labeled training set constructed above, and the function returns the model object classifier trained on the training set.

[0071] Then you can use the model save function to save the model object to the specified file, such as classifier.save_model("fasttext.model.bin"), where the input parameter fasttext.model.bin is the name of the saved file. You can specify the storage path, which will be saved in the current directory by default.

[0072] It can be seen that the biggest feature of the fastText model is that it is simple, with only one hidden layer and one output layer, so the training speed is very fast. It can be trained in minutes on an ordinary CPU, which is several orders of magnitude faster than the training of deep learning models. The fastText model combines the ideas of natural language processing and the hierarchical ideas of neural networks to build a fast text classifier solution, so it can ensure detection accuracy while also improving prediction efficiency.

[0073] After training the fastText model, you can use the following process to test the performance of the trained fastText model.

[0074] First, load the model: classifier = fasttext.load_model("fasttext.model.bin"), where fasttext_test.txt is the test data, and the format is the same as fasttext_train.txt. Input the test set into the fastText model, and get the accuracy and recall of the model on the test set, as shown below:

[0075] result=classifier.test("fasttext_test.txt");

[0076] Among them, result.precision is the accuracy of the model, and result.recall is the recall rate of the model.

[0077] If the accuracy or recall rate of the model does not meet the set requirements, you can use a new training set to retrain the fastText model until the accuracy and recall rate of the trained fastText model meet the set requirements.

[0078] The following experimental data compares the difference in training time between the fastText model and other models. For example, the support vector machine (SVC) and long short-term memory (LSTM) models are used for model training of malicious domain name detection, and compared with the fastText model. SVC is a machine learning model and LSTM is a deep learning model. The following table summarizes the preliminary classification results of the models trained in the three ways under the same data set and machine conditions. The evaluation indicators in the table show that the classification effect of the fastText model is equivalent to that of the deep learning model, and both are greater than the machine learning model.

[0079] Model Accuracy Accuracy Recall SVC 0.9444 0.9166 0.9583 LSTM 0.9724 0.9708 0.9891 fastText 0.9745 0.9622 0.9965

[0080] In addition, experiments show that in terms of model training time, fastText>LSTM>SVC. The training time for machine learning and deep learning models is more than 1 hour, while the fastText model only takes a few minutes, which greatly shortens the time required for model training while ensuring the detection effect. Therefore, using the fastText model to predict malicious domain names can ensure high prediction accuracy and low prediction time at the same time.

[0081] When using the fastText model to detect the domain name to be detected, for example, using the model's prediction function to make a classification prediction for the domain name to be detected "jmowysfox.org", if the predicted classification label is normal, it means that the domain name to be detected is a normal domain name; if the predicted classification label is abnormal, it means that the domain name to be detected is a malicious domain name.

[0082] The prediction process is like pre=classifier.predict("jmowysfo xo rg$j jm mo owwy ys sf fo ox x..o or rg g$$jm jmo mow owy wys ysf sfo fox ox.xoor org rg$").

[0083] Among them, pre represents the detection result. For example, if the detection result obtained is: (('__label__abnormal'), array([0.99966717])), it means that the detection result is abnormal, indicating that the domain name to be detected "jmowysfox.org" is a malicious domain name, and 0.99966717 represents the probability that the domain name to be detected is a malicious domain name.

[0084] In some other embodiments, in order to further improve the detection accuracy, the hidden layer in the fastText model can also be set to multiple layers, such as two or three layers, so that more hidden features in the input word segmentation can be extracted through the hidden layer, thereby improving the prediction accuracy. It can be understood that when the hidden layer is multiple layers, the training process and prediction process of the model are more similar to the above process, and no more details will be given here.

[0085] In the above implementation process, multiple word segmentations of the domain name to be detected are obtained, and the multiple word segmentations are input into the fastText model. The fastText model is used to detect whether the domain name to be detected is a malicious domain name to obtain a detection result. Since the fastText model can realize the text classification function using a shallow neural network, in the text classification task, the shallow network can often achieve an accuracy comparable to that of the deep network. Therefore, the fastText model is used to detect malicious domain names, which can achieve the detection accuracy that can be achieved by the deep network. Compared with the existing detection method based on regular expressions or black and white lists, the detection accuracy of this solution is higher.

[0086] On the basis of the above embodiment, in order to further explore the association relationship between each word segmentation to further improve the prediction accuracy of the fastText model, the N-grama model can be used to segment the domain name to be detected to obtain multiple initial word segmentations, and then the data association analysis method can be used to analyze the association relationship between each initial word segmentation, and multiple word segmentations with strong associations can be screened out according to the association relationship between each initial word segmentation.

[0087] Among them, the way of segmenting the domain name to be detected using the N-grama model can refer to the relevant description in the above embodiment, and will not be repeated here. The correlation between each initial segmentation can be understood as the probability that the second initial segmentation will appear when the first initial segmentation appears in the two initial segmentations. Its probability can be obtained through a transaction set that has been counted in advance. The transaction set refers to a combination of a large number of malicious domain names. Each domain name can be called a transaction. A transaction contains multiple items, and an item can refer to a segmentation.

[0088] The following is an introduction to several concepts that appear in data association analysis:

[0089] Association rules (i.e., association relationships): are used to discover the connections between initial participles. Association rules are implications of the form X→Y, where X and Y are respectively called the predecessor and successor of the association rule.

[0090] Support: The support of association rule X→Y is denoted as Supp(X→Y), which represents the frequency ratio of item sets X and Y in the transaction set.

[0091] Confidence: It can also be called credibility. The confidence of association rule X→Y can be recorded as conf(X→Y), which is the ratio of the number of times item sets X and Y appear in the transaction set to the number of times item set X appears in the transaction set.

[0092] Lifting represents the probability of having item set Y when there is item set X in a transaction, which reflects the correlation between item set X and item set Y in the association rule.

[0093] Strong association rules (i.e. strong association relationships): used to represent valuable association rules.

[0094] For example, the association rule {go}→{o} implies the probability that the participle "o" will appear when the participle "go" appears. If the probability is large, the association rule is called a strong association rule, indicating that when the participle "go" appears, there is a high probability that the participle "o" will appear later.

[0095] Therefore, after segmenting the domain name to be detected and obtaining multiple initial segmentations, the support, confidence or lift data between each two adjacent initial segmentations can be calculated through the transaction set, and then the data can be used to determine whether there is a strong association rule (i.e., a strong association relationship) between the two adjacent initial segmentations. For example, if the support of two adjacent initial segmentations is greater than the set support threshold, the two adjacent initial segmentations are considered to have a strong association relationship, or if the confidence of two adjacent initial segmentations is greater than the set confidence threshold, the two adjacent initial segmentations are considered to have a strong association relationship, or if the lift of two adjacent initial segmentations is greater than the lift threshold, the two adjacent initial segmentations are considered to have a strong association relationship. Of course, at least one of the three of support, confidence and lift meets the condition to consider that two adjacent initial segmentations have a strong association relationship.

[0096] Alternatively, the association relationship analysis may be performed not only on two adjacent initial segmented words, but also on any two initial segmented words, and the analysis method is similar. In this way, multiple initial segmented words with strong association relationships can be screened out, and these initial segmented words can be used as multiple segmented words input into the fastText model, so that the association relationship between the segmented words can be preliminarily analyzed through the data association analysis method, and then the segmented words with strong association relationships are input into the fastText model. The fastText model is used to further analyze whether there is a certain connection between these segmented words with strong association relationships, thereby further improving the accuracy of malicious domain name detection.

[0097] In other embodiments, in order to increase the amount of data input into the fastText model so that the fastText model can detect malicious domain names based on more data, the N-grama model can be used to segment the domain name to be detected, and after obtaining multiple initial segmentations, the multiple initial segmentations are combined according to a set combination method to obtain multiple combined segmentations.

[0098] Among them, the combination method can be set such as combining at least every two adjacent initial participles, or combining at least any two initial participles among multiple initial participles. Of course, there can be other combination methods, such as combining multiple initial participles with strong correlations screened by the data association analysis method in pairs or in any combination. After such combination, a large number of combined participles can be obtained, and then the combined participles and multiple initial participles can be input into the fastText model, so that the input data volume of the fastText model is larger, so that the fastText model can analyze the rationality of the arrangement order between each participle in the domain name to be detected by analyzing the implicit relationship in a large amount of data, and then the fastText model can use more data to detect malicious domain names and improve the detection accuracy.

[0099] On the basis of the above embodiment, in order to reduce the prediction workload of the fastText model, after obtaining the domain name to be detected, a preliminary detection can be performed on the domain name to be detected, such as calculating the similarity between the domain name to be detected and multiple malicious domain names stored in advance, and obtaining the similarity between the domain name to be detected and each malicious domain name. If the similarity is greater than the set similarity by more than a set ratio, the domain name to be detected is determined to be a suspicious domain name, and then the domain name to be detected is subjected to subsequent word segmentation processing.

[0100] Of course, the domain name to be detected can also be compared with each normal domain name in the pre-built normal domain name library. If the domain name to be detected is consistent with a normal domain name, it means that the domain name to be detected is a normal domain name. At this time, the domain name to be detected does not need to be processed later, so for the domain name to be detected that is originally a normal domain name, there is no need to use the fastText model for detection, which reduces the number of detections of the fastText model and improves the detection efficiency. If the domain name to be detected is inconsistent with each normal domain name in the normal domain name library, it will be compared with each malicious domain name for similarity.

[0101] A large number of malicious domain names are pre-stated. If there are 1,000 malicious domain names in the database, the domain name to be detected will be compared with each malicious domain name for similarity. If the number of similarities greater than the set similarity exceeds the set ratio, the domain name to be detected is considered to be a suspicious malicious domain name. For example, the similarity is set to 80% (the value of the set similarity can be flexibly set according to actual needs), and the ratio is set to 60% (the value of the set ratio can also be flexibly set according to actual needs). Therefore, if the number of similarities greater than 80% exceeds 60%, it means that more than 600 malicious domain names have a similarity greater than 80% with the domain name to be detected. At this time, it can be considered that the domain name to be detected is very likely to be a malicious domain name, that is, a suspicious domain name.

[0102] In the above implementation process, the similarity between the domain name to be detected and multiple malicious domain names is calculated in advance, and subsequent detection is performed only when the domain name to be detected is a suspicious domain name. This can reduce the detection amount of the subsequent fastText model. When there are a large number of domain names to be detected in the network, this can improve the detection efficiency.

[0103] When calculating the similarity between the domain name to be detected and the malicious domain name in the above embodiment, the domain name to be detected and the malicious domain name can be converted into word vectors, and then the cosine similarity or Euclidean distance between the two word vectors is compared, so that the similarity between the two domain names can be obtained.

[0104] Alternatively, the fuzzy hash value of the domain name to be detected and the fuzzy hash value of each malicious domain name may be calculated, and then the fuzzy hash value of the domain name to be detected and the fuzzy hash value of each malicious domain name are compared for similarity to obtain the similarity between the domain name to be detected and each malicious domain name.

[0105] The fuzzy hash values ​​of the domain name to be detected and each malicious domain name can be calculated by using the currently mature context triggered piecewise hashing (CTPH) algorithm based on text segmentation to calculate a fuzzy hash value of the file.

[0106] The algorithm is as follows:

[0107] 1. Sharding: Read a part of the content in the domain name, calculate it with a weak hash algorithm, and get a hash value. Usually, the fixed-length content is read byte by byte, just like the sliding window in the network protocol, sliding in the domain name with a fixed window, and calculating the content in the window each time. Therefore, for convenience, a rolling hashing algorithm is usually used. The rolling hash here means that, for example, the hash value h1 of abcdef has been calculated, and the hash value of bcdefg is to be calculated next. There is no need to completely recalculate, only h1-X(a)+Y(g), where X and Y are two functions, that is, only the effect of the difference on the hash value needs to be increased or decreased accordingly. This kind of hashing can greatly speed up the speed of sharding judgment.

[0108] The commonly used Alder-32[4] algorithm and CRC32 algorithm can be used as weak hash algorithms for sharding. In addition to the weak hash algorithm, a sharding trigger value n is also required to control the sharding condition.

[0109] In order to avoid the situation where the number of shards is too small, resulting in the entire domain name triggering the sharding condition only once, or not triggering the sharding condition at all, you can decide how to shard based on the length of the domain name and the actual content of the domain name.

[0110] 2. Calculate the hash value for each piece

[0111] For each shard, the hash value can be calculated using a traditional algorithm such as MD5, or a hash algorithm such as Fowler-Noll-Vohash[5].

[0112] 3. Connection hash value

[0113] By concatenating the hash values ​​of each shard, you can get the fuzzy hash value of the entire domain name. If the shard condition parameter n has different values ​​for different domain names, you also need to include n in the fuzzy hash value.

[0114] Therefore, the fuzzy hash values ​​of the domain name to be detected and each malicious domain name can be calculated in the above manner, and then the fuzzy hash value of the domain name to be detected and the fuzzy hash value of the malicious domain name can be compared for similarity. The similarity comparison method can use a distance algorithm such as Hamming distance or minimum edit distance to calculate the distance value, and this distance value can be used to evaluate the similarity.

[0115] In the above implementation process, the fuzzy hash value can more accurately compare the similarity between two strings, because the addition, modification, deletion and other changes of data in a certain shard of the domain name will only affect the hash of the shard, and have little impact on the global situation, so the impact on the final similarity is not significant. Even if several consecutive characters are changed, or multiple changes are made, the fuzzy hash algorithm can still make an effective judgment, and then by comparing the fuzzy hash values ​​between the domain names, the similarity between the two domain names can be more accurately judged.

[0116] Please refer to Figure 3 , Figure 3 This is a block diagram of a malicious domain name detection device 200 provided in an embodiment of the present application. The device 200 may be a module, program segment or code on an electronic device. It should be understood that the device 200 is similar to the above-mentioned Figure 1 The method embodiment corresponds to and can be executed Figure 1 The various steps involved in the method embodiment and the specific functions of the device 200 can be found in the above description. To avoid repetition, the detailed description is appropriately omitted here.

[0117] Optionally, the device 200 includes:

[0118] A domain name acquisition module 210 is used to acquire a domain name to be detected;

[0119] A word segmentation module 220, used to segment the domain name to be detected to obtain multiple word segments;

[0120] The detection module 230 is used to input the multiple word segments into the fastText model, detect whether the domain name to be detected is a malicious domain name through the fastText model, and obtain a detection result.

[0121] Optionally, the fastText model includes an input layer, a hidden layer and an output layer, the multiple word segmentations are used as inputs of the input layer, the input layer is used to convert the multiple word segmentations into corresponding word vectors, the hidden layer is used to perform superposition and averaging processing on the word vectors, and the output layer is used to output the detection result of the domain name to be detected based on the processing result of the hidden layer.

[0122] Optionally, the word segmentation module 220 is used to segment the domain name to be detected by using an N-grama model to obtain multiple word segments.

[0123] Optionally, the word segmentation module 220 is used to segment the domain name to be detected using an N-grama model to obtain multiple initial word segmentations; use a data association analysis method to analyze the association relationship between the initial word segmentations; and screen out multiple word segmentations with strong associations based on the association relationship between the initial word segmentations.

[0124] Optionally, the word segmentation module 220 is used to use the N-grama model to segment the domain name to be detected to obtain multiple initial word segmentations; combine the multiple initial word segmentations according to a set combination method to obtain multiple combined word segmentations; wherein the input data of the fastText model includes the multiple initial word segmentations and the multiple combined word segmentations.

[0125] Optionally, the device 200 further includes:

[0126] The pre-detection module is used to calculate the similarity between the domain name to be detected and multiple malicious domain names stored in advance to obtain the similarity between the domain name to be detected and each malicious domain name; if the similarity is greater than the set similarity by more than a set ratio, the domain name to be detected is determined to be a suspicious domain name.

[0127] Optionally, the pre-detection module is used to calculate the fuzzy hash value of the domain name to be detected, and calculate the fuzzy hash value of each malicious domain name stored in advance; compare the fuzzy hash value of the domain name to be detected with the fuzzy hash value of each malicious domain name for similarity, and obtain the similarity between the domain name to be detected and each malicious domain name.

[0128] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0129] Please refer to Figure 4 , Figure 4A structural diagram of an electronic device for executing a malicious domain name detection method provided in an embodiment of the present application, the electronic device may include: at least one processor 310, such as a CPU, at least one communication interface 320, at least one memory 330 and at least one communication bus 340. Among them, the communication bus 340 is used to realize direct connection and communication between these components. Among them, the communication interface 320 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 330 can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 330 can optionally also be at least one storage device located away from the aforementioned processor. Computer-readable instructions are stored in the memory 330. When the computer-readable instructions are executed by the processor 310, the electronic device executes the above-mentioned Figure 1 The method process is shown.

[0130] Understandably, Figure 4 The structure shown is for illustration only, and the electronic device may also include Figure 4 More or fewer components as shown, or with Figure 4 Different configurations shown. Figure 4 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0131] The present application embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the following is performed: Figure 1 The method process in the method embodiment shown is executed by the electronic device.

[0132] The present embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining a domain name to be detected; segmenting the domain name to be detected to obtain multiple segmented words; inputting the multiple segmented words into a fastText model, detecting whether the domain name to be detected is a malicious domain name through the fastText model, and obtaining a detection result.

[0133] In summary, the embodiments of the present application provide a malicious domain name detection method, device, electronic device and storage medium. The method obtains multiple word segments of the domain name to be detected, inputs the multiple word segments into the fastText model, and detects whether the domain name to be detected is a malicious domain name through the fastText model to obtain a detection result. Since the fastText model can realize the text classification function using a shallow neural network, in the text classification task, the shallow network can often achieve an accuracy comparable to that of a deep network. Therefore, the fastText model is used to detect malicious domain names, which can achieve the detection accuracy that can be achieved by a deep network. Compared with the existing detection method based on regular expressions or black and white lists, the detection accuracy of this solution is higher.

[0134] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0135] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0137] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0138] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting malicious domain names, characterized in that: The method comprises: Get the domain name to be tested; Segmenting the domain name to be detected to obtain multiple segmented words; Input the multiple word segments into the fastText model, and detect whether the domain name to be detected is a malicious domain name through the fastText model to obtain a detection result; The segmenting of the domain name to be detected to obtain multiple segmentations includes: Using an N-grama model to segment the domain name to be detected to obtain multiple initial segmentations; The data association analysis method is used to analyze the association relationship between each initial participle; According to the correlation between each initial participle, multiple participles with strong correlation are selected.

2. The method according to claim 1, characterized in that The fastText model includes an input layer, a hidden layer and an output layer. The multiple word segmentations are used as inputs of the input layer. The input layer is used to convert the multiple word segmentations into corresponding word vectors. The hidden layer is used to perform superposition and averaging processing on the word vectors. The output layer is used to output the detection result of the domain name to be detected based on the processing result of the hidden layer.

3. The method according to claim 1, characterized in that After obtaining the domain name to be detected, segmenting the domain name to be detected, and before obtaining a plurality of segmented words, the method further includes: Calculating similarity between the domain name to be detected and a plurality of pre-stored malicious domain names to obtain similarity between the domain name to be detected and each malicious domain name; If the number of similarities greater than the set similarity exceeds a set ratio, the domain name to be detected is determined to be a suspicious domain name, and the step of performing word segmentation on the domain name to be detected is executed.

4. The method according to claim 3, characterized in that The calculating the similarity between the domain name to be detected and a plurality of pre-stored malicious domain names to obtain the similarity between the domain name to be detected and each malicious domain name includes: Calculating the fuzzy hash value of the domain name to be detected, and calculating the fuzzy hash value of each pre-stored malicious domain name; The fuzzy hash value of the domain name to be detected is compared with the fuzzy hash values ​​of each malicious domain name for similarity, so as to obtain the similarity between the domain name to be detected and each malicious domain name.

5. A malicious domain name detection device, characterized in that: The device comprises: A domain name acquisition module is used to obtain the domain name to be detected; A word segmentation module, used to segment the domain name to be detected to obtain multiple word segments; A detection module, used for inputting the multiple word segments into a fastText model, detecting whether the domain name to be detected is a malicious domain name through the fastText model, and obtaining a detection result; Among them, the word segmentation module is specifically used to use the N-grama model to segment the domain name to be detected to obtain multiple initial word segments; use the data association analysis method to analyze the association relationship between each initial word segment; and screen out multiple word segments with strong associations based on the association relationship between each initial word segment.

6. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 4 is executed.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is performed.

Images