A malicious code clustering method and system

Abstract

The present application relates to a kind of malicious code clustering method and system, it is related to computer field, method includes obtaining malicious code fusion semantic text;The feature extraction is carried out to the malicious code fusion semantic text, obtains fusion semantic sequence;According to the fusion semantic sequence, text vector prediction is carried out using malicious code behavior semantic representation model, obtains behavior semantic vector;The malicious code behavior semantic representation model includes distributed storage model and distributed bag-of-words library model;According to the behavior semantic vector, clustering is carried out using malicious code clustering model, obtains the malicious code after clustering;The malicious code clustering model is the self-organizing mapping neural network of well-trained.This application can improve the clustering effect of malicious code.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, and particularly to a malicious code clustering method and system. BACKGROUND

[0002] With the high interconnection of cyberspace, the threat of malicious code is a problem that cannot be avoided in the field of cyberspace security. Most of these malicious codes are derived from existing malicious codes, that is, malicious code developers save costs by modifying existing malicious codes to produce a new malicious code. Such malicious codes often contain similar malicious code behavior characteristics, such as similar control flow, code writing habits, code fragments, key technologies, and API calling interfaces. Therefore, by analyzing the behavior characteristics of malicious code families, new malicious codes can be discovered, which helps to study the variation rules of malicious codes and further trace them.

[0003] The malicious code behavior analysis method is an important research direction in the field of malicious code analysis. With the sharp rise in the number of malicious code variants and the widespread use of malicious code obfuscation techniques, malicious code behavior analysis has become increasingly difficult. However, code obfuscation can change the syntax structure to some extent, but it cannot change the behavior of malicious code, that is, regardless of how the code changes, the behavior behind the code will not change, so the malicious code behavior semantic analysis method has become a research hotspot in the field of malicious code analysis.

[0004] Current behavior semantic analysis methods mainly focus on the study of underlying behaviors such as "file modification", "process creation", and "registry modification", without considering the context semantic relationship between these behaviors, and cannot directly obtain higher-level behaviors of malicious codes, that is, it is difficult to abstract higher-level behaviors such as "file read / write" and "network transmission" by judging the changes in the underlying system. SUMMARY

[0005] The purpose of the present application is to provide a malicious code clustering method and system to improve the clustering effect of malicious codes.

[0006] To achieve the above purpose, the present application provides the following solutions:

[0007] A malicious code clustering method, comprising:

[0008] obtaining a malicious code fusion semantic text;

[0009] performing feature extraction on the malicious code fusion semantic text to obtain a fusion semantic sequence;

[0010] According to the fusion semantic sequence, a malicious code behavior semantic representation model is used for text vector prediction to obtain a behavior semantic vector; the malicious code behavior semantic representation model comprises a distributed storage model and a distributed bag-of-words library model.

[0011] According to the behavior semantic vector, a malicious code clustering model is used for clustering to obtain clustered malicious codes; the malicious code clustering model is a trained self-organizing mapping neural network.

[0012] Optionally, the feature extraction on the malicious code fusion semantic text is performed to obtain a fusion semantic sequence, and specifically comprises:

[0013] The malicious code fusion semantic text is subjected to dependency analysis and program block, and a program control flow graph is generated.

[0014] The program control flow graph is traversed, and the fusion semantic sequence is determined according to basic block information and API function information in the program control flow graph.

[0015] Optionally, the text vector prediction on the fusion semantic sequence by using the malicious code behavior semantic representation model to obtain the behavior semantic vector specifically comprises:

[0016] The fusion semantic sequence is preprocessed to obtain a one-hot encoding;

[0017] According to the one-hot encoding, the distributed storage model and the distributed bag-of-words library model are used for prediction respectively to obtain a distributed storage model sentence vector and a distributed bag-of-words library model sentence vector;

[0018] The distributed storage model sentence vector and the distributed bag-of-words library model sentence vector are spliced to obtain the behavior semantic vector.

[0019] Optionally, the training process of the malicious code clustering model comprises:

[0020] With the behavior semantic vector of a sample set as input and the clustered malicious codes of the sample set as output, the self-organizing mapping neural network is trained by using a competitive learning strategy to obtain a malicious code clustering model.

[0021] A malicious code clustering system comprises:

[0022] An acquisition module is configured to acquire malicious code fusion semantic text;

[0023] A feature extraction module is configured to perform feature extraction on the malicious code fusion semantic text to obtain a fusion semantic sequence;

[0024] The prediction module is configured to perform text vector prediction on the fused semantic sequence by using a malicious code behavior semantic representation model to obtain a behavior semantic vector.

[0025] The clustering module is configured to perform clustering on the behavior semantic vector by using a malicious code clustering model to obtain clustered malicious codes.

[0026] Optionally, the feature extraction module specifically comprises:

[0027] The program control flow graph generation unit is configured to perform dependency analysis and program block on the malicious code fused semantic text to generate a program control flow graph.

[0028] The traversal unit is configured to traverse the program control flow graph, and determine the fused semantic sequence according to basic block information and API function information in the program control flow graph.

[0029] Optionally, the prediction module specifically comprises:

[0030] The preprocessing unit is configured to perform preprocessing on the fused semantic sequence to obtain one-hot encoding.

[0031] The prediction unit is configured to perform prediction on the one-hot encoding by using the distributed storage model and the distributed bag-of-words library model respectively to obtain a distributed storage model sentence vector and a distributed bag-of-words library model sentence vector.

[0032] The splicing unit is configured to splice the distributed storage model sentence vector and the distributed bag-of-words library model sentence vector to obtain a behavior semantic vector.

[0033] Optionally, the training process of the malicious code clustering model comprises:

[0034] The self-organizing mapping neural network is trained by using a competition learning strategy, and a malicious code clustering model is obtained.

[0035] According to the specific embodiments of the present application, the following technical effects are provided:

[0036] The application obtains malicious code fusion semantic text; feature extraction is performed on the malicious code fusion semantic text to obtain a fusion semantic sequence; a malicious code behavior semantic representation model is used to perform text vector prediction according to the fusion semantic sequence to obtain a behavior semantic vector; the malicious code behavior semantic representation model comprises a distributed storage model and a distributed bag-of-words library model; a malicious code clustering model is used to perform clustering according to the behavior semantic vector to obtain clustered malicious codes; the malicious code clustering model is a trained self-organizing mapping neural network. The malicious code behavior semantic representation model is used to effectively learn the context information of malicious code behaviors, and the malicious code clustering model is used to realize malicious code clustering, thereby improving the clustering effect of malicious codes. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 A malicious code part control flow graph is provided for the present application.

[0039] Figure 2 A fusion semantic feature schematic diagram is provided for the present application.

[0040] Figure 3 A PV-DM model schematic diagram is provided for the present application.

[0041] Figure 4 A PV-DBOW model schematic diagram is provided for the present application.

[0042] Figure 5 A malicious code clustering schematic diagram is provided for the present application.

[0043] Figure 6 A field structure diagram is provided for the present application.

[0044] Figure 7 A SOM clustering model schematic diagram is provided for the present application.

[0045] Figure 8 A neighborhood change schematic diagram is provided for the present application.

[0046] Figure 9 A malicious code clustering method flowchart schematic diagram is provided for the present application. DETAILED DESCRIPTION

[0047] With reference to the accompanying drawings: clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of the present application.

[0048] The purpose of the present application is to provide a malicious code clustering method and system to improve the clustering effect of malicious code.

[0049] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0050] As shown in Figure 9 The present application provides a malicious code clustering method, comprising:

[0051] Step 101: Obtain a malicious code fusion semantic text.

[0052] Step 102: Feature extraction is performed on the malicious code fusion semantic text to obtain a fusion semantic sequence.

[0053] In step 102, the feature extraction of the malicious code fusion semantic text to obtain a fusion semantic sequence specifically includes: performing dependency analysis and program block on the malicious code fusion semantic text to generate a program control flow graph; traversing the program control flow graph, and determining a fusion semantic sequence according to the basic block information and API function information in the program control flow graph.

[0054] For malicious code semantic analysis, the calculation of program control flow graph CFG is a very basic and important step. For the acquisition of CFG, the disassembly tool IDA Pro is usually used to generate a disassembly file.asm, and the control flow graph is generated through function call relationship and program jump, but the granularity is relatively coarse and the readability is poor. Therefore, the Angr binary analysis tool is used to calculate CFG in the present application. Compared with IDA Pro, Angr has the following advantages: first, angr can generate static CFG and dynamic CFG through static analysis and symbolic execution, and the granularity is finer with basic blocks as nodes; second, the control flow graph constructed in general is lack of function call relationship between functions, while angr can automatically generate inter-process control flow graph ICFG of the program from the executable file, which includes both function call information and program control flow information. In this way, by traversing the global ICFG, the key semantic information of the malicious code can be obtained.

[0055] The traditional semantic feature extraction method is to extract API call sequences and assembly code sequences as two different feature forms through disassembly files or dynamic detection reports. When malicious code implements malicious behavior, it needs to call API functions, and this action is performed according to assembly instructions. In order to mine the deeper relationship between API calls and assembly instructions, that is, the API function call mode of malicious code, key nodes are screened out from ICFG according to API correlation, the assembly instruction sequence of the key nodes is extracted, and the API functions associated with them are combined to form a mixed sequence containing API call sequences and basic block sequences. This sequence is written into a file as a text feature for clustering analysis. In order to clearly describe this process, how to screen key nodes and extract mixed sequences is illustrated by taking Figure 1 as an example.

[0056] The ICFG of malicious code contains a large number of basic block nodes. Key basic blocks are screened out by whether they contain API call instructions, and the remaining invalid basic blocks are discarded. Figure 1 is a local control flow graph of a malicious code. From the graph, it can be seen that node 0x500084 represents calling the API function InitializeCriticalSection, and node 0x500080 represents calling the API function EnterCriticalSection, which can be reached from basic block 0x401850 and basic block 0x401874, respectively. Here, the two API call paths (0x401850->0x500084) and (0x401874->0x500080) are mainly concerned. For the API call path (0x401850->0x500084), the intermediate basic block 0x4011a4 indirectly calls the API function through secondary jump, and there is no actual semantic information, so the information of the intermediate basic block will be discarded when extracting the fused semantic sequence. For the API call path (0x401874->0x500080), the same is true, and the fused semantic sequence extracted does not contain the information of the intermediate basic block 0x4011ac.

[0057] Therefore, the steps of the algorithm are as follows:

[0058] Step 1: load the binary malicious code program into the angr analysis system;

[0059] Step 2: perform dependency analysis and program blocking to generate the control flow graph ICFG;

[0060] Step 3: traverse the entire ICFG from the root node, and store the API function call relationship, basic block information, and API function call address in the basic block for each API call path;

[0061] Step four: according to the information obtained in step three, generate a fusion semantic sequence containing API functions and assembly instruction sequences for each API call path;

[0062] Step five: combine each fusion semantic sequence and write it into the text invention.

[0063] As shown in Figure 2 , the fusion semantic sequence generation process of API call sequence and API associated basic block sequence is described.

[0064] Step 103: text vector prediction is performed according to the fusion semantic sequence by using a malicious code behavior semantic representation model to obtain a behavior semantic vector; the malicious code behavior semantic representation model includes a distributed storage model and a distributed bag of words library model.

[0065] Step 103, specifically comprising: pre-processing the fusion semantic sequence to obtain one-hot encoding; according to the one-hot encoding, respectively using the distributed storage model and the distributed bag of words library model to predict, obtaining a distributed storage model sentence vector and a distributed bag of words library model sentence vector; the distributed storage model sentence vector and the distributed bag of words library model sentence vector are spliced to obtain a behavior semantic vector.

[0066] From the perspective of word vector splicing, the deep semantic of text is expressed by Doc2vec document vector (Distributed representations of Sentences and Documents, Doc2vec), and finally the final malicious code behavior semantic representation is obtained by vector splicing.

[0067] After feature extraction, each malicious code corresponds to a text of a fusion semantic sequence. Since the operand representation of assembly instructions is various and carries less semantic information, in order to reduce the influence of noise on malicious code behavior semantic analysis, the fusion semantic sequence text is standardized before word segmentation, as shown in Figure 2 The specific processing rules of pre-processing are as follows:

[0068] 1) Replace the memory address type operand with "addr".

[0069] 2) For constant type operands, replace all greater than 5000 with "imme", and keep those less than 5000.

[0070] After normalization, the fused semantic text needs to be segmented, such as push, ebp, mov, ebp, esp, call, InitializeCriticalSection, and each represents a word. In this way, the fused semantic sequence extracted from the malicious code can be regarded as a paragraph or a sentence composed of API function words and operation code operand words.

[0071] Doc2vec is also called paragraph vector, which is developed on the basis of word2vec. It does not limit the length of the sentence, and any long or short sentence can be used as a training sample. The algorithm not only generates a word vector for each word, but also generates a sentence vector for each sentence, solving the problem of word2vec that the order between words and the sentence syntax. In the Doc2vec paragraph vector framework, each sentence is mapped to a unique vector represented by a list in matrix D, and each word in the sentence is also mapped to a unique vector represented by a list in matrix W. Both paragraph vectors and word vectors will appear in the prediction process of the next word. Therefore, the paragraph vector can be regarded as another word, which functions as a memory that can remember the current context and contains the content of the entire sentence.

[0072] Like the word2vec algorithm, Doc2vec also has two models: the distributed memory model of paragraph vectors (PV-DM) and the distributed bag of words version of paragraph vector (PV-DBOW).

[0073] Distributed memory model (PV-DM)

[0074] In the actual training process of the model, the length of the context depends on the size of the sliding window, which is set initially and does not change. For a whole sentence, the context is obtained by sampling from the sliding window on the sentence. For the same sentence, the context information obtained each time is different, and only contains partial information of the entire sentence. Therefore, in the prediction of the context information generated from the same sentence, the sentence vector is shared in all contexts, but the sentence vector cannot be shared across sentences, while the word vector matrix can be shared across sentences.

[0075] As Figure 3As shown, the PV-DM model samples a fixed length of words from a sentence each time, takes one of the words as the predicted word, and takes the others as the input word. During each training, a small part of the words in the sentence is intercepted by sliding, and the paragraph vector is shared in several training of the same sentence. Thus, in each training process, not only the words are trained, but also the paragraph vector of the paragraph is trained in the process of several times of sliding of a sentence each time, and the theme expressed by the vector is more and more accurate. The model structure mainly includes an input layer, a hidden layer and an output layer.

[0076] The input layer includes vectors w of 2a context words i and a sentence vector p storing memory information i .

[0077] The vector of the hidden layer depends not only on the word vector of each word, but also on the sentence vector of the whole sentence. The calculation method of the hidden layer vector is to average the vectors of 2a context words and 1 sentence vector in the document. The variables in the formula are represented by single letters, and the properties or variable parts can be represented by subscripts. That is

[0078]

[0079] The output layer adopts a hierarchical softmax structure, and the core idea is to construct a Huffman tree, the leaf nodes represent each word in the table, and the root node is the input of the output layer, and then the probability of each word is obtained through the binary classification of the Huffman tree layer by layer. The word with the maximum probability is the predicted intermediate word, which is compared with the real intermediate word, and the smaller the error is, the better it is, and the weight matrix is updated according to the error.

[0080] The sentence vector and the word vector are trained by stochastic gradient descent and back propagation. In the process of changing the stochastic gradient, a fixed length of context is sampled from the sliding window of the sentence each time, and then the error gradient is calculated through the network, and the parameter information in the model is updated according to the gradient error. Figure 3

[0081] Suppose there are N sentences and M words in the corpus, and each sentence is mapped to a p-dimensional vector and each word is mapped to a q-dimensional vector through model training. Thus, there are N*p+M*q parameters in the model in total.

[0082] In the Huffman tree, the softmax mapping from the hidden layer to the output layer is not completed at once, but is completed layer by layer along the structure of the tree, that is, the binary logistic regression method is used to judge whether to go along the left subtree or the right subtree. The method of judging is to calculate the probability of the two sides using the sigmoid function, and to go along the side with the larger probability. The calculation formula is:​

[0083]

[0084] p(right) = 1 - p(left) (3)

[0085] where x w is the word vector of the current internal node, θ is the model parameter of logistic regression that needs to be solved from the training samples. p(left) is the probability of walking along the left subtree, σ is the sigmoid function, and the specific expression of the function is is the transpose of the word vector of the current internal node, and p(right) is the probability of walking along the right subtree.

[0086] Returning to the hierarchical softmax itself, the goal of model training is to find suitable word vectors of all nodes and parameters of all internal nodes, so as to maximize the average log-likelihood function, and the calculation method is as follows:

[0087]

[0088] where M is the number of words, k is the size of the sliding window, d i is the paragraph vector of the sentence in which the context word is located. i is the serial number of the middle word in the sliding window, and P is the probability of predicting the target word w i .

[0089] After training, all word vectors in the training samples and the sentence vectors corresponding to each sentence are obtained. When predicting a new sentence, the new sentence vector is initialized and then put into the model to continuously iterate to obtain the final stable sentence vector by stochastic gradient descent. However, the word vectors in the model and the parameters from the hidden layer to the output layer do not change during the prediction of the new sentence vector, and only the vector of the new sentence is updated in the continuous iteration.

[0090] Distributed bag-of-words model PV-DBOW

[0091] The second model of Doc2vec is the distributed bag-of-words model PV-DBOW as shown in Figure 4 . This training method ignores the context information of the paragraph and makes the model randomly predict a word in the paragraph.

[0092] The specific process is to randomly take a word from the window as a prediction task when sliding the fixed window each time, thereby forming a multi-classification task of word prediction for a given paragraph vector, and the model structure is as shown in Figure 4 .

[0093] For a given training text, the size of the sliding window is fixed, and as the window slides, the words in the paragraph are predicted based on the paragraph vector of the paragraph where the current window is located.

[0094] Both model training processes treat a text as a paragraph or a sentence, and the words in the text as input, train the text vector and network weight, and in the prediction stage, fix the word vector and network weight, and only update the vector of the new text until convergence. Depending on the subsequent task, the performance of the PV-DBOW model and the PV-DM model will be different. Which model performs better needs to be determined according to the specific circumstances of the subsequent task. However, the combination of the two often performs better in completing the task. Therefore, in the present application, the final representation of the malicious code behavior semantics is obtained by splicing the vectors obtained by the distributed storage model PV-DM and the distributed bag-of-words model PV-DBOW. The process of representing malicious code behavior semantics by splicing the two models is shown in Table 1.

[0095] Table 1 Process of representing malicious code behavior semantics by splicing the two models

[0096]

[0097] First, the preprocessed text is one-hot encoded; second, the obtained one-hot encoding is put into the pre-trained PV-DM and PV-DBOW models respectively to obtain the corresponding sentence vector; third, the two sentence vectors are spliced to obtain the complete behavior semantic feature.

[0098] Step 104: clustering using a malicious code clustering model according to the behavior semantic vector to obtain clustered malicious codes; the malicious code clustering model is a trained self-organizing mapping neural network (SOM).

[0099] The training process of the malicious code clustering model includes: taking the behavior semantic vector of the sample set as input, taking the clustered malicious codes of the sample set as output, and training the self-organizing mapping neural network using a competitive learning strategy to obtain the malicious code clustering model.

[0100] After the malicious code behavior semantic vector is extracted, it needs to be clustered and identified. Each vector represents a malicious code, and the similarity of malicious codes is determined by the spatial distance between vectors. The malicious code clustering analysis process is as shown in Figure 5 .

[0101] SOM is a kind of unsupervised competitive learning neural network, by simulating the characteristics of human brain processing signal, relying on the mutual competition between neurons and the near neighbor relationship function to maintain the original topology of input layer. The network structure is proposed by Kohonen in 1981, compared with traditional clustering algorithm, its characteristics are using competitive learning strategy, unsupervised learning model, no additional label is needed, can be mapped to the surface or plane in the case of maintaining the topology of input data, with high accuracy and robustness.

[0102] The network structure of SOM includes input layer and competitive layer. The number of neurons of input layer depends on the dimension of input vector, one neuron corresponds to one dimension. The competitive layer is also called output layer, the structure of competitive layer depends on the number of neurons and the topology of neurons, the common topology is two-dimensional plane structure, commonly used are rectangle and hexagon, as shown in Figure 6 The neighborhood topology is centered at 0, 1 is directly adjacent to 0, and 2 is farthest from 0

[0103] In the SOM of the application, the competitive layer adopts two-dimensional plane topology structure, the arrangement mode of neurons adopts rectangular arrangement, and the winning neurons are selected through the competitive learning strategy. The SOM clustering model for malicious code behavior semantic vector is as shown in Figure 7 .

[0104] In the clustering analysis process of malicious code, the model input layer has n neurons, indicating that an n-dimensional vector is input, representing the behavior semantics of malicious code, and the competitive layer is a two-dimensional hexagonal lattice neuron, and the input layer neurons and the competitive layer neurons are fully connected. The specific learning process of the malicious code clustering model based on SOM includes competition, cooperation and update.

[0105] Competition

[0106] The competition process needs to first assign initial values to the weight vectors of the output layer neurons and perform normalization processing, and assign an initial value to the learning rate. The normalization processing formula is as follows:

[0107]

[0108]

[0109] Among them, is the normalized input vector, is the normalized neuron weight vector, for the input vector X=(x1, x2,..., xn), by calculating the input vector X and each competitive layer neuron weight vector W n i ​The node with the minimum distance is selected as the winning neuron. The inner product between the vectors is calculated as the competition strategy, and the mathematical formula is as follows:

[0110]

[0111] where x i is an element in the input vector X=(x1, x2,..., xn), y j is an element in the weight vector of a competitive layer node. N is the dimension of the input vector X. When the input vector is X, the tth neuron wins, and the following condition must be met:

[0112]

[0113] W t is the weight vector of the winning neuron.

[0114] Cooperation

[0115] After determining the winning neuron, the shape of the neighborhood needs to be selected. Figure 6 Two common neighborhood shapes for self-organizing mapping neural networks are square neighborhood and hexagonal neighborhood. The malware clustering model in this paper uses hexagonal neighborhood. In the neighborhood range, the dark gray solid circle in the center represents the winning neuron, and the surrounding hollow neurons represent other neurons in the neighborhood. The neurons in the neighborhood will be in an active state and will be updated to different degrees, while the neurons outside the neighborhood will be in a dormant state and will be inhibited to different degrees.

[0116] In the training process, the neighborhood range depends on the size of the neighborhood radius, which is denoted as N i (n), representing the topological neighborhood radius after n iterations, i.e., the area covered by the neighborhood. During training, N i (n) changes constantly, and its change rule is as follows:

[0117]

[0118] where INT(·) represents the integer function, N i (0) is the initial value of the topological neighborhood, and N is the number of iterations

[0119] Figure 8 The formula (9) describes the change of the neighborhood under the constraint condition. By analyzing formula (9) and the overall change of the hexagonal neighborhood shape, it can be seen that the topological range of the neighborhood is in a state of contraction as the number of iterations increases.

[0120] Update

[0121] The weight state of all winning neurons in the neighborhood interval is updated until the end condition is met.

[0122] W j (n+1)=W j (n)+θ(u,v,j)η(n)(X-W j (n)) (10)

[0123] Wherein, the value of n is from 1 to N, and θ (u, v, j) is a neighborhood node update amplitude constraint, and the amplitude of each update is calculated based on the distance of the neighborhood node from the winning node, and the basic idea is that the closer to the winning node, the greater the update amplitude; the farther from the winning node, the smaller the update amplitude. η (n) is a learning rate, and η (n) will gradually decrease with the increase of the training number, that is, the amplitude of adjustment will gradually decrease, and the adjustment of the weight gradually tends to the cluster center. The change rule of η (n) is:

[0124]

[0125] Wherein, η (0) is an initial learning rate, and N is the total iteration number. When the learning rate η (n) is less than the initial set threshold, the training is ended;Otherwise, the adjusted weight state is normalized, and the above steps are repeated to iterate the training until the end condition is met.

[0126] The application uses a new malicious code behavior feature extraction method, extracts key information reflecting the behavior of malicious code by analyzing a large number of malicious code binary files. The malicious code behavior semantic representation model based on doc2vec effectively learns the context semantic information of the malicious code behavior, and the malicious code clustering model based on self-organizing mapping neural network SOM is used to realize the clustering of malicious code by mapping the sample to the winning neuron through the learning mode of neuron competition. Starting from the behavior semantic feature, the malicious code executable file is analyzed by disassembly, and the API call sequence and the basic block sequence associated with the API of the malicious code are selected as the semantic feature, the malicious code fusion semantic text is generated, and the clustering method based on self-organizing mapping neural network is used to cluster the malicious code by means of the text analysis idea.

[0127] The application also provides specific data sets and experimental schemes

[0128] The data set used in the experiment of the application is derived from the VX Heavens malicious sample library, which provides a lot of virus source codes and is the main data source for malicious code research. The data set of the application includes six kinds of malicious code families such as trojan, backdoor, worm, downloader and the like. In order to ensure the effectiveness and normativity of the calculation, the original data is cleaned before the behavior semantics extraction and clustering analysis, and the samples that cannot be correctly decompiled and cannot extract effective behavior semantics are removed, and finally 1752 effective malicious code fusion semantics texts are obtained, the size of the semantics text is the longest 895 words and the shortest 98 words. Table 2 illustrates the malicious code family information and sample quantity. The data set is divided into a training set and a test set according to an 8:2 ratio.

[0129] Table 2 Malicious code family information

[0130] Family Name Sample Number OnLineGames 301 Obfuscated 259 Small 289 VB 311 Agent 285 Bifrose 307

[0131] In order to verify the effectiveness of the malicious code behavior semantic representation model based on Doc2vec proposed in the application, a distributed storage model DM and a distributed bag of words model DBOW are set as comparative experiments. In the data processing process, the dimension of the malicious code fusion semantic sequence vector is set to 300, and the sliding window size is set to 15. The number of input layer neurons of the malicious code SOM clustering model is the dimension of the input vector, and in the application, the number of input layer neurons is the dimension of the malicious code behavior semantic vector, and the competitive layer topology structure is set to 14*14.

[0132] In the evaluation of the malicious code clustering model, external evaluation indicators and internal evaluation indicators are used for evaluation. The external evaluation indicators need to use the real label information of the samples for comparative analysis, and the commonly used ones include adjusted rand index (ARI), V-measure, FMI and clustering accuracy AC. The internal indicators can be evaluated without other data, mainly including silhouette coefficient (SC). These indicators comprehensively measure the intra-cluster tightness, inter-cluster separation, classification accuracy and clustering accuracy of the clustering model.

[0133] The value range of ARI is-1 to 1, the larger the value is, the better, which reflects the overlap degree of two divisions. The calculation formula is:

[0134]

[0135] Wherein, E(RI) is the expected value of the rand index, and RI is the rand index, and the calculation formula is:

[0136]

[0137] Wherein, The number of combinations of randomly extracting 2 samples from n samples, a is the number of sample pairs of similar samples belonging to the same cluster, d is the number of sample pairs of similar samples belonging to different clusters, and the denominator represents the number of combinations of any two samples belonging to the same class, which is the total number of sample pairs that can be formed in the data set.

[0138] V-measure is a harmonic mean of homogeneity (h value) and completeness (c value). Homogeneity refers to each cluster containing only samples of a single class, and completeness refers to samples of a given class being assigned to the same cluster. The V-measure value is the harmonic mean of the h value and the c value, and the calculation formula is as follows:

[0139]

[0140] Obviously, the value range of V-measure is 0 to 1, and the larger the V-measure value, the better the clustering effect.

[0141] FMI is the geometric mean of the recall and precision calculated from the clustering result and the true value, and the value range is 0 to 1, and the closer to 1, the better. The calculation formula is:

[0142]

[0143] Where a represents the number of sample pairs of similar samples belonging to the same cluster, b represents the number of sample pairs of dissimilar samples belonging to the same cluster, and c represents the number of sample pairs of similar samples belonging to different clusters.

[0144] The clustering accuracy AC is used to compare the clustering label and the data true label. The calculation formula is:

[0145]

[0146] Where r i ,s i respectively represent the clustering label and the true label corresponding to the sample x i , n is the total number of data, and δ represents the indicator function. If r i ,s i are the same, the function value is 1, otherwise 0.

[0147] For a certain sample, the average distance between the sample and other sample points in the cluster is defined as the cohesion degree a of the cluster, and the average distance between the sample and all sample points in the nearest cluster is defined as the separation degree b between the clusters, then the calculation formula of the silhouette coefficient SC of the sample is as follows:

[0148]

[0149] For the set of all samples, the silhouette coefficient is the average of the silhouette coefficients of each sample. The value ranges from -1 to 1, and when the inter-cluster separation b is much larger than the cohesion a, the value of the silhouette coefficient is approximately 1. Therefore, the closer the value of the index is to 1, the better the clustering effect is.

[0150] The application obtains a binary malicious code sample; semantic feature extraction is performed on the malicious code binary file in step one using angr tool, the extracted semantic features are saved in text form to form a malicious code fusion semantic text; the extracted malicious code fusion semantic text is preprocessed, mainly including text normalization processing and word segmentation processing; the preprocessed malicious code fusion semantic text is used as a corpus, sentence vector training is performed based on a Doc2vec algorithm, a malicious code semantic representation model is trained, and a word vector dictionary and a text vector dictionary are obtained; each malicious code fusion semantic text will be converted into a unique corresponding vector, and the vector is the trained text vector. The text vector is used as the input data set of SOM, and a malicious code clustering model is trained. When performing malicious code clustering analysis, the to-be-tested sample is preprocessed to obtain a new malicious code fusion semantic text; the new malicious code fusion semantic text is input into the pre-trained malicious code semantic representation model, text vector prediction is performed, and the predicted text vector is used as the semantic vector of the sample. The semantic vector of the previous step is used as the input of the clustering model, and clustering analysis is performed, and finally the clustered malicious code is obtained.

[0151] The application also provides a malicious code clustering system, comprising:

[0152] The acquisition module is configured to acquire a malicious code fusion semantic text.

[0153] The feature extraction module is configured to perform feature extraction on the malicious code fusion semantic text to obtain a fusion semantic sequence.

[0154] The prediction module is configured to perform text vector prediction on the fusion semantic sequence by using a malicious code behavior semantic representation model to obtain a behavior semantic vector.

[0155] The clustering module is configured to perform clustering on the behavior semantic vector by using a malicious code clustering model to obtain clustered malicious code.

[0156] In actual application, the feature extraction module specifically comprises:

[0157] A program control flow graph generation unit is configured to perform dependency analysis and program block division on the malicious code fusion semantic text to generate a program control flow graph; and a traversal unit is configured to traverse the program control flow graph to determine a fusion semantic sequence based on basic block information and API function information in the program control flow graph.

[0158] In actual application, the prediction module specifically includes:

[0159] A preprocessing unit is configured to preprocess the fusion semantic sequence to obtain one-hot encoding; a prediction unit is configured to perform prediction based on the one-hot encoding using the distributed storage model and the distributed bag-of-words library model to obtain a distributed storage model sentence vector and a distributed bag-of-words library model sentence vector; and a splicing unit is configured to splice the distributed storage model sentence vector and the distributed bag-of-words library model sentence vector to obtain a behavior semantic vector.

[0160] In actual application, the training process of the malicious code clustering model includes:

[0161] The self-organizing mapping neural network is trained using a competitive learning strategy, with the behavior semantic vector of the sample set as input and the clustered malicious code of the sample set as output, to obtain a malicious code clustering model.

[0162] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts of each embodiment can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0163] The principles and implementation manners of the present application are described by using specific examples in the present application, and the above embodiment description is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for clustering malicious code, characterized in that, include: Obtain malicious code combined with semantic text; Feature extraction is performed on the fused semantic text of the malicious code to obtain a fused semantic sequence; the algorithm steps are as follows: Step 1: Load the binary malware program into the angr analysis system; Step 2: Perform dependency analysis and program segmentation to generate a control flow graph (ICFG); Step 3: Traverse the entire ICFG starting from the root node, and store the API function call relationship, basic block information, and the address of the API function call in the basic block for each API call path; Step 4: Based on the information obtained in Step 3, generate a fused semantic sequence containing API functions and assembly instruction sequences for each API call path; Step 5: Combine each fused semantic sequence and write it into the document; Based on the fused semantic sequence, a malicious code behavior semantic representation model is used to predict text vectors to obtain behavior semantic vectors; the malicious code behavior semantic representation model includes a distributed storage model and a distributed bag-of-words model. Based on the behavioral semantic vectors, a malicious code clustering model is used to cluster the malicious code to obtain the clustered malicious code; the malicious code clustering model is a trained self-organizing map neural network.

2. The malicious code clustering method according to claim 1, characterized in that, The step of extracting features from the fused semantic text of the malicious code to obtain the fused semantic sequence specifically includes: Dependency analysis and program segmentation are performed on the malicious code fused with semantic text to generate a program control flow graph; Traverse the program control flow graph and determine the fusion semantic sequence based on the basic block information and API function information in the program control flow graph.

3. The malicious code clustering method according to claim 1, characterized in that, The step of predicting text vectors using a malicious code behavior semantic representation model based on the fused semantic sequence to obtain behavior semantic vectors specifically includes: The fused semantic sequence is preprocessed to obtain one-hot encoding; Based on the one-hot encoding, prediction is performed using the distributed storage model and the distributed bag-of-words model respectively to obtain the sentence vector of the distributed storage model and the sentence vector of the distributed bag-of-words model. The sentence vectors of the distributed storage model and the sentence vectors of the distributed bag-of-words model are concatenated to obtain the behavioral semantic vector.

4. The malicious code clustering method according to claim 1, characterized in that, The training process of the malicious code clustering model includes: Using the behavioral semantic vectors of the sample set as input and the clustered malicious code of the sample set as output, the self-organizing map neural network is trained using a competitive learning strategy to obtain a malicious code clustering model.

5. A malicious code clustering system, characterized in that, include: The acquisition module is used to acquire malicious code fused semantic text; The feature extraction module is used to extract features from the malicious code's fused semantic text to obtain a fused semantic sequence; the algorithm steps are as follows: Step 1: Load the binary malware program into the angr analysis system; Step 2: Perform dependency analysis and program segmentation to generate a control flow graph (ICFG); Step 3: Traverse the entire ICFG starting from the root node, and store the API function call relationship, basic block information, and the address of the API function call in the basic block for each API call path; Step 4: Based on the information obtained in Step 3, generate a fused semantic sequence containing API functions and assembly instruction sequences for each API call path; Step 5: Combine each fused semantic sequence and write it into the document; The prediction module is used to predict text vectors based on the fused semantic sequence using a malicious code behavior semantic representation model to obtain behavior semantic vectors; the malicious code behavior semantic representation model includes a distributed storage model and a distributed bag-of-words model. The clustering module is used to cluster malicious code based on the behavioral semantic vector using a malicious code clustering model to obtain clustered malicious code; the malicious code clustering model is a trained self-organizing map neural network.

6. The malicious code clustering system according to claim 5, characterized in that, The feature extraction module specifically includes: The program control flow graph generation unit is used to perform dependency analysis and program segmentation on the malicious code fused semantic text to generate a program control flow graph. The traversal unit is used to traverse the program control flow graph and determine the fusion semantic sequence based on the basic block information and API function information in the program control flow graph.

7. The malicious code clustering system according to claim 5, characterized in that, The prediction module specifically includes: The preprocessing unit is used to preprocess the fused semantic sequence to obtain one-hot encoding; The prediction unit is used to perform predictions based on the one-hot encoding using the distributed storage model and the distributed bag-of-words model respectively, to obtain the sentence vector of the distributed storage model and the sentence vector of the distributed bag-of-words model. The concatenation unit is used to concatenate the sentence vectors of the distributed storage model and the sentence vectors of the distributed bag-of-words model to obtain behavioral semantic vectors.

8. The malicious code clustering system according to claim 5, characterized in that, The training process of the malicious code clustering model includes: Using the behavioral semantic vectors of the sample set as input and the clustered malicious code of the sample set as output, the self-organizing map neural network is trained using a competitive learning strategy to obtain a malicious code clustering model.

Images