An attack pattern extraction method based on phrase similarity
Through a phrase similarity-based method and the BiLSTM twin neural network model, the problem of attack pattern extraction in unstructured threat intelligence text was solved, and efficient and accurate attack pattern recognition and mapping to the ATT&CK matrix were achieved.
Patent Information
- Application Number
- CN202211398631.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-09
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2042-11-09
AI Technical Summary
Existing technologies have difficulty in accurately extracting attack patterns from unstructured threat intelligence text, especially because the complexity and diversity of attack patterns make it difficult for machines to understand and identify them.
A phrase similarity-based method is adopted, through unsupervised learning and the BiLSTM twin neural network model, using Word2Vec for text vectorization representation, combined with the ATT&CK matrix to extract attack patterns, including quality phrase extraction, data labeling and phrase pair similarity model training.
The accuracy and efficiency of attack pattern extraction are improved, and attack patterns can be better extracted from unstructured text and mapped to the ATT&CK matrix, taking into account the complexity and diversity of attack patterns.
Smart Images
Figure CN115759081B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of information security technology, and in particular relates to an attack pattern extraction method based on phrase similarity. Background Art
[0002] Cyber threat intelligence (CTI) refers to the information and knowledge used for defense and reconnaissance in cyber warfare. With the emergence of advanced cyberattacks such as APTs, polymorphic threats, zero-day threats, and complex threats, attack methods have become more complex, covert, and persistent, while also becoming more organized, targeted, and destructive. The attack knowledge contained in cyber threat intelligence is crucial for effectively identifying and rapidly responding to cyber threats.
[0003] An attack pattern is an abstract representation of the methods used by an attacker against their target. It summarizes the techniques, tools, or programs employed into specific patterns, describing high-level threat indicators. Cunning attackers will forge traces of their victims at various stages of a cyberattack, such as through IP spoofing, attempting to attribute the attack to others, thereby concealing themselves and making detection more difficult. Low-level threat indicators such as IP addresses, ports, domain names, and hash values, which are relatively easy to obtain, are easily modified and disguised. However, attack patterns, which represent the attacker's methods, are the attacker's fundamental characteristics and long-term behavior, and they better reflect the attacker's characteristics and preferences.
[0004] Identifying attack patterns is extremely labor-intensive and resource-intensive, often requiring investigation and evidence collection on the victim's network or host. Nowadays, various threat intelligence reports and cybersecurity incident reports describe attack patterns in unstructured text formats, providing security analysts with the opportunity to intelligently extract attack patterns from natural language text.
[0005] Named entity recognition (NER) is a fundamental task in natural language processing, similar to but not identical to attack pattern extraction. Named entity recognition is used to extract security entities such as malware and tools that are identified by uniform names and clearly appear in the text. In threat intelligence text, attack patterns often appear as phrases, such as "spearphishing emails," "web shells," "registry modifications," and "key logging." The semantics contained in these phrases constitute attack patterns, but there's no definitive, unified way to describe them. Attack patterns often convey meaning only in a single paragraph, often in a way that's understandable to humans. For example, a security analyst might read the sentence "The first method is to lure the target into opening an email attachment" and conclude that the attacker used a "phishing email" attack pattern to lure the target, even though the sentence doesn't explicitly describe "phishing email." Therefore, NER is unsuitable for attack pattern extraction, significantly increasing the difficulty of extracting attack patterns. Summary of the Invention
[0006] (1) Technical issues to be resolved
[0007] The technical problem to be solved by the present invention is: how to propose an attack pattern extraction method to address the problem that the attack pattern description summarized in unstructured threat intelligence text cannot be directly understood by machines and the attack pattern itself is complex and diverse.
[0008] (2) Technical solution
[0009] In order to solve the above technical problems, the present invention provides an attack pattern extraction method based on phrase similarity, comprising the following steps:
[0010] Step 1: Quality phrase extraction
[0011] Extracting a list of phrases ranked by quality, i.e., a quality phrase list, from the unstructured security text as a list of candidate phrases for the attack pattern;
[0012] Step 2: Data labeling for unsupervised learning based on step 1
[0013] Use the unsupervised Levenshtein distance algorithm to preliminarily measure the glyph similarity between candidate phrases of the attack pattern and technical objects in the ATT&CK matrix. Then manually classify and annotate the candidate phrases of the attack pattern to obtain training data.
[0014] Step 3: Train the phrase pair similarity model based on step 2
[0015] The phrase pair similarity model includes a text feature representation model. During the training process of the text feature representation model, the Skip-gram model in the distributed word vector representation model Word2Vec is used to perform text vectorization on the training data, converting the processing of text into vector operations in the vector space. At the same time, by capturing the contextual features of the current words, semantically similar words are mapped to similar positions in the vector space, thereby using the similarity of phrases in the vector space to indicate the similarity of phrases in the text.
[0016] Preferably, in step 1, the AutoPhrase algorithm is used to automatically extract quality phrases from the security text, and positive samples are constructed by referencing quality phrases in the knowledge base for remote supervision training, and part-of-speech information is introduced during extraction.
[0017] Preferably, the knowledge base is a Wikipedia knowledge base.
[0018] Preferably, a phrase is a series of words that appear continuously in a text and is a complete semantic unit. The specific extraction process in step 1 is as follows:
[0019] 1) First, the initial security dataset in the security text is segmented into sentences;
[0020] 2) Use n-gram to cut sentences to construct a candidate quality phrase set and set the phrase length threshold n;
[0021] 3) Match with the quality phrases in the knowledge base. If it is a hit, it is put into the positive example pool; if it is not a hit, it is put into the duplicate example pool;
[0022] 4) Train an ensemble classifier based on random forest, and use this ensemble classifier to classify the word sequence w1w2...w n Set a quality estimate Q, phrase(w1w2...w n ) represents a complete semantic unit consisting of a sequence of words;
[0023] Q(w1w2…w n )=p(phrase(w1w2…w n )|w1w2…w n )∈[0,1]
[0024] Where p represents the conditional probability;
[0025] 5) Introducing part-of-speech information to provide shallow semantic information;
[0026] a) First, process the secure text into a word sequence of length n with part-of-speech tags Ω = Ω1Ω2...Ω n ,Ω is composed of word w i and part-of-speech tag t i <w i , t i > composition;
[0027] b) Determine the boundary index B and divide the word sequence Ω into m segments; B = {b1, b2, ..., b m+1 )(1=b1<b2<...<b m+1 =n+1)
[0028] c) determining the phrase part-of-speech quality score T;
[0029]
[0030] t is a part-of-speech tag, and its subscript represents the index. (b i ,b i+1 +1) indicates that (b i ,b i+1 +1) position;
[0031] 6) Calculate the joint probability of the word sequence Ω with part-of-speech tags and the boundary index B:
[0032]
[0033] w (bi,bi+1) For the sequence (b i ,b i+1 +1) word sequence at position;
[0034] 7) Use maximum likelihood estimation for calculation and iteratively optimize parameters using the idea of dynamic programming;
[0035]
[0036] 8) Finally, after maximizing the formula in step 7, the final sequence boundary index B is obtained. The word sequence is divided according to this index to obtain a list of quality phrases.
[0037] Preferably, in step 2, the Levenshtein distance between the two phrase sequences is lev a，b Defined as
[0038]
[0039] Among them, lev a，b(i, j) represents the Levenshtein distance between the first i characters of phrase sequence a and the first j characters of phrase sequence b. The closer the Levenshtein distance of a phrase pair is to 1, the closer the glyphs of the two phrases are.
[0040] Preferably, in step 3, Word2vec is used as the text feature representation model to vectorize the phrases in the training data, and then a BiLSTM-based twin neural network structure is used to train and evaluate the phrase pair similarity model, and the best phrase pair similarity model is saved, that is, the phrase pair similarity model that minimizes the loss value.
[0041] Preferably, in step 3, in order to predict the similarity of phrase pairs, a fully connected layer is connected after the BiLSTM layer to perform dimensionality conversion, and a Sigmoid function is selected as the activation function to compress the prediction result to the [0, 1] interval:
[0042]
[0043] Preferably, the method further includes step 4, outputting and evaluating the results: vectorizing the phrase pairs using the same Word2Vec model, and then inputting the vectors into the best phrase pair similarity model to complete the matching of quality phrases with technical objects in the ATT&CK matrix. After outputting the final matching results, the accuracy of the matching results is evaluated.
[0044] The present invention also provides a system for implementing the method.
[0045] The present invention also provides an application of the method in the field of information security technology.
[0046] (3) Beneficial effects
[0047] The present invention proposes an attack pattern extraction method based on phrase similarity, which can extract specific attack patterns from unstructured threat intelligence texts, that is, the abstraction of the attack methods used by attackers against targets, and map them to the ATT&CK matrix. Among them, the present invention starts from the smallest semantic expression unit of the attack pattern - the phrase, and converts the attack pattern extraction task into a phrase pair matching task, which can more accurately extract the attack pattern in the unstructured threat intelligence text, and takes into account the complexity and diversity of the attack pattern entities. The present invention uses unsupervised learning to assist in efficient data labeling, constructs a training data set, and trains a text representation model based on Word2Vec and a twin network model based on BiLSTM for phrase pair matching, which can improve efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 Schematic diagram of the method of the present invention;
[0049] Figure 2 This is the structure diagram of the phrase pair similarity model. DETAILED DESCRIPTION
[0050] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.
[0051] To more accurately extract attack patterns from unstructured threat intelligence text, and considering the complexity and diversity of attack pattern entities, this paper transforms the attack pattern extraction task into a phrase-pair matching task, starting from the smallest semantic expression unit of an attack pattern—a phrase. To improve efficiency, this paper utilizes unsupervised learning to assist in efficient data annotation, constructs a training dataset, and trains a Word2Vec-based text representation model and a BiLSTM-based twin network model for phrase-pair matching.
[0052] refer to Figure 1 、 Figure 2 The phrase-similarity-based attack pattern extraction method proposed in this paper consists of four main steps: high-quality phrase extraction, efficient data annotation with the aid of unsupervised learning, training a phrase-pair similarity model, and outputting and evaluating the results. Each step of the method is described in detail below.
[0053] 1. Quality Phrase Extraction
[0054] Extract a list of high-quality phrases from unstructured threat intelligence text as candidate phrases for attack patterns. To extract attack patterns from unstructured security text descriptions, the attack pattern extraction task is transformed into a phrase pair similarity task, which involves determining the degree of similarity between the phrases contained in the text and the attack pattern. Therefore, the accuracy of attack pattern extraction depends heavily on the quality of the phrases extracted from the threat intelligence text.
[0055] This paper uses the AutoPhrase algorithm to automatically extract high-quality phrases from secure text. It constructs positive samples by referencing high-quality phrases from an existing knowledge base for remote supervised training, and introduces part-of-speech (POS) information to improve the accuracy of phrase extraction. The input of the quality phrase extraction model is the secure text and the Wikipedia general knowledge base, and the output is a list of phrases extracted from the secure text, sorted by quality. A phrase is a series of words that appear continuously in a text and is a complete semantic unit. The specific extraction process is as follows:
[0056] 1) First, the initial security dataset is segmented;
[0057] 2) Use n-gram segmentation to construct a candidate quality phrase set, where the phrase length threshold n = 6;
[0058] 3) Match the phrases with the quality phrases in the Wikipedia general knowledge base. If they match, they are put into the positive example pool; if they do not match, they are put into the duplicate example pool.
[0059] 4) Train an ensemble classifier based on random forest, and use this classifier to classify the word sequence w1w2...w n Set a quality estimate Q, phrase(w1w2...w n ) represents a complete semantic unit consisting of a sequence of words;
[0060] Q(w1w2…w n )=p(phrase(w1w2…w n )|w1w2…w n )∈[0,1]
[0061] Where p represents the conditional probability;
[0062] 5) Introducing part-of-speech information to provide shallow semantic information;
[0063] a) First, process the secure text into a word sequence of length n with part-of-speech tags Ω = Ω1Ω2...Ω n ,Ω is composed of word w i and part-of-speech tag t i <w i , t i > composition;
[0064] b) determining a boundary index B and dividing the word sequence Ω into m segments;
[0065] B={b1,b2,...,b m+1}(1=b1<b2<...<b m+1 =n+1)
[0066] c) determining the phrase part-of-speech quality score T, i.e., the conditional probability of the corresponding word sequence as a complete semantic unit;
[0067]
[0068] t is a part-of-speech tag, and its subscript represents the index. (b i ,b i+1 +1) indicates that (b i ,b i+1 +1) position.
[0069] 6) Calculate the joint probability of the word sequence Ω with part-of-speech tags and the boundary index B;
[0070]
[0071] w (bi,bi+1) For the sequence (b i ,b i+1 +1) The word sequence at position .
[0072] 7) Use maximum likelihood estimation for calculation and iteratively optimize parameters using the idea of dynamic programming;
[0073]
[0074] 8) Finally, after maximizing the above formula, we get the final sequence boundary index B. By dividing the word sequence according to this index, we can obtain a list of quality phrases.
[0075] 2. Efficient data labeling with unsupervised learning assistance based on step 1
[0076] This paper uses an unsupervised learning-assisted method for data annotation. The basic idea is to use the unsupervised Levenshtein distance algorithm to initially measure the glyph similarity between candidate attack pattern phrases and the technique objects in the ATT&CK matrix. The candidate attack pattern phrases are then manually classified and annotated. The Levenshtein algorithm, also known as the edit distance, simply measures the edit distance between two strings. This is the minimum number of single-character edits required to transform one sequence into another, such as deletions, insertions, and substitutions.
[0077] The Levenshtein distance between two phrase sequences is lev a，b Defined as
[0078]
[0079] Among them, lev a，b (i, j) represents the Levenshtein distance between the first i characters of phrase sequence a and the first j characters of phrase sequence b. If the Levenshtein distance of a phrase pair is closer to 1, it means that the glyphs of the two phrases are closer.
[0080] The Levenshtein algorithm can simply compare the glyph differences between phrase pairs, but it cannot indicate the semantic similarity between them. To semantically match candidate attack pattern phrases with technical objects in the ATT&CK matrix and improve attack pattern extraction accuracy, the Levenshtein distance between the candidate attack pattern phrases extracted in step 1 and the technical objects in the ATT&CK knowledge base is calculated. High-quality phrases are manually classified and annotated to efficiently generate training data.
[0081] 3. Train the phrase pair similarity model based on step 2
[0082] The candidate phrases of the attack pattern extracted from the security text are in text format. In order to make them understandable to computers, they need to be vectorized. The phrase pair similarity model includes a text feature representation model. In the actual training process of the text feature representation model, the Skip-gram model in the current mainstream distributed word vector representation model Word2Vec is used to perform text vectorization on the phrases in the training data, converting the processing of the text into vector operations in the vector space. At the same time, by capturing the contextual features of the current words, semantically similar words are mapped to similar positions in the vector space, thereby using the similarity of phrases in the vector space to indicate the similarity of phrases in the text. The Skip-gram model is a three-layer neural network that can represent the similarity of phrases in the text based on the given word w. j , predict the context w1,...,w of the word j-1 , w j+1 ,...,w n The Skip-gram model updates parameters through backpropagation. When the model is iterated and trained a certain number of times, the final word vector output is the parameter matrix from the input layer to the hidden layer, where the transpose of each row in the matrix is the word vector of the corresponding word.
[0083] After Word2Vec training, the quality phrases extracted from the security text (i.e., candidate phrases of the attack pattern) and the technical phrases in the ATT&CK matrix are converted into low-dimensional, dense, and continuous word vectors in the vector space to facilitate subsequent similarity comparison. In the phrase pair similarity model, the present invention selects the classic Siamese neural network architecture, which consists of two neural networks with the same structure and shared weights, so that two samples can be received as input. Since the weights between the two sub-network structures in the twin network are shared, the two inputs can be mapped to the same space for similarity comparison.
[0084] Therefore, this step is to use Word2vec as the text feature representation model to vectorize the phrases in the training data, and then use the BiLSTM-based twin neural network structure to train and evaluate the phrase pair similarity model, and save the best phrase pair similarity model (that is, the phrase pair similarity model that minimizes the loss value).
[0085] In the selection of subnetworks within the twin neural network structure, the present invention uses a bidirectional long short-term memory (BiLSTM) network model. As a classic sequence modeling model, LSTM controls information flow through a carefully designed gate structure and selectively retains information. It can combine short-term and long-term memory, effectively resolving long-range dependency issues. This gives it significant advantages in text feature encoding and has been widely used in the field of natural language processing.
[0086] When LSTM extracts features from sequence data, it only considers the state of the previous moment and the current input. Compared with LSTM, BiLSTM can encode from two directions, converting the output of the forward LSTM into and the output of the backward LSTM Splice and get the final output at the current moment After summarizing all moments, we get the entire output of the BiLSTM encoding layer H = {h1, h2, ..., h T}, as the feature encoding of the current text sequence. The output of the BiLSTM model depends not only on the state of the previous moment, but also on the state of the next moment. Therefore, it can capture the bidirectional semantic dependency in the sequence and make more full use of the semantic information of the phrase sequence.
[0087] In order to predict the similarity of phrase pairs, a fully connected layer is added after the BiLSTM layer for dimensionality conversion, and the Sigmoid function is selected as the activation function to compress the prediction results to the [0, 1] interval.
[0088]
[0089] 4. Output and evaluate the results
[0090] The phrase pairs are vectorized using the same Word2Vec model. The vectors are then fed into the optimal phrase pair similarity model to match the quality phrases with the technical objects in the ATT&CK matrix. After the final matching results are output, the accuracy and f1 score of the matching results are evaluated.
[0091] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for extracting attack patterns based on phrase similarity, characterized in that: The following steps are involved: Step 1: Quality phrase extraction Extracting a list of phrases ranked by quality, i.e., a quality phrase list, from the unstructured security text as a list of candidate phrases for the attack pattern; Step 2: Data labeling for unsupervised learning based on step 1 Use the unsupervised Levenshtein distance algorithm to preliminarily measure the glyph similarity between candidate phrases of the attack pattern and technical objects in the ATT&CK matrix. Then manually classify and annotate the candidate phrases of the attack pattern to obtain training data. Step 3: Train the phrase pair similarity model based on step 2 The phrase pair similarity model includes a text feature representation model. During the training process of the text feature representation model, the Skip-gram model in the distributed word vector representation model Word2Vec is used to perform text vectorization on the training data, converting the processing of text into vector operations in the vector space. At the same time, by capturing the contextual features of the current words, semantically similar words are mapped to similar positions in the vector space, thereby using the similarity of phrases in the vector space to indicate the similarity of phrases in the text.
2. The method according to claim 1, wherein In step 1, the AutoPhrase algorithm is used to automatically extract quality phrases from security texts. Positive samples are constructed by referencing quality phrases in the knowledge base for remote supervision training, and part-of-speech information is introduced during extraction.
3. The method according to claim 2, wherein The knowledge base is the Wikipedia knowledge base.
4. The method according to claim 2, wherein A phrase is a series of words that appear continuously in a text and is a complete semantic unit. The specific extraction process in step 1 is as follows: 1) First, the initial security dataset in the security text is segmented into sentences; 2) Use n-gram to cut sentences to construct a candidate quality phrase set and set the phrase length threshold n; 3) Match with the quality phrases in the knowledge base, and put them into the positive example pool if they match. Misses are put into the duplicate pool; 4) Train an ensemble classifier based on random forest, and use this ensemble classifier to classify the word sequence w1w2...w n Set a quality estimate Q, phrase(w1w2...w n ) represents a complete semantic unit consisting of a word sequence; Q(w1w2…w n )=p(phrase(w1w2…w n )|w1w2...w n )∈[0,1] Where p represents the conditional probability; 5) Introducing part-of-speech information to provide shallow semantic information; a) First, process the secure text into a word sequence of length n with part-of-speech tags Ω = Ω1Ω2...Ω n ,Ω is composed of word w i and part-of-speech tag t i Composition, expressed as <w i , t i > b) determining a boundary index B and dividing the word sequence Ω into m segments; B={b1,b2,...,b m+1 }(1=b1<b2<…<b m+1 =n+1) c) determining the phrase part-of-speech quality score T; t is a part-of-speech tag, and its subscript represents the index. (b i ,b i+1 +1) indicates that (b i ,b i+1 +1) position; 6) Calculate the joint probability of the word sequence Ω with part-of-speech tags and the boundary index B: w (bi,bi+1) For the sequence (b i ,b i+1 +1) word sequence at position; 7) Use maximum likelihood estimation for calculation and iteratively optimize parameters using the idea of dynamic programming; 8) Finally, after maximizing the formula in step 7, the final sequence boundary index B is obtained. The word sequence is divided according to this index to obtain a list of quality phrases.
5. The method according to claim 4, wherein In step 2, the Levenshtein distance between the two phrase sequences is lev a，b Defined as Among them, lev a，b (i, j) represents the Levenshtein distance between the first i characters of phrase sequence a and the first j characters of phrase sequence b. The closer the Levenshtein distance of a phrase pair is to 1, the closer the glyphs of the two phrases are.
6. The method according to claim 5, wherein In step 3, Word2vec is used as the text feature representation model to vectorize the phrases in the training data. Then, a BiLSTM-based twin neural network structure is used to train and evaluate the phrase pair similarity model, and the best phrase pair similarity model is saved, that is, the phrase pair similarity model that minimizes the loss value.
7. The method according to claim 6, wherein In step 3, in order to predict the similarity of phrase pairs, a fully connected layer is added after the BiLSTM layer to perform dimensionality conversion, and the Sigmoid function is selected as the activation function to compress the prediction results to the [0, 1] interval:
8. The method according to claim 6, wherein It also includes step 4, output and evaluate the results: the phrase pairs are vectorized using the same Word2Vec model, and the vectors are then input into the best phrase pair similarity model to complete the matching of quality phrases with technical objects in the ATT&CK matrix. After outputting the final matching results, the accuracy of the matching results is evaluated.
9. Application of the method according to any one of claims 1 to 8 in the field of information security technology.