LDA-based advanced persistent threat organization discrimination method
By combining the LDA model and TF-IDF parameters, the problem of excessive dictionary dimension in APT attack analysis is solved, achieving efficient and accurate APT organization identification and clustering.
Patent Information
- Application Number
- CN202310054321.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-03
- Publication Date
- 2026-06-12
- Estimated Expiration
- 2043-02-03
AI Technical Summary
Existing technologies lack corpus support in APT attack analysis. Excessive dictionary dimensionality leads to large storage space consumption and affects clustering efficiency and accuracy. Keywords cannot be explicitly assigned practical meaning, and the clustering results are not of practical help.
Using an LDA model-based approach, an APT-specific dictionary is constructed offline, stop words are removed, and initial training is performed to generate a new dictionary. This dictionary is then further trained using TF-IDF parameters. In the online phase, cosine similarity is calculated to determine the organization to which the attack belongs.
It significantly reduces dictionary dimensionality, improves clustering efficiency and accuracy, achieves high-precision APT organization identification, and the clustering results have practical significance.
Smart Images

Figure CN118449705B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a technology in the field of information security, specifically a method for identifying advanced persistent threat (APT) organizations based on the Potential Dirichlet Distribution Model (LDA). Background Technology
[0002] The existing cybersecurity field, such as APT attacks, lacks relevant corpus support. Many professional terms are not found in dictionaries, requiring the creation of custom dictionaries, and existing corpora cannot be used to assist training. In typical text analysis tasks, topics belong to different domains and have obvious inter-class distinctions, while current application requirements involve further subdivision within reports of the same domain, which also presents challenges in keyword selection.
[0003] Because security reports contain shared Indicators of Compromise (IOCs), which include a large amount of hash values and domain names, after word segmentation and other processing, directly using all distinct words to generate the dictionary used by the Bag-of-Words (BOW) model would result in a dictionary exceeding 100,000 dimensions. Even after removing high-frequency and low-frequency keywords, the dictionary size still exceeds 20,000 dimensions, leading to a very sparse document-word probability distribution matrix. This not only consumes a large amount of storage space but also significantly impacts the efficiency of clustering learning. Furthermore, the statistically obtained word frequencies are quite low, affecting the accuracy of subsequent clustering calculations. Ultimately, the keywords selected for each topic appear with very low frequencies, failing to assign any clear practical meaning to the topics, and the clustering results are of no help in classifying APT reports. Summary of the Invention
[0004] This invention addresses the problems of excessively high dictionary dimensionality and the inability to correlate keywords obtained from model training with attacking organizations in existing technologies. It proposes an advanced persistent threat (APT) organization identification method based on LDA. This method utilizes natural language processing technology to analyze and cluster features such as attack vectors disclosed in reports published by security companies to determine the APT organization to which the attack belongs. It does not rely on a supervised dictionary and requires no manual intervention in the dictionary construction process. It is suitable for application scenarios where the words in APT organization analysis reports are highly specialized, the vocabulary generated after removing stop words is high-dimensional, and the topic clustering and keyword selection are difficult.
[0005] This invention is achieved through the following technical solution:
[0006] This invention relates to an advanced persistent threat (APT) organization identification method based on LDA. In the offline phase, a database is constructed based on publicly known attack information and reports of the organizations to which they belong. For each APT organization, an initial LDA model with one topic is trained, and the dictionary is updated. Then, a secondary training is performed on an LDA model with the number of topics equal to the number of APT organizations. Term frequencies and inverse document frequencies are statistically obtained, and the LDA model is used to cluster representative attack methods and attack targets of each APT organization, identifying key features. In the online phase, the term frequency-inverse document frequency (TF-IDF) parameter is calculated in real-time based on the statistically obtained term frequencies and inverse document frequencies. The cosine similarity between the TF-IDF parameter and the cluster centers obtained from the LDA model is calculated to identify the APT organization to which the attack belongs, and the similarity between the TF-IDF parameter and the typical attack vectors of the organization is output.
[0007] The offline phase specifically includes:
[0008] Step A1: Crawl analysis reports released by major security vendors containing known and publicly available attack information and information about the organizations to which they belong, and store the full text of the reports in the corresponding databases according to the different APT organizations to which they belong.
[0009] Step A2: Perform text preprocessing on the obtained report, specifically including: removing all punctuation marks, removing stop words, converting all words to lowercase, and restoring the part-of-speech of all words;
[0010] Step A3: Generate a dictionary based on the cleaned text;
[0011] Step A4: Using the bag-of-words model, the cleaned text from step A2 is converted into a vector representation based on the dictionary generated in step A3;
[0012] Step A5: For each APT organization involved, use the analysis report of that organization to train an LDA model with 1 topic, and output the 25 feature words with the highest weights for each APT organization.
[0013] Step A6: Summarize the feature words output by each APT organization, remove the feature words that are output by multiple APT organizations, construct a new dictionary with reduced dimensionality and strong feature characteristics, and store the new dictionary locally;
[0014] Step A7: Based on the generated new dictionary, use the bag-of-words model to convert the cleaned text from step A2 into a vector representation;
[0015] Step A8: Based on the generated vector representation, count the term frequency and inverse document frequency, calculate the term frequency-inverse document frequency (TF-IDF) parameter, and store the counted term frequency and inverse document frequency locally;
[0016] Step A9: Perform secondary training on the LDA model with the number of topics equal to the number of APT organizations based on the TF-IDF parameters, and store the trained LDA model locally;
[0017] Step A10: Using the LDA model trained in step A9 and the TF-IDF parameters generated in step A8, generate the clustering results of each report, evaluate the clustering results using confusion matrix, and output characteristic keywords such as representative attack methods and attack targets of each APT organization.
[0018] The online phase includes:
[0019] Step B1: Preprocess the report text released by the security company, specifically including: removing all punctuation marks, converting all words to lowercase, and restoring the part-of-speech of all words;
[0020] Step B2: Based on the dictionary generated in step A6, use the bag-of-words model to convert the preprocessed report text into a vector representation;
[0021] Step B3: Calculate the TF-IDF parameters based on the generated vector representation and the term frequency and inverse document frequency obtained in step A8;
[0022] Step B4: Based on the calculated TF-IDF parameters and the LDA model generated in step A9, determine the APT organization to which the attack in the report belongs by calculating the cosine similarity between the TF-IDF parameters and the cluster centers obtained in the LDA model, and output its similarity with the typical attack vectors in the organization.
[0023] Technical effect
[0024] This invention performs initial training of the LDA model for each organization. Without manual intervention, it automatically selects characteristic keywords for each organization and removes keywords that appear repeatedly, resulting in a new dictionary. The LDA model is then further trained based on this new dictionary. Compared to existing technologies, this invention significantly reduces the dimensionality of dictionary construction while improving its feature set, thereby achieving high-precision clustering of texts with low inter-class discriminative power. Attached Figure Description
[0025] Figure 1 This is a flowchart of the offline phase of the present invention;
[0026] Figure 2 This is a flowchart of the online phase of the present invention. Detailed Implementation
[0027] like Figure 1 As shown in the figure, this embodiment relates to an advanced persistent threat organization identification method based on LDA, including an offline phase and an online phase.
[0028] The offline phase includes:
[0029] Step 1) Crawl analysis reports released by major security vendors with clear APT organization classifications, save the full text of each report to a separate file, and store them in the corresponding database according to the different APT organizations they belong to. In this embodiment, analysis reports on 7 recently active APT organizations are collected. In specific applications, the number of APT organizations is not limited.
[0030] Step 2) Use Python to perform text preprocessing on the obtained report, specifically including: removing all punctuation marks contained in string.punctuation, removing all English stop words contained in nltk.corpus.stopwords, converting all words to lowercase, and using nltk.stem.wordnet.WordNetLemmatizer to perform part-of-speech tagging on all words;
[0031] Step 3) Use gensim.corpora.Dictionary to generate a dictionary from the cleaned text;
[0032] Step 4) Use the doc2bow function of the Dictionary class to convert the cleaned text from step 2) into a vector representation based on the generated dictionary;
[0033] Step 5) For the 7 APT organizations involved in this embodiment, use the vectors generated from their respective analysis reports to train an LDA model with 1 topic using gensim.models.ldamodel.LdaModel. The dictionary used is the dictionary generated in step 3), with chunksize set to 5, passes set to 200, and iterations set to 500. After the model training is completed, use the print_topics function to output the 25 feature words with the highest weights for each APT organization.
[0034] Step 6) Summarize the feature words output by the 7 APT organizations involved in this embodiment, remove the feature words that are output by three or more APT organizations and are not representative, construct a new dictionary with significantly reduced dimensionality and more representative feature words using gensim.corpora.Dictionary, and store the new dictionary locally.
[0035] Step 7) Based on the generated new dictionary, use the doc2bow function of the Dictionary class to convert the cleaned text from step 2) into a vector representation;
[0036] Step 8) Based on the generated vector representation, use gensim.models.TfidfModel to count the term frequency and inverse document frequency, calculate the TF-IDF parameters, and store the term frequency and inverse document frequency locally;
[0037] Step 9) Based on the calculated TF-IDF parameters, train an LDA model with the number of topics equal to the number of APT organizations (7 in this example) using gensim.models.ldamodel.LdaModel. Use the dictionary generated in step 6), set chunksize to 5, passes to 200, iterations to 500, and store the model locally.
[0038] Step 10) Based on the trained LDA model and the TF-IDF parameters generated in Step 8), use the get_document_topics function to generate the clustering results of each report. The clustering results can be evaluated using the confusion matrix, etc. In addition, the print_topics function can output the representative attack methods and attack targets of each APT organization.
[0039] like Figure 2 The online phase includes:
[0040] Step A) Use Python to perform text preprocessing on the reports from security companies that have been crawled. Specifically, this includes: removing all punctuation marks from string.punctuation, converting all words to lowercase, and using nltk.stem.wordnet.WordNetLemmatizer to perform part-of-speech tagging on all words.
[0041] Step B) Based on the dictionary generated in step 6) during the previous training process, use the doc2bow function of the Dictionary class to convert the report text into a vector representation;
[0042] Step C) Calculate the TF-IDF parameters using gensim.models.TfidfModel based on the generated vector representation and the word frequency and inverse document frequency counted in step 8) during the previous training process;
[0043] Step D) Based on the calculated TF-IDF parameters and the LDA model generated in step 9) during the previous training process, use the get_document_topics function to generate the clustering results of the report, determine the APT organization to which the attack in the report belongs, and output the similarity between the attack disclosed in the report and the typical attack vector in the APT organization.
[0044] Through specific practical experiments, in an Ubuntu system with a Python 3.7.11 programming environment, this embodiment collected a total of 61 reports from 7 APT organizations for experiments. From each organization's reports, 3 reports were randomly selected, totaling 21 reports, for the initial training of the LDA model and feature word selection. Then, all 61 reports were used for secondary training of the LDA model, and the clustering results were evaluated accordingly. The sample size distribution of each organization in the dataset and the specific values of each evaluation metric are shown in the table below:
[0045]
[0046]
[0047] The specific clustering confusion matrix is shown in the table below:
[0048] Organization 1 Organization 2 Organization 3 Organization 4 Organization 5 Organization 6 Organization 7 Organization 1 3 0 0 0 0 1 1 Organization 2 0 5 1 0 0 1 0 Organization 3 0 0 4 0 0 0 1 Organization 4 0 0 0 9 1 1 0 Organization 5 0 0 0 1 5 1 0 Organization 6 0 0 0 1 0 16 0 Organization 7 0 1 0 0 0 0 8
[0049] Existing technologies often find that security reports from different APT organizations actually belong to the same domain, sharing multiple common keywords. Using traditional clustering models with TF-IDF as a parameter is not only time-consuming but also frequently results in large clusters with over 30 samples, rendering the clustering results meaningless and worthless. In contrast, this invention preprocesses the text during the initial clustering process by removing stop words and restoring part-of-speech tags. Then, using traditional word frequency (TF-IDF) as a parameter, it clusters reports from each APT organization separately. After obtaining the high-frequency keywords from each organization's reports, keywords appearing frequently across multiple organizations' reports are removed, resulting in a significantly reduced vocabulary. Finally, using this new vocabulary and TF-IDF as a parameter, the reports from each organization are re-clustered. As shown above, the final results significantly reduce the time consumption and improve the clustering effect, achieving an average F1-score of 0.80 across the seven classes, which is quite ideal for a multi-class classification task.
[0050] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.
Claims
1. A method for identifying advanced persistent threat organizations based on LDA, characterized in that, In the offline phase, a database is constructed based on known and publicly available attack information and reports of the organizations to which they belong. For each APT organization involved, an LDA model with 1 topic is initially trained and the dictionary is updated. Then, an LDA model with the number of topics equal to the number of APT organizations is trained a second time. The word frequency and inverse document frequency are obtained by statistics, and the representative attack methods and attack target feature keywords of each APT organization are obtained by clustering through the LDA model. During the online phase, the term frequency-inverse document frequency parameter is calculated in real time based on the statistically obtained term frequency and inverse document frequency of the text under test. The APT organization to which the attack belongs is obtained by calculating the cosine similarity between the parameter and the cluster centers obtained in the LDA model, and the similarity between the parameter and the typical attack vectors in the organization is output. The offline phase specifically includes: Step A1: Crawl analysis reports released by major security vendors containing known and publicly available attack information and information about the organizations to which they belong, and store the full text of the reports in the corresponding databases according to the different APT organizations to which they belong. Step A2: Perform text preprocessing on the obtained report, specifically including: removing all punctuation marks, removing stop words, converting all words to lowercase, and restoring the part-of-speech of all words; Step A3: Generate a dictionary based on the cleaned text; Step A4: Using the bag-of-words model, the cleaned text from step A2 is converted into a vector representation based on the dictionary generated in step A3; Step A5: For each APT organization involved, use the analysis report of that organization to train an LDA model with 1 topic, and output the 25 feature words with the highest weights for each APT organization. Step A6: Summarize the feature words output by each APT organization, remove the feature words that are output by multiple APT organizations, construct a new dictionary with reduced dimensionality and strong feature characteristics, and store the new dictionary locally; Step A7: Based on the generated new dictionary, use the bag-of-words model to convert the cleaned text from step A2 into a vector representation; Step A8: Based on the generated vector representation, count the term frequency and inverse document frequency, calculate the term frequency-inverse document frequency (TF-IDF) parameter, and store the counted term frequency and inverse document frequency locally; Step A9: Perform secondary training on the LDA model with the number of topics equal to the number of APT organizations based on the TF-IDF parameters, and store the trained LDA model locally; Step A10: Using the LDA model trained in step A9 and the TF-IDF parameters generated in step A8, generate the clustering results of each report, evaluate the clustering results using the confusion matrix, and output the representative attack methods and attack target characteristic keywords of each APT organization.
2. The method for identifying advanced persistent threat organizations based on LDA according to claim 1, characterized in that, The offline phase specifically includes: Step 1) Crawl analysis reports released by major security vendors with clear APT organization classifications, save the full text of each report to a separate file, and store them in the corresponding database according to the different APT organizations they belong to. Collect analysis reports on 7 recently active APT organizations. In the specific application process, there is no limit to the number of APT organizations. Step 2) Use Python to perform text preprocessing on the obtained report, specifically including: removing all punctuation marks contained in string.punctuation, removing all English stop words contained in nltk.corpus.stopwords, converting all words to lowercase, and using nltk.stem.wordnet.WordNetLemmatizer to perform part-of-speech tagging on all words; Step 3) Use gensim.corpora.Dictionary to generate a dictionary from the cleaned text; Step 4) Use the doc2bow function of the Dictionary class to convert the cleaned text from step 2) into a vector representation based on the generated dictionary; Step 5) For each of the 7 APT organizations, use the vectors generated from their respective analysis reports to train an LDA model with 1 topic using gensim.models.ldamodel.LdaModel. The dictionary used is the one generated in step 3), with chunksize set to 5, passes set to 200, and iterations set to 500. After the model training is complete, use the print_topics function to output the 25 feature words with the highest weights for each APT organization. Step 6) Summarize the feature words output by the 7 APT organizations, remove the feature words that are output by three or more APT organizations and are not representative, construct a new dictionary with significantly reduced dimensionality and more representative feature words using gensim.corpora.Dictionary, and store the new dictionary locally. Step 7) Based on the generated new dictionary, use the doc2bow function of the Dictionary class to convert the cleaned text from step 2) into a vector representation; Step 8) Based on the generated vector representation, use gensim.models.TfidfModel to count the term frequency and inverse document frequency, calculate the TF-IDF parameters, and store the term frequency and inverse document frequency locally; Step 9) Based on the calculated TF-IDF parameters, train an LDA model with the number of topics equal to the number of APT organizations using gensim.models.ldamodel.LdaModel. Use the dictionary generated in step 6), set chunksize to 5, passes to 200, and iterations to 500, and store the model locally. Step 10) Based on the LDA model obtained from training and the TF-IDF parameters generated in Step 8), use the get_document_topics function to generate the clustering results of each report, use the confusion matrix to evaluate the clustering results, and use the print_topics function to output the representative attack methods and attack target characteristics of each APT organization.
3. The method for identifying advanced persistent threat organizations based on LDA according to claim 1, characterized in that, The online phase includes: Step B1: Preprocess the report text released by the security company, specifically including: removing all punctuation marks, converting all words to lowercase, and restoring the part-of-speech of all words; Step B2: Based on the dictionary generated in step A6, use the bag-of-words model to convert the preprocessed report text into a vector representation; Step B3: Calculate the TF-IDF parameters based on the generated vector representation and the term frequency and inverse document frequency obtained in step A8; Step B4: Based on the calculated TF-IDF parameters and the LDA model generated in step A9, determine the APT organization to which the attack in the report belongs by calculating the cosine similarity between the TF-IDF parameters and the cluster centers obtained in the LDA model, and output its similarity with the typical attack vectors in the organization.
4. The method for identifying advanced persistent threat organizations based on LDA according to claim 1 or 3, characterized in that, The online phase specifically includes: Step A) Use Python to perform text preprocessing on the reports from security companies that have been crawled. Specifically, this includes: removing all punctuation marks from string.punctuation, converting all words to lowercase, and using nltk.stem.wordnet.WordNetLemmatizer to perform part-of-speech tagging on all words. Step B) Based on the dictionary generated in step 6) during the previous training process, use the doc2bow function of the Dictionary class to convert the report text into a vector representation; Step C) Calculate the TF-IDF parameters using gensim.models.TfidfModel based on the generated vector representation and the word frequencies and inverse document frequencies counted in step 8) during the previous training process; Step D) Based on the calculated TF-IDF parameters and the LDA model generated in step 9) during the previous training process, use the get_document_topics function to generate the clustering results of the report, determine the APT organization to which the attack in the report belongs, and output the similarity between the attack disclosed in the report and the typical attack vector in the APT organization.
Citation Information
Patent Citations
APT organization identification method and system based on stacking integration and storage medium
CN111797394A
Attack skill extraction method based on condition co-occurrence degree
CN114547290A
