APT attack technology identification and matching method and system based on threat intelligence

Abstract

The present invention provides a method, system, storage medium, and electronic device for identifying and matching APT attack technologies based on threat intelligence. The method comprises: obtaining multiple threat intelligence statements; training an initial classification model based on the multiple threat intelligence statements to obtain a final classification model, and sequentially inputting the sentences to be detected into the final classification model to obtain a classification result for each sample to be detected; converting each word in the sentence to be detected for which the classification result is relevant into a high-dimensional context embedding vector representation, and performing entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation; matching the structured relationship triple with any ATT&CK technical information, and obtaining the target ATT&CK technical information corresponding to the sentence to be detected based on the matching result. The present invention can solve the problems of high false positive rate and missed negative phenomenon.

Description

Technical Field

[0001] The present invention relates to the field of network attack identification technology, and in particular to a method and system for identifying and matching APT attack technologies based on threat intelligence. Background Art

[0002] In recent years, the complexity and stealthiness of APT (Advanced Persistent Threat) attacks have continued to grow, making traditional security measures difficult to effectively counter. These attacks often employ multi-stage, multi-method strategies, combining technical intrusions with social engineering tactics, lurking within systems for extended periods and performing covert and precise operations within target systems. Traditional defenses struggle to effectively detect and respond to these attacks. Furthermore, the varying formats of threat intelligence (CTI) released by different vendors and security organizations complicate and hinder intelligence integration and analysis.

[0003] To address these challenges, the industry has recently proposed template matching solutions. Early approaches to extracting APT techniques and tactics (TTPs) primarily relied on template matching technology. This technology, employed by some tools, matches attack signatures in text using predefined patterns. While template matching can effectively extract structured attack information to a certain extent, its primary drawback is its lack of flexibility. Template matching methods can only identify predefined keywords or fixed patterns, making it difficult to capture semantic variations. Consequently, when faced with novel attack patterns or complex, ever-changing cross-platform data, they are prone to high false positives and false negatives. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for identifying and matching APT attack technologies based on threat intelligence, aiming to solve the problem that traditional technologies are prone to high false alarm rates and missed alarms.

[0005] In a first aspect, the present invention provides a method for identifying and matching APT attack technologies based on threat intelligence, the method comprising:

[0006] Acquire historical threat intelligence data, and preprocess and sentence segment the historical threat intelligence data to obtain multiple threat intelligence sentences;

[0007] Training an initial classification model based on the multiple threat intelligence statements to obtain a final classification model, and sequentially inputting the sentences to be tested into the final classification model to obtain a classification result for each sample to be tested, the classification result including relevant and irrelevant;

[0008] Convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and perform entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation;

[0009] The technical and tactical description corpus from the MITRE official website is read in. The technical and tactical description corpus contains multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the target ATT&CK technical information corresponding to the statement to be detected is obtained based on the matching result.

[0010] Furthermore, the step of training the initial classification model according to the plurality of threat intelligence statements to obtain the final classification model includes:

[0011] Each threat intelligence sentence is labeled as relevant or irrelevant, and the labeled threat intelligence sentence is embedded to convert each threat intelligence sentence into a high-dimensional vector representation to obtain a sentence embedding vector;

[0012] The sentence embedding vector is used as the input of the initial classification model to train a final classification model.

[0013] Furthermore, the step of inputting the sentences to be tested into the final classification model in sequence to obtain the classification result of each sample to be tested includes:

[0014] The final classification model obtains the predicted probability of the sentence to be detected according to the following formula:

[0015]

[0016] Among them, P i represents the predicted probability that the i-th sample to be detected is a related sentence, σ represents the sigmoid activation function, which is used to convert the linear output into a probability in the range of [0,1], ω represents the weight vector of the model, X i represents the sentence embedding vector of the i-th sample to be tested, and b represents the bias term of the model;

[0017] Determining whether the predicted probability of the sentence to be detected is greater than a preset probability threshold;

[0018] If the predicted probability of the sentence to be detected is greater than a preset probability threshold, the classification result of the sentence to be detected is determined to be relevant;

[0019] If the predicted probability of the sentence to be detected is less than or equal to a preset probability threshold, the classification result of the sentence to be detected is determined to be irrelevant.

[0020] Furthermore, the step of using the sentence embedding vector as the input of the initial classification model to train a final classification model includes:

[0021] The loss function of the initial classification model is constructed according to the following formula:

[0022]

[0023] Among them, L represents the loss value, N represents the total number of samples to be tested, and y i represents the true label of the i-th sample to be tested, logP i Indicates that the classification result is the relevant predicted probability P i The logarithm of log(1-P i ) indicates the predicted probability that the classification result is irrelevant (1-P i )

[0024] Minimize the loss value and obtain the final classification model.

[0025] Furthermore, the step of converting each word in the sentence to be detected as being related in the classification result into a high-dimensional context embedding vector representation, and performing entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation, includes:

[0026] Each word in the sentence to be detected that is classified as relevant is converted into a high-dimensional context embedding vector representation according to the following formula:

[0027] X=BERT(T)

[0028] Where T indicates that the classification result is the relevant sentence to be detected, X represents the high-dimensional context embedding vector representation, and BERT(·) represents the BERT word embedding operation;

[0029] Entity annotation is performed on each of the high-dimensional context embedding vector representations, and a head entity is obtained according to the entity annotation result, so as to extract entities related to the head entity based on the head entity to obtain a structured relationship triple.

[0030] Furthermore, the technical and tactical description corpus read from the MITRE official website includes multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the step of obtaining target ATT&CK technical information corresponding to the statement to be detected based on the matching result includes:

[0031] Similarity matching is performed according to the following formula:

[0032]

[0033] Among them, d i Indicates the similarity between the sample to be tested and the i-th ATT&CK technical information, u title 、u desc 、u extra They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation extracted from the sample to be tested, respectively. They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation of the i-th ATT&CK technical information in the technical and tactical description corpus, respectively. α, β, and γ all represent weights.

[0034] A maximum similarity value is screened out from all similarity values, and ATT&CK technical information corresponding to the maximum similarity value is extracted from the technical and tactical description corpus.

[0035] Furthermore, the steps of obtaining historical threat intelligence data, preprocessing and sentence segmenting the historical threat intelligence data, and obtaining a plurality of threat intelligence sentences include:

[0036] De-noising and text cleaning are performed on the historical threat intelligence data to remove special characters, advertising information, comments, and text sentences in the historical threat intelligence data;

[0037] Perform sentence segmentation on historical threat intelligence data after denoising and text cleaning.

[0038] In a second aspect, the present invention provides a threat intelligence-based APT attack technology identification and matching system, the system comprising:

[0039] A data acquisition module is used to obtain historical threat intelligence data, and preprocess and sentence segment the historical threat intelligence data to obtain multiple threat intelligence sentences;

[0040] A model training module is used to train the initial classification model based on the multiple threat intelligence statements to obtain a final classification model, and sequentially input the sentences to be tested into the final classification model to obtain a classification result for each sample to be tested, wherein the classification result includes relevant and irrelevant results;

[0041] A vector embedding module, configured to convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and perform entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation, wherein the structured relationship triple includes a vector representation of the technical title, a vector representation of the technical description, and a vector representation of additional technical information;

[0042] The matching module is used to read the technical and tactical description corpus from the MITRE official website, where the technical and tactical description corpus contains multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the target ATT&CK technical information corresponding to the statement to be detected is obtained based on the matching result.

[0043] In a third aspect, the present invention provides a storage medium storing one or more programs, which, when executed by a processor, implement the above-mentioned APT attack technology identification and matching method based on threat intelligence.

[0044] In a fourth aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein:

[0045] The memory is used to store computer programs;

[0046] When the processor is used to execute the computer program stored in the memory, the above-mentioned APT attack technology identification and matching method based on threat intelligence is implemented.

[0047] Compared with the prior art, the present invention has the following advantages:

[0048] 1. This invention takes the lead in text preprocessing by segmenting the text and extracting technical and tactical related sentences. This method changes the traditional way of directly analyzing the entire text of threat intelligence. The system can focus more on content directly related to technical and tactical tasks and remove irrelevant content, thereby improving matching accuracy and saving computing resources.

[0049] 2. This invention proposes to extract entities and relationships from multi-source threat intelligence through a BERT-CRF combination. This approach uses an attention mechanism to focus on the relevant information of the head entity, and combines it with a BiLSTM layer to identify relationships. The extracted entities and relationships are then mapped to ATT&CK. The mapped TTPs help security teams identify and trace threat behavior patterns, determine possible subsequent steps for the current attack, and form a holistic attack scenario assessment. In unstructured CTI from multiple sources, this solution uses contextual semantic understanding of intelligence content, rather than being limited to surface string matching, a multi-level parsing approach. This approach can capture key information and relationships within the APT attack chain.

[0050] 3. This paper proposes a multi-dimensional similarity calculation method to match the entity relationships related to attack patterns obtained from threat intelligence with MITRE ATT&CK technology. It adds multi-dimensional weight matching, so that titles, descriptions, additional information, etc. are weighted according to importance (other weights can be dynamically adjusted according to specific situations), enhancing the algorithm's accuracy in noise filtering and false alarm reduction, and further improving the accuracy of detecting new APT threats. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of a method for identifying and matching APT attack technologies based on threat intelligence, proposed in one embodiment of the present invention;

[0052] Figure 2 This is an overall framework diagram of the APT attack technology identification and matching method based on threat intelligence proposed in one embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of sentence segmentation results according to an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of entity relationship extraction according to an embodiment of the present invention;

[0055] Figure 5 This is a structural diagram of the APT attack technology identification and matching system based on threat intelligence proposed in one embodiment of the present invention.

[0056] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meanings understood by people with ordinary skills in the field to which the invention belongs. The words "including" and similar words used in this article mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0058] like Figures 1 to 2 As shown, an embodiment of the present invention provides a method for identifying and matching APT attack technologies based on threat intelligence, the method comprising steps S101 to S104, wherein:

[0059] Step S101: Acquire historical threat intelligence data, and perform preprocessing and sentence segmentation on the historical threat intelligence data to obtain multiple threat intelligence sentences;

[0060] It should be noted that this historical threat intelligence data comes from APT reports from well-known companies, security blogs, hacker forum posts, open source threat announcements released by security vendors, etc., to ensure comprehensive coverage of threat intelligence data.

[0061] In addition, some embodiments use automated crawler technology to obtain APT-related data, namely historical threat intelligence data, from multiple threat intelligence sources and the ATT&CK developed by MITRE. Distributed crawlers are built using Python frameworks (Scrapy or Pyppeteer), and specific crawling rules and anti-crawling mechanism bypass strategies are designed for different websites (some using manual copying) to ensure efficient crawling of threat intelligence data. Timestamps and update frequencies are set to continuously obtain the latest APT attack reports.

[0062] Furthermore, in some embodiments, the historical threat intelligence data obtained is mostly incomplete, inconsistent, missing, or redundant text data. Directly training deep learning algorithms on such data often yields suboptimal results. Therefore, to improve learning effectiveness and quality, data preprocessing and cleaning are required before use to remove textual noise.

[0063] Therefore, the previously extracted historical threat intelligence data was denoised and preprocessed to include the removal of irrelevant information such as special characters, advertising information, comments, text sentences, etc.

[0064] Specifically, using Scikit-learn to clean text data can effectively reduce noise. Common steps include removing stop words and performing stemming. Removing stop words (such as frequently occurring words like "的", "是", "啊", etc., which usually do not contain substantial meaning) helps reduce the interference of irrelevant information.

[0065] Performing stemming using NLP tools, which is to restore words to their most basic form, so that words in different forms (such as "利用", "利用了") are grouped into one category to improve the generalization ability of the model.

[0066] Parsing using HTML parsing technology to delete HTML tags (such as 、 <font>etc.), remove junk content from APT intelligence texts, such as advertisements, comments, irrelevant news, promotional content, etc., through natural language processing and text filtering technology.

[0067] At the same time, special characters commonly found in APT threat intelligence (such as Greek letters, mathematical symbols, square symbols, emoticons, and modifiers) can also interfere with the training of natural language processing models. To this end, regular expressions can be used for text matching to remove these irrelevant special characters.

[0068] Furthermore, in some embodiments, historical threat intelligence data in NLP tasks is often a long article or paragraph. To facilitate subsequent processing and analysis, the text must first be segmented into sentences. This is because sentences, as an appropriate semantic granularity, can provide sufficient information in smaller units without compromising computational efficiency due to excessive length. Furthermore, many NLP algorithms process data at the sentence level, so segmenting the text into clauses helps improve processing accuracy and effectiveness.

[0069] When processing historical threat intelligence data related to APT, since the text may contain special characters, irregular paragraphs, and abnormal formats, sentence segmentation can make each sentence the smallest processing unit, facilitating the generation of subsequent training data. Use the sent_tokenize() method in the NLTK library, which can accurately segment sentences based on language rules. For text with line breaks or other special characters, you can first use Python's split() function for preliminary segmentation, and then combine sent_tokenize() to complete sentence-level segmentation, thereby achieving more efficient natural language processing and training. For example,< / font> <font> Figure 3 shown.

[0070] Step S102: Training an initial classification model based on the plurality of threat intelligence statements to obtain a final classification model, and sequentially inputting the sentences to be tested into the final classification model to obtain a classification result for each sample to be tested, the classification result including relevant and irrelevant.

[0071] In this step, we first embed threat intelligence sentences using BGE M3-Embedding, converting each sentence into a high-dimensional vector representation. These embedding vectors not only contain contextual information but are also lightweight and efficient. This allows for rapid capture of sentence semantics, enabling the model to understand whether each sentence contains content relevant to APT attack patterns, making it ideal for large-scale data processing scenarios.

[0072] Then, during the model construction process, a logistic regression binary classification model is trained using the sentence embedding vector as input. This model's goal is to determine whether a sentence contains information relevant to APT attack techniques and tactics. The logistic regression model outputs a probability value, predicting whether the sentence is "relevant" or "irrelevant" (where 0 indicates "irrelevant" and 1 indicates "relevant"). In other words, the initial classification model used in this step is a logistic regression binary classification model.

[0073] Furthermore, the cross-entropy loss function is used to train the initial classification model. Logistic regression is a model used for binary classification problems. The output is a probability value, indicating the possibility that a sample belongs to a certain category (for example, "relevant" or "irrelevant"). Its core is to map the output of the linear model to between 0 and 1 through the sigmoid activation function. The formula is as follows:

[0074]

[0075] Among them, P i represents the predicted probability that the i-th sample to be detected is a related sentence, σ represents the sigmoid activation function, which is used to convert the linear output into a probability in the range of [0,1], ω represents the weight vector of the model, X i represents the sentence embedding vector of the i-th sample to be tested, and b represents the bias term of the model.

[0076] In addition, the goal of logistic regression is to make the probability P of the model output i As close to the true label as possible. This is a supervised learning task, and we need a loss function to measure the gap between the model's prediction results and the true label. The cross entropy loss function is just right for this probabilistic prediction task, and the formula is as follows:

[0077]

[0078] Among them, L represents the loss value, which is the difference between the model prediction result and the true label. The goal is to minimize this value during training; N represents the total number of samples to be tested, which is the number of samples used when calculating the loss at one time, usually the number of samples in a batch; y i Represents the true label of the i-th sample to be tested. For binary classification problems, the true label y i Either 0 (indicating negative class) or 1 (indicating positive class); logP i Indicates that the classification result is the relevant predicted probability P i The logarithm of the positive sample (the sample with the true label of 1), the model's predicted probability P i The closer it is to 1, the smaller the loss value; conversely, the loss value increases when the prediction is wrong; log(1-P i ) indicates the predicted probability that the classification result is irrelevant (1-P i ), for negative samples (samples with a true label of 0), if the model predicts the probability P i The closer it is to 0, the greater the 1-P i The closer to 1, the smaller the loss value; if P i The closer it is to 1, the greater the loss value.

[0079] Finally, after training the final classification model, the BGE M3-Embedding model is used to convert each sentence into a vector representation, which is then input into the trained final classification model to output whether the sentence is a relevant sentence. The specific judgment process is as follows:

[0080] Determining whether the predicted probability of the sentence to be detected is greater than a preset probability threshold;

[0081] If the predicted probability of the sentence to be detected is greater than a preset probability threshold, the classification result of the sentence to be detected is determined to be relevant;

[0082] If the predicted probability of the sentence to be detected is less than or equal to a preset probability threshold, the classification result of the sentence to be detected is determined to be irrelevant.

[0083] For example, if the probability threshold is preset to 0.8, if the model outputs a probability of 0.85, the sentence will be classified as "relevant". If it is 0.75, because it is less than the threshold of 0.8, the sentence will be filtered as "irrelevant".

[0084] Step S103: Convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and perform entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation;

[0085] like Figure 4 As shown, in this step, each word in the sentence to be detected that is classified as relevant is converted into a high-dimensional context embedding vector representation according to the following formula:

[0086] X=BERT(T)

[0087] Here, T indicates that the classification result is the relevant sentence to be detected, X represents the high-dimensional context embedding vector representation, and BERT(·) represents the BERT word embedding operation.

[0088] It should be noted that in the CRF header entity extraction layer, CRF is used to label each word based on the vector X generated by the embedding layer to determine the category of the entity (such as "B-Attacker", "B-Tool", "I-Tool", etc.).

[0089] Furthermore, for entity relationship extraction, BERT-CRF is fused with an attention mechanism. First, the BERT word embedding layer converts each word in a sentence into a high-dimensional vector embedding to capture contextual semantic information. Next, the CRF head entity extraction layer uses the BIO annotation method to label each word with its entity category (for example, labeling "hacker" as an attacker and "CobaltStrike" as a tool). The model then applies attention features to the identified head entity through a head-entity-based attention mechanism semantic fusion layer. This helps the model focus on the relationship between the head entity and other entities or words in the sentence, focusing on contextual information related to the head entity and enhancing the semantic representation. The following BiLSTM layer further captures sequential dependencies, helping to understand the interactive relationships between words. Finally, the model outputs start and end position markers for the tail entity, identifying the relationship between the head and tail entities, such as "hacker uses Cobalt Strike" and "hacker attacks the victim's system," thereby generating structured relationship triples suitable for cybersecurity applications such as the ATT&CK framework.

[0090] For example, for example, the input sentence is: The hacker used Cobalt Strike to attack the victim's system. The first is the BERT word embedding layer: the input sentence is encoded by the BERT model, and each word is converted into a context embedding vector. These embedding vectors contain the semantic and contextual information of the word, helping the model understand the role of the word in the sentence. For each word in the sentence, a corresponding embedding vector is generated, such as: "The", "hacker", "used", "Cobalt", "Strike", "to", "attack", "victim's", "system". Then comes the CRF head entity extraction layer: the CRF layer is responsible for identifying and labeling entity categories from the BERT embedding. Through this layer, the model assigns entity labels to each word and identifies the key entities in the sentence. In this example, the annotation results are: "The" → O, "hacker" → B-Attacker, "used" → O, "Cobalt" → B-Tool, "Strike" → I-Tool, "to" → O, "attack" → O, "the" → O, "victim's" → B-Target, "system" → I-Target. The final output annotation sequence is: O B-Attacker O B-Tool I-Tool OO B-Target I-Target. Next, the semantic fusion layer uses the attention mechanism based on the head entity: using "hacker" as the head entity, the model applies an attention mechanism, focusing on words related to "hacker" in the sentence. The attention mechanism enables the model to focus on the contextual information related to "hacker," thereby extracting the entities related to "hacker": "Cobalt Strike" and "victim's system." By focusing on the context, this layer enhances the model's understanding of the underlying relationships between entities, making subsequent relationship recognition more accurate. Finally, the BiLSTM layer passes the embedded data after attention fusion to the BiLSTM layer, further capturing the sequential dependencies of the sentence. The BiLSTM layer combines contextual information to analyze the relationships between "hacker," "Cobalt Strike," and "victim's system." The BiLSTM layer output is used to locate the start and end points of the tail entity and help identify the specific relationship between the head and tail entities. The final output is the relationship and entity labels: Through BiLSTM and classifier analysis, the model identifies and extracts the following relationship groups: hacker - uses - Cobalt Strike; hacker - attacks - victim's system.

[0091] Step S104: Read the technical and tactical description corpus from the MITRE official website, where the technical and tactical description corpus contains multiple ATT&CK technical information pieces, each of which includes a technical title, a technical description, and additional technical information. Match the structured relationship triple with any of the ATT&CK technical information pieces, and obtain the target ATT&CK technical information corresponding to the statement to be detected based on the matching result.

[0092] It should be noted that in this step, the corpus of technical and tactical descriptions from the MITRE official website is first read in, and a keyword list is generated and stored in the corpus using attack pattern recognition and extraction. The ATT&CK technical and tactical descriptions (including titles, descriptions, etc.) are converted into vector embeddings, and then the overlap ratio with multiple pairs of entity and relationship classifications extracted from threat intelligence is calculated, and the technical and tactical descriptions with the maximum similarity value are output.

[0093] Specifically, we use keyword matching and cosine similarity to map entities to titles, descriptions, and additional information. We then verify whether the entities and relationships extracted by BERT-CRF and the attention mechanism are consistent with known attack patterns (MITRE ATT&CK techniques) to determine the accuracy of the attack phrase. The formula is as follows:

[0094]

[0095] Among them, d i Indicates the similarity between the sample to be tested and the i-th ATT&CK technical information, u title 、u desc 、u extra They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation extracted from the sample to be tested, respectively. They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation of the i-th ATT&CK technical information in the technical and tactical description corpus. α, β, and γ all represent weights, which are used to optimize the weighting strategy. The importance of titles, descriptions, and additional information of different attack techniques varies, and the weights can be dynamically adjusted based on the actual intelligence content.

[0096] Then, the maximum similarity value is screened out from all the similarity values, and the ATT&CK technical information corresponding to the maximum similarity value is extracted from the technical and tactical description corpus.

[0097] The main purpose of the above steps is to standardize and structure unstructured threat intelligence. First, entity extraction is achieved through a BERT-CRF combination. An attention mechanism is used to focus on the relevant information of the head entity. A BiLSTM layer is then used to identify relationships between entities. The extracted entities and relationships are mapped to the MITRE ATT&CK framework, which not only improves detection accuracy but also provides relevant mitigation strategies. This mapping process plays a vital role in intelligence sharing and rapid response.

[0098] Furthermore, techniques and tactics from threat intelligence are compared with known techniques and tactics in the ATT&CK framework. If the similarity between the extracted techniques and tactics and the known techniques and tactics is too low, this may indicate the presence of new techniques and tactics. Based on this, in some other embodiments of the present invention, a critical threshold is set. When the calculated similarity values ​​are all less than the critical threshold, it indicates the presence of new techniques and tactics. At this time, the automatically detected "new technology candidates" will enter the manual review stage, where security experts will analyze their context, details, and differences from other techniques and tactics. This manual review further verifies whether these techniques and tactics are indeed new technologies or variations or repetitions of existing techniques and tactics.

[0099] In summary, the threat intelligence-based APT attack technique identification and matching method described above addresses the shortcomings of traditional methods in addressing emerging threats and large-scale data. The system extracts entities and relations from threat intelligence using a BERT-CRF combination. An attention mechanism focuses on relevant information about the head entity, and a BiLSTM layer is used to identify relationships. This automatically parses unstructured data from CTI reports across multiple platforms, extracting sentences related to attack techniques and tactics, such as attackers, tools, and vulnerabilities, and converting them into high-dimensional vectors. This process reduces reliance on template matching and, combined with a cosine similarity matching algorithm, accurately maps extracted entities to MITRE ATT&CK techniques and tactics, providing reliable intelligence support for security teams. When suspicious new techniques and tactics are detected, manual analysis is introduced to review the extracted techniques and tactics, detailed descriptions, and related sentences to assess their reliability, thereby providing security analysts with more accurate decision-making.

[0100] On the other hand, the present invention further proposes a storage medium on which one or more programs are stored. When the program is executed by a processor, the above-mentioned APT attack technology identification and matching method based on threat intelligence is implemented.

[0101] On the other hand, the present invention also proposes an electronic device, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the above-mentioned threat intelligence-based APT attack technology identification and matching method.

[0102] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0103] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0104] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement the hardware: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0105] While the embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations of these embodiments are possible. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the claims. Furthermore, the invention described herein is susceptible to other embodiments and may be practiced or implemented in a variety of ways.< / font>

Claims

1. A method for identifying and matching APT attack technologies based on threat intelligence, characterized in that: The method comprises: Acquire historical threat intelligence data, and preprocess and sentence segment the historical threat intelligence data to obtain multiple threat intelligence sentences; Training an initial classification model based on the plurality of threat intelligence sentences to obtain a final classification model includes: labeling each threat intelligence sentence as relevant or irrelevant, and embedding the labeled threat intelligence sentences to convert each threat intelligence sentence into a high-dimensional vector representation to obtain a sentence embedding vector; and using the sentence embedding vector as input to the initial classification model to train a final classification model; The sentences to be tested are sequentially input into the final classification model to obtain the classification results of each sample to be tested, wherein the classification results include relevant and irrelevant results, including: the final classification model obtains the predicted probability of the sentence to be tested according to the following formula: ， in, Indicates the predicted probability that the i-th sample to be tested is a related sentence, express Activation function, used to convert linear output into a probability in the range of [0,1], represents the weight vector of the model, Represents the sentence embedding vector of the i-th sample to be tested, Represents the bias term of the model; Determine whether the predicted probability of the sentence to be detected is greater than a preset probability threshold; if the predicted probability of the sentence to be detected is greater than the preset probability threshold, determine that the classification result of the sentence to be detected is relevant; if the predicted probability of the sentence to be detected is less than or equal to the preset probability threshold, determine that the classification result of the sentence to be detected is irrelevant; Convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and perform entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation; The technical and tactical description corpus from the MITRE official website is read in. The technical and tactical description corpus contains multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the target ATT&CK technical information corresponding to the statement to be detected is obtained based on the matching result.

2. The APT attack technology identification and matching method based on threat intelligence according to claim 1 is characterized in that: The step of using the sentence embedding vector as the input of the initial classification model to train a final classification model includes: The loss function of the initial classification model is constructed according to the following formula: ， Among them, L represents the loss value, N represents the total number of samples to be tested, represents the true label of the i-th sample to be tested, Indicates that the classification result is the relevant predicted probability The logarithm of Indicates the predicted probability that the classification result is irrelevant The logarithm of Minimize the loss value and obtain the final classification model.

3. The APT attack technology identification and matching method based on threat intelligence according to claim 1 is characterized in that: The steps of converting each word in the sentence to be detected whose classification result is related into a high-dimensional context embedding vector representation, and performing entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation, include: Each word in the sentence to be detected that is classified as relevant is converted into a high-dimensional context embedding vector representation according to the following formula: ， in, Indicates that the classification result is a related sentence to be detected. represents the high-dimensional context embedding vector representation, Represents the BERT word embedding operation; Entity annotation is performed on each of the high-dimensional context embedding vector representations, and a head entity is obtained according to the entity annotation result, so as to extract entities related to the head entity based on the head entity to obtain a structured relationship triple.

4. The APT attack technology identification and matching method based on threat intelligence according to claim 1 is characterized in that: The technical and tactical description corpus read from the MITRE official website includes multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the step of obtaining the target ATT&CK technical information corresponding to the statement to be detected based on the matching result includes: Similarity matching is performed according to the following formula: ， in, Indicates the similarity value between the sample to be tested and the i-th ATT&CK technical information. 、 、 They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation extracted from the sample to be tested, respectively. 、 、 They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation of the i-th ATT&CK technical information in the technical and tactical description corpus, respectively. 、 、 All represent weights; A maximum similarity value is screened out from all similarity values, and ATT&CK technical information corresponding to the maximum similarity value is extracted from the technical and tactical description corpus.

5. The APT attack technology identification and matching method based on threat intelligence according to claim 1 is characterized in that: The steps of obtaining historical threat intelligence data, preprocessing the historical threat intelligence data and performing sentence segmentation to obtain multiple threat intelligence sentences include: De-noising and text cleaning are performed on the historical threat intelligence data to remove special characters, advertising information, comments, and text sentences in the historical threat intelligence data; Perform sentence segmentation on historical threat intelligence data after denoising and text cleaning.

6. A threat intelligence-based APT attack technology identification and matching system, characterized by: The system comprises: A data acquisition module is used to obtain historical threat intelligence data, and preprocess and sentence segment the historical threat intelligence data to obtain multiple threat intelligence sentences; A model training module is configured to train an initial classification model based on the plurality of threat intelligence statements to obtain a final classification model, including: labeling each threat intelligence statement as relevant or irrelevant, and embedding the labeled threat intelligence statements to convert each threat intelligence statement into a high-dimensional vector representation to obtain a sentence embedding vector; and using the sentence embedding vector as input to the initial classification model to train a final classification model; The sentences to be tested are sequentially input into the final classification model to obtain the classification results of each sample to be tested, wherein the classification results include relevant and irrelevant results, including: the final classification model obtains the predicted probability of the sentence to be tested according to the following formula: ， in, Indicates the predicted probability that the i-th sample to be tested is a related sentence, express Activation function, used to convert linear output into a probability in the range of [0,1], represents the weight vector of the model, Represents the sentence embedding vector of the i-th sample to be tested, Represents the bias term of the model; Determine whether the predicted probability of the sentence to be detected is greater than a preset probability threshold; if the predicted probability of the sentence to be detected is greater than the preset probability threshold, determine that the classification result of the sentence to be detected is relevant; if the predicted probability of the sentence to be detected is less than or equal to the preset probability threshold, determine that the classification result of the sentence to be detected is irrelevant; A vector embedding module, configured to convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and perform entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation, wherein the structured relationship triple includes a vector representation of the technical title, a vector representation of the technical description, and a vector representation of additional technical information; The matching module is used to read the technical and tactical description corpus from the MITRE official website, where the technical and tactical description corpus contains multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the target ATT&CK technical information corresponding to the statement to be detected is obtained based on the matching result.

7. A storage medium, characterized in that: The storage medium stores one or more programs, which, when executed by the processor, implement the APT attack technology identification and matching method based on threat intelligence as described in any one of claims 1 to 5.

8. An electronic device comprising a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the APT attack technology identification and matching method based on threat intelligence as described in any one of claims 1 to 5.

Images