APT attack technology identification and matching method and system based on threat intelligence

Abstract

The invention provides an APT attack technology identification and matching method and system based on threat intelligence, a storage medium and electronic equipment. The method comprises the following steps: acquiring a plurality of threat intelligence statements; training the initial classification model according to the plurality of threat intelligence statements to obtain a final classification model, and sequentially inputting the to-be-detected statements into the final classification model to obtain a classification result of each to-be-detected sample; converting each word in the to-be-detected statement of which the classification result is related into a high-dimensional context embedding vector representation, and performing entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each high-dimensional context embedding vector representation; the structured relation triple is connected with any ATTamp; matching the CK technical information, and obtaining a target ATTamp corresponding to the to-be-detected statement according to a matching result; technical information is CK. According to the invention, the problems of high false alarm rate and missing alarm phenomenon can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of network attack identification, and in particular to a method and system for identifying and matching APT attack technologies based on threat intelligence. Background Art

[0002] In recent years, the complexity and concealment of APT attacks have continued to increase, making it difficult for traditional security protection methods to effectively respond. These attacks often use multi-stage and multi-method strategies, combining technical intrusion and social engineering methods, lurking in the system for a long time, and performing covert and precise operations in the target system. Traditional defense methods are difficult to effectively detect and respond to. At the same time, the threat intelligence (CTI) released by different manufacturers and security organizations has different formats, making the integration and analysis of intelligence complex and difficult.

[0003] In order to solve these challenges, the industry has proposed template matching solutions in recent years. In terms of APT techniques and tactics (TTPs) extraction, early methods mainly rely on template matching technology. This type of technology matches attack features in text through predefined patterns, and some tools use this method. Template matching technology can effectively extract structured attack information to a certain extent, but its main problem is the lack of flexibility. Template matching methods can only identify predefined keywords or fixed patterns, and it is difficult to capture semantic changes. Therefore, when faced with new attack patterns or complex and changeable cross-platform data, it is easy to have high false positive rates and missed reports. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for identifying and matching APT attack technologies based on threat intelligence, aiming to solve the problem that traditional technologies are prone to high false alarm rates and missed alarms.

[0005] In a first aspect, the present invention provides a method for identifying and matching APT attack technologies based on threat intelligence, the method comprising:

[0006] Acquire historical threat intelligence data, and perform preprocessing and sentence segmentation on the historical threat intelligence data to obtain multiple threat intelligence sentences;

[0007] The initial classification model is trained according to the plurality of threat intelligence statements to obtain a final classification model, and the sentences to be detected are sequentially input into the final classification model to obtain a classification result of each sample to be detected, wherein the classification result includes relevant and irrelevant;

[0008] Convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and perform entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each of the high-dimensional context embedding vector representations, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation;

[0009] The technical and tactical description corpus of the MITRE official website is read in, and the technical and tactical description corpus contains multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the target ATT&CK technical information corresponding to the statement to be detected is obtained according to the matching result.

[0010] Furthermore, the step of training the initial classification model according to the plurality of threat intelligence statements to obtain the final classification model includes:

[0011] Label each threat intelligence sentence as relevant or irrelevant, and embed the labeled threat intelligence sentences to convert each threat intelligence sentence into a high-dimensional vector representation to obtain a sentence embedding vector.

[0012] The sentence embedding vector is used as the input of the initial classification model, and the final classification model is trained.

[0013] Furthermore, the step of inputting the sentences to be detected into the final classification model in sequence to obtain the classification result of each sample to be detected includes:

[0014] The final classification model obtains the predicted probability of the sentence to be detected according to the following formula:

[0015]

[0016] Among them, P i represents the predicted probability that the i-th sample to be tested is a related sentence, σ represents the sigmoid activation function, which is used to convert the linear output into a probability in the range of [0,1], ω represents the weight vector of the model, and X i represents the sentence embedding vector of the i-th sample to be tested, and b represents the bias term of the model;

[0017] Determine whether the predicted probability of the sentence to be detected is greater than a preset probability threshold;

[0018] If the predicted probability of the sentence to be detected is greater than a preset probability threshold, the classification result of the sentence to be detected is determined to be relevant;

[0019] If the predicted probability of the sentence to be detected is less than or equal to a preset probability threshold, the classification result of the sentence to be detected is determined to be irrelevant.

[0020] Furthermore, the step of using the sentence embedding vector as the input of the initial classification model to train a final classification model includes:

[0021] The loss function of the initial classification model is constructed according to the following formula:

[0022]

[0023] Among them, L represents the loss value, N represents the total number of samples to be tested, and y i represents the true label of the i-th sample to be tested, logP i Indicates that the classification result is the relevant predicted probability P i The logarithm of log(1-P i ) indicates the predicted probability that the classification result is irrelevant (1-P i )

[0024] Minimize the loss value and get the final classification model.

[0025] Furthermore, the step of converting each word in the sentence to be detected whose classification result is related into a high-dimensional context embedding vector representation, and extracting entity relationships from the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each of the high-dimensional context embedding vector representations, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation comprises:

[0026] According to the following formula, each word in the sentence to be detected with the classification result as relevant is converted into a high-dimensional context embedding vector representation:

[0027] X=BERT(T)

[0028] Where T indicates that the classification result is the relevant sentence to be detected, X indicates the high-dimensional context embedding vector representation, and BERT(·) indicates the BERT word embedding operation;

[0029] Entity annotation is performed on each of the high-dimensional context embedding vector representations, and a head entity is obtained according to the entity annotation result, so as to extract entities related to the head entity based on the head entity to obtain a structured relationship triple.

[0030] Further, the technical and tactical description corpus read into the MITRE official website includes multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the step of obtaining the target ATT&CK technical information corresponding to the sentence to be detected according to the matching result includes:

[0031] Similarity matching is performed according to the following formula:

[0032]

[0033] Among them, d i Indicates the similarity value between the sample to be tested and the i-th ATT&CK technical information, u title 、u desc 、u extra They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation extracted from the sample to be detected. They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation of the i-th ATT&CK technical information in the technical and tactical description corpus, respectively. α, β, and γ all represent weights.

[0034] A maximum similarity value is screened out from all similarity values, and ATT&CK technical information corresponding to the maximum similarity value is extracted from the technical and tactical description corpus.

[0035] Furthermore, the step of obtaining historical threat intelligence data, preprocessing and sentence segmenting the historical threat intelligence data, and obtaining a plurality of threat intelligence sentences includes:

[0036] De-noising and text cleaning are performed on the historical threat intelligence data to remove special characters, advertising information, comments, and text sentences in the historical threat intelligence data;

[0037] Perform sentence segmentation on historical threat intelligence data after denoising and text cleaning.

[0038] In a second aspect, the present invention provides an APT attack technology identification and matching system based on threat intelligence, the system comprising:

[0039] A data acquisition module is used to acquire historical threat intelligence data, and preprocess and sentence segment the historical threat intelligence data to obtain multiple threat intelligence sentences;

[0040] A model training module is used to train the initial classification model according to the multiple threat intelligence statements to obtain a final classification model, and input the sentences to be detected into the final classification model in sequence to obtain a classification result for each sample to be detected, wherein the classification result includes relevant and irrelevant;

[0041] A vector embedding module, used to convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and extract entity relationships from the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each of the high-dimensional context embedding vector representations, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation;

[0042] A matching module is used to read the technical and tactical description corpus of the MITRE official website, wherein the technical and tactical description corpus contains multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information, and to match the structured relationship triple with any of the ATT&CK technical information, and obtain the target ATT&CK technical information corresponding to the statement to be detected according to the matching result.

[0043] In a third aspect, the present invention provides a storage medium, which stores one or more programs, which, when executed by a processor, implement the above-mentioned APT attack technology identification and matching method based on threat intelligence.

[0044] In a fourth aspect, the present invention provides an electronic device, the electronic device comprising a memory and a processor, wherein:

[0045] The memory is used to store computer programs;

[0046] When the processor is used to execute the computer program stored in the memory, the above-mentioned APT attack technology identification and matching method based on threat intelligence is implemented.

[0047] Compared with the prior art, the present invention has the following advantages:

[0048] 1. The present invention takes the lead in text preprocessing to segment the text and extract technical and tactical related sentences. This method changes the traditional way of direct full-text analysis of threat intelligence. The system can focus more on the content directly related to technical and tactical tasks and remove irrelevant content, thereby improving matching accuracy and saving computing resources.

[0049] 2. The present invention proposes to extract entities and relationships from multi-source threat intelligence through BERT-CRF combination, focus on the relevant information of the head entity through the attention mechanism, and combine the BiLSTM layer to realize relationship recognition to extract entities and relationships and map them to ATT&CK. The mapped TTPs help the security team to identify and trace the threat behavior patterns, judge the possible subsequent steps of the current attack, and form an overall attack scenario judgment. In unstructured CTI from multiple sources, this solution is based on contextual semantic understanding of intelligence content, and is not limited to the multi-level parsing method of surface string matching, which can capture key information and relationships in the APT attack chain.

[0050] 3. The present invention proposes a multi-dimensional similarity calculation method to match the entity relationships related to the attack patterns obtained from threat intelligence with the MITRE ATT&CK technology, and adds multi-dimensional weight matching, so that the title, description, additional information, etc. are weighted according to importance (other weights can be dynamically adjusted according to specific situations), which enhances the accuracy of the algorithm in noise filtering and false alarm reduction, and further improves the accuracy of detecting new APT threats. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of a method for identifying and matching APT attack technologies based on threat intelligence proposed in one embodiment of the present invention;

[0052] Figure 2 The overall framework diagram of the APT attack technology identification and matching method based on threat intelligence proposed in one embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a sentence segmentation result according to an example of an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of entity relationship extraction according to an example of an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of the structure of an APT attack technology identification and matching system based on threat intelligence proposed in one embodiment of the present invention.

[0056] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0058] like Figure 1 to Figure 2 As shown, an embodiment of the present invention provides a method for identifying and matching APT attack technologies based on threat intelligence, the method comprising steps S101 to S104, wherein:

[0059] Step S101: Acquire historical threat intelligence data, and perform preprocessing and sentence segmentation on the historical threat intelligence data to obtain multiple threat intelligence sentences;

[0060] It should be pointed out that the historical threat intelligence data comes from APT reports of well-known companies, security blogs, hacker forum posts, open source threat announcements issued by security vendors, etc., to ensure comprehensive coverage of threat intelligence data.

[0061] In addition, in some embodiments, automated crawler technology is used to obtain APT-related data, i.e., historical threat intelligence data, from multiple threat intelligence sources and ATT&CK proposed by MITRE. A distributed crawler is built through a Python framework (Scrapy or Pyppeteer), and specific crawling rules and anti-crawling mechanism bypass strategies are designed for different websites (partly by manual copying) to ensure efficient crawling of threat intelligence data. Set timestamps and update frequencies to continuously obtain the latest APT attack reports.

[0062] In addition, in some embodiments, most of the historical threat intelligence data obtained are incomplete, inconsistent, missing or redundant text data. If the deep learning algorithm is directly used for training on such data, the results are often unsatisfactory. Therefore, in order to improve the learning effect and quality, the data needs to be preprocessed and cleaned before use to remove text noise.

[0063] Therefore, the previously extracted historical threat intelligence data was denoised and preprocessed to include the removal of irrelevant information, such as special characters, advertising information, comments, text sentences, etc.

[0064] Specifically, using Scikit-learn to clean text data can effectively reduce noise. Common steps include removing stop words and performing stemming. Removing stop words (such as frequently occurring words like "的", "是", "啊", etc., which usually do not contain substantial meaning) helps reduce the interference of irrelevant information.

[0065] Performing stemming using NLP tools, stemming is to restore words to their most basic form, grouping different morphological forms of words (such as "利用" and "利用了") into one category to improve the generalization ability of the model.

[0066] Using HTML parsing technology for parsing, deleting HTML tags (such as 、 <font>etc.), through natural language processing and text filtering technology, remove junk content in APT intelligence text, such as advertisements, comments, irrelevant news, promotional content, etc.

[0067] At the same time, special characters commonly found in APT threat intelligence (such as Greek letters, mathematical symbols, square symbols, emoticons, and modifiers) can also interfere with the training of natural language processing models. To this end, regular expressions can be used for text matching to remove these irrelevant special characters.

[0068] In addition, in some embodiments, in NLP tasks, historical threat intelligence data is usually a long article or paragraph. In order to facilitate subsequent processing and analysis, the text must be divided into sentences first. The reason for this is that sentences, as a suitable semantic granularity, can provide sufficient information in smaller units without affecting the computational efficiency due to being too long. In addition, many NLP algorithms are processed at the sentence level, so dividing the text into clauses helps improve the accuracy and effect of processing.

[0069] When processing historical threat intelligence data related to APT, since the text may contain special characters, irregular paragraphs, and format anomalies, sentence segmentation can make each sentence the smallest processing unit, which is convenient for the generation of subsequent training data. Use the sent_tokenize() method in the NLTK library, which can accurately segment sentences based on language rules. For text with line breaks or other special characters, you can first use Python's split() function for preliminary segmentation, and then combine sent_tokenize() to complete sentence-level segmentation, thereby achieving more efficient natural language processing and training. For example,< / font> <font> Figure 3 shown.

[0070] Step S102: training the initial classification model according to the plurality of threat intelligence statements to obtain a final classification model, and inputting the sentences to be detected into the final classification model in sequence to obtain a classification result for each sample to be detected, wherein the classification result includes relevant and irrelevant.

[0071] In this step, we first use BGE M3-Embedding to embed threat intelligence sentences and convert each sentence into a high-dimensional vector representation. These embedding vectors not only contain certain contextual information, but also have the advantages of being lightweight and efficient. This can quickly capture the semantic information of sentences, allowing the model to understand whether each sentence contains content related to the APT attack mode, which is very suitable for application in large-scale data processing scenarios.

[0072] Then, in the process of model construction, the sentence embedding vector is used as input to train a logistic regression binary classification model. The goal of this model is to determine whether a sentence contains information related to APT attack techniques and tactics. The logistic regression model outputs a probability value to predict whether the sentence is "relevant" or "irrelevant" (where 0 means "irrelevant" and 1 means "relevant"). In other words, the initial classification model used in this step is a logistic regression binary classification model.

[0073] Furthermore, the cross-entropy loss function is used to train the initial classification model. Logistic regression is a model for binary classification problems. The output is a probability value, which indicates the possibility that a sample belongs to a certain category (for example, "relevant" or "irrelevant"). The core is to map the output of the linear model to between 0 and 1 through the sigmoid activation function. The formula is as follows:

[0074]

[0075] Among them, P i represents the predicted probability that the i-th sample to be tested is a related sentence, σ represents the sigmoid activation function, which is used to convert the linear output into a probability in the range of [0,1], ω represents the weight vector of the model, and X i represents the sentence embedding vector of the i-th sample to be tested, and b represents the bias term of the model.

[0076] In addition, the goal of logistic regression is to make the probability P of the model output i As close to the true label as possible. This is a supervised learning task, and we need a loss function to measure the gap between the model's prediction results and the true label. The cross entropy loss function is just suitable for this probability prediction task, and the formula is as follows:

[0077]

[0078] Among them, L represents the loss value, which is the difference between the model prediction result and the true label. The goal is to minimize this value during training; N represents the total number of samples to be tested, the number of samples used to calculate the loss at one time, usually the number of samples in a batch; y i Represents the true label of the i-th sample to be tested. For binary classification problems, the true label y i Either 0 (indicating negative class) or 1 (indicating positive class); logP i Indicates that the classification result is the relevant predicted probability P i The logarithm of the model is P, for positive samples (samples with the true label of 1), i The closer it is to 1, the smaller the loss value; conversely, the loss value increases when the prediction is wrong; log(1-P i ) indicates the predicted probability that the classification result is irrelevant (1-P i ), for negative samples (samples with the true label of 0), if the model predicts the probability P i The closer it is to 0, the greater the 1-P i The closer it is to 1, the smaller the loss value; if P i The closer it is to 1, the greater the loss value.

[0079] Finally, after training the final classification model, the BGE M3-Embedding model is used to convert each sentence of the sample to be tested into a vector representation, and then input into the trained final classification model to output whether the sentence is a related sentence. The specific judgment process is as follows:

[0080] Determine whether the predicted probability of the sentence to be detected is greater than a preset probability threshold;

[0081] If the predicted probability of the sentence to be detected is greater than a preset probability threshold, the classification result of the sentence to be detected is determined to be relevant;

[0082] If the predicted probability of the sentence to be detected is less than or equal to a preset probability threshold, the classification result of the sentence to be detected is determined to be irrelevant.

[0083] For example, the preset probability threshold is 0.8: if the probability output by the model is 0.85, the sentence will be classified as "relevant". If it is 0.75, because it is less than the threshold 0.8, the sentence will be filtered as "irrelevant".

[0084] Step S103: converting each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and performing entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each of the high-dimensional context embedding vector representations, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation;

[0085] like Figure 4 As shown, in this step, each word in the sentence to be detected with the classification result as relevant is converted into a high-dimensional context embedding vector representation according to the following formula:

[0086] X=BERT(T)

[0087] Where T indicates that the classification result is the relevant sentence to be detected, X indicates the high-dimensional context embedding vector representation, and BERT(·) indicates the BERT word embedding operation.

[0088] It should be noted that in the CRF header entity extraction layer, CRF is used to label each word based on the vector X generated by the embedding layer to determine the category of the entity (such as "B-Attacker", "B-Tool", "I-Tool", etc.).

[0089] In addition, in entity relationship extraction, BERT-CRF is fused with the attention mechanism. First, each word in the sentence is converted into a high-dimensional vector embedding through the BERT word embedding layer to capture contextual semantic information. Next, the CRF head entity extraction layer uses the BIO annotation method to annotate the entity category of each word (for example, "hacker" is marked as an attacker and "CobaltStrike" is marked as a tool). Then, the model applies attention features to the identified head entity through the semantic fusion layer of the attention mechanism based on the head entity (mainly used to help the model focus on the relationship between the head entity and other entities or words in the sentence.), focusing on the contextual information related to the head entity and enhancing the semantic representation. The following BiLSTM layer further captures the sequence dependency, which helps to understand the interactive relationship between words. Finally, the model outputs the start and end position markers of the tail entity, identifies the relationship between the head entity and the tail entity, such as "hacker uses Cobalt Strike" and "hacker attacks victim's system", thereby generating structured relationship triples suitable for network security applications such as the ATT&CK framework.

[0090] For example, for example, the input sentence is: The hacker used Cobalt Strike to attack the victim's system. The first is the BERT word embedding layer: the input sentence is encoded by the BERT model, and each word is converted into a context embedding vector. These embedding vectors contain the semantic and contextual information of the word, helping the model understand the role of the word in the sentence. For each word in the sentence, a corresponding embedding vector is generated, for example: "The", "hacker", "used", "Cobalt", "Strike", "to", "attack", "victim's", "system". Then there is the CRF head entity extraction layer: the CRF layer is responsible for identifying and labeling entity categories from the BERT embedding. Through this layer, the model assigns entity labels to each word and identifies the key entities in the sentence. In this example, the annotation results are: "The"→O, "hacker"→B-Attacker, "used"→O, "Cobalt"→B-Tool, "Strike"→I-Tool, "to"→O, "attack"→O, "the"→O, "victim's"→B-Target, "system"→I-Target. The final output annotation sequence is: O B-Attacker O B-Tool I-Tool OO B-Target I-Target. Then comes the semantic fusion layer based on the attention mechanism of the head entity: with "hacker" as the head entity, the model applies the attention mechanism and focuses on the words related to "hacker" in the sentence. The attention mechanism enables the model to focus on the contextual information related to "hacker", thereby extracting entities related to "hacker": "Cobalt Strike" and "victim's system". This layer enhances the model's understanding of the potential relationship between entities by focusing on the context, making subsequent relationship recognition more accurate. Finally, the BiLSTM layer is used. The embedding after fusion by the attention mechanism is passed to the BiLSTM layer to further capture the sequential dependencies of the sentence. The BiLSTM layer can analyze the relationship between "hacker", "Cobalt Strike" and "victim's system" in combination with contextual information. The output of the BiLSTM layer is used to locate the start and end positions of the tail entity and help identify the specific relationship between the head entity and the tail entity. The final output relationship and entity label: Through the analysis of BiLSTM and classifier, the model identifies and extracts the following relationship groups: hacker - uses - Cobalt Strike; hacker - attacks - victim's system.

[0091] Step S104: read in the technical and tactical description corpus of the MITRE official website, wherein the technical and tactical description corpus includes multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information, and match the structured relationship triple with any of the ATT&CK technical information, and obtain the target ATT&CK technical information corresponding to the statement to be detected according to the matching result.

[0092] It should be noted that in this step, the corpus of technical and tactical descriptions from the MITRE official website is first read in, and a keyword list is generated and stored in the corpus using attack pattern recognition and extraction. The ATT&CK techniques and tactics (including titles, descriptions, etc.) are converted into vector embeddings, and then the overlap ratios with multiple pairs of entity and relationship classifications extracted from threat intelligence are calculated, and the techniques and tactics with the maximum similarity value are output.

[0093] Specifically, we use keyword matching and cosine similarity to map entities to titles, descriptions, and additional information, and verify whether the entities and relationships extracted by BERT-CRF and the attention mechanism are consistent with known attack patterns (MITRE ATT&CK technology) to determine the accuracy of the attack phrase. The formula is as follows:

[0094]

[0095] Among them, d i Indicates the similarity value between the sample to be tested and the i-th ATT&CK technical information, u title 、u desc 、u extra They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation extracted from the sample to be detected. They respectively represent the technical title vector representation, technical description vector representation and additional technical information vector representation of the i-th ATT&CK technical information in the technical and tactical description corpus. α, β and γ all represent weights, which are used to optimize the weighted strategy. The importance of titles, descriptions and additional information of different attack technologies is different, and the weights can be dynamically adjusted based on the actual intelligence content.

[0096] Then, the maximum similarity value is screened out from all the similarity values, and the ATT&CK technical information corresponding to the maximum similarity value is extracted from the technical and tactical description corpus.

[0097] The main purpose of the above steps is to standardize and structure unstructured threat intelligence. First, entity extraction is achieved through the BERT-CRF combination, and the attention mechanism is used to focus on the relevant information of the head entity, and then the BiLSTM layer is combined to identify the relationship between entities. The extracted entities and relationships are mapped to the MITRE ATT&CK framework, which not only improves the accuracy of detection, but also provides corresponding mitigation strategies. This mapping process plays an important role in intelligence sharing and rapid response.

[0098] In addition, the techniques and tactics in the threat intelligence are compared with the known techniques and tactics in the ATT&CK framework. If the similarity between the extracted techniques and tactics and the known techniques and tactics is too low, it may indicate the existence of new techniques and tactics. Based on this, in some other embodiments of the present invention, by setting a critical threshold, when the calculated similarity values ​​are all less than the critical threshold, it indicates the existence of new techniques and tactics. At this time, the automatically detected "new technology candidates" will enter the manual review stage, and security experts will analyze their context, details and differences from other techniques and tactics. Through manual assisted review, it is further verified whether these techniques and tactics are indeed new technologies, or variants or repetitions of existing techniques and tactics.

[0099] In summary, according to the above-mentioned APT attack technology identification and matching method based on threat intelligence, the shortcomings of traditional methods in dealing with new threats and large-scale data are solved. The system combines the entities and relations of threat intelligence with BERT-CRF to realize entity extraction, focuses on the relevant information of the head entity through the attention mechanism, and realizes relationship recognition by combining the BiLSTM layer. The entities and relations are automatically parsed from the unstructured data in the CTI reports of multiple platforms, and the sentences related to the attack techniques and tactics such as attackers, tools and vulnerabilities are extracted and converted into high-dimensional vectors. This process reduces the dependence on template matching, and combines the cosine similarity matching algorithm to accurately map the extracted entities to the MITRE ATT&CK technique and tactics items, providing reliable intelligence support for the security team. When suspicious new techniques and tactics are detected, manual judgment is introduced to evaluate the reliability of the results by reviewing the extraction results, detailed descriptions and related sentences of the techniques and tactics, thereby providing security analysts with more accurate decision-making basis.

[0100] On the other hand, the present invention further proposes a storage medium on which one or more programs are stored, and when the program is executed by a processor, the above-mentioned APT attack technology identification and matching method based on threat intelligence is implemented.

[0101] On the other hand, the present invention also proposes an electronic device, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the above-mentioned APT attack technology identification and matching method based on threat intelligence.

[0102] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0103] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0104] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0105] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.< / font>

Claims

1. A method for identifying and matching APT attack technologies based on threat intelligence, characterized in that: The method comprises: Acquire historical threat intelligence data, and perform preprocessing and sentence segmentation on the historical threat intelligence data to obtain multiple threat intelligence sentences; The initial classification model is trained according to the plurality of threat intelligence statements to obtain a final classification model, and the sentences to be detected are sequentially input into the final classification model to obtain a classification result of each sample to be detected, wherein the classification result includes relevant and irrelevant; Convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and perform entity relationship extraction on the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each of the high-dimensional context embedding vector representations, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation; The technical and tactical description corpus of the MITRE official website is read in, and the technical and tactical description corpus contains multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the target ATT&CK technical information corresponding to the statement to be detected is obtained according to the matching result.

2. The APT attack technology identification and matching method based on threat intelligence according to claim 1 is characterized in that: The step of training the initial classification model according to the plurality of threat intelligence statements to obtain the final classification model comprises: Label each threat intelligence sentence as relevant or irrelevant, and embed the labeled threat intelligence sentences to convert each threat intelligence sentence into a high-dimensional vector representation to obtain a sentence embedding vector. The sentence embedding vector is used as the input of the initial classification model, and the final classification model is trained.

3. The APT attack technology identification and matching method based on threat intelligence according to claim 2 is characterized in that: The step of inputting the sentences to be detected into the final classification model in sequence to obtain the classification result of each sample to be detected includes: The final classification model obtains the predicted probability of the sentence to be detected according to the following formula: Among them, P i represents the predicted probability that the i-th sample to be tested is a related sentence, σ represents the sigmoid activation function, which is used to convert the linear output into a probability in the range of [0,1], ω represents the weight vector of the model, and X i represents the sentence embedding vector of the i-th sample to be tested, and b represents the bias term of the model; Determine whether the predicted probability of the sentence to be detected is greater than a preset probability threshold; If the predicted probability of the sentence to be detected is greater than a preset probability threshold, the classification result of the sentence to be detected is determined to be relevant; If the predicted probability of the sentence to be detected is less than or equal to a preset probability threshold, the classification result of the sentence to be detected is determined to be irrelevant.

4. The APT attack technology identification and matching method based on threat intelligence according to claim 3 is characterized in that: The step of using the sentence embedding vector as the input of the initial classification model and training to obtain the final classification model includes: The loss function of the initial classification model is constructed according to the following formula: Among them, L represents the loss value, N represents the total number of samples to be tested, and y i represents the true label of the i-th sample to be tested, logP i Indicates that the classification result is the relevant predicted probability P i The logarithm of log(1-P i ) indicates the predicted probability that the classification result is irrelevant (1-P i ) Minimize the loss value and get the final classification model.

5. The APT attack technology identification and matching method based on threat intelligence according to claim 1 is characterized in that: The step of converting each word in the sentence to be detected whose classification result is related into a high-dimensional context embedding vector representation, and extracting entity relationships from the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each of the high-dimensional context embedding vector representations, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation comprises: According to the following formula, each word in the sentence to be detected with the classification result as relevant is converted into a high-dimensional context embedding vector representation: X=BERT(T) Where T indicates that the classification result is the relevant sentence to be detected, X indicates the high-dimensional context embedding vector representation, and BERT(·) indicates the BERT word embedding operation; Entity annotation is performed on each of the high-dimensional context embedding vector representations, and a head entity is obtained according to the entity annotation result, so as to extract entities related to the head entity based on the head entity to obtain a structured relationship triple.

6. The APT attack technology identification and matching method based on threat intelligence according to claim 1 is characterized in that: The technical and tactical description corpus read into the MITRE official website includes multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information. The structured relationship triple is matched with any of the ATT&CK technical information, and the step of obtaining the target ATT&CK technical information corresponding to the sentence to be detected according to the matching result includes: Similarity matching is performed according to the following formula: Among them, d i Indicates the similarity value between the sample to be tested and the i-th ATT&CK technical information, u title 、u desc 、u extra They represent the technical title vector representation, technical description vector representation and additional technical information vector representation extracted from the sample to be detected. They represent the technical title vector representation, technical description vector representation, and additional technical information vector representation of the i-th ATT&CK technical information in the technical and tactical description corpus, respectively. α, β, and γ all represent weights. A maximum similarity value is screened out from all similarity values, and ATT&CK technical information corresponding to the maximum similarity value is extracted from the technical and tactical description corpus.

7. The APT attack technology identification and matching method based on threat intelligence according to claim 1 is characterized in that: The step of obtaining historical threat intelligence data, and preprocessing and sentence segmenting the historical threat intelligence data to obtain multiple threat intelligence sentences includes: De-noising and text cleaning are performed on the historical threat intelligence data to remove special characters, advertising information, comments, and text sentences in the historical threat intelligence data; Perform sentence segmentation on historical threat intelligence data after denoising and text cleaning.

8. A threat intelligence-based APT attack technology identification and matching system, characterized in that: The system comprises: A data acquisition module is used to acquire historical threat intelligence data, and preprocess and sentence segment the historical threat intelligence data to obtain multiple threat intelligence sentences; A model training module is used to train the initial classification model according to the multiple threat intelligence statements to obtain a final classification model, and input the sentences to be detected into the final classification model in sequence to obtain a classification result for each sample to be detected, wherein the classification result includes relevant and irrelevant; A vector embedding module, used to convert each word in the sentence to be detected that is classified as relevant into a high-dimensional context embedding vector representation, and extract entity relationships from the high-dimensional context embedding vector representation to obtain a structured relationship triple corresponding to each of the high-dimensional context embedding vector representations, wherein the structured relationship triple includes a technical title vector representation, a technical description vector representation, and an additional technical information vector representation; A matching module is used to read the technical and tactical description corpus of the MITRE official website, wherein the technical and tactical description corpus contains multiple ATT&CK technical information, each of which includes a technical title, a technical description, and additional technical information, and to match the structured relationship triple with any of the ATT&CK technical information, and obtain the target ATT&CK technical information corresponding to the statement to be detected according to the matching result.

9. A storage medium, characterized in that: The storage medium stores one or more programs, which, when executed by the processor, implement the APT attack technology identification and matching method based on threat intelligence as described in any one of claims 1-7.

10. An electronic device, comprising a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the APT attack technology identification and matching method based on threat intelligence as described in any one of claims 1-7.

Images