A method and system for constructing an attack chain and tracing the source of an attack based on textual threat intelligence

Abstract

This invention proposes a method and system for attack chain construction and attack attribution based on text-based threat intelligence. The method includes: collecting text-based threat intelligence from known attack organizations' attack events; preprocessing the collected data to meet subsequent analysis requirements; extracting network security entities and relationships from the collected text-based threat intelligence to form network security event description triplets; constructing security event attack chains based on timelines and logical order for these network security event description triplets; and finally, training an attack organization characteristic model using the security event attack chains and predicting the attribution of attack events from unknown sources based on this model. This invention starts with text-based threat intelligence, extracts network security event attack chains, and combines this with known attack sources to characterize attack organization attack feature models, effectively serving network security personnel in attributing and holding accountable those involved in network security events from unknown sources.

Description

Technical Field

[0001] This invention proposes a method and system for constructing attack chains and tracing attack sources based on text-based threat intelligence, belonging to the field of network security technology. Background Technology

[0002] With the rapid development of network information technology, the frequency and intensity of cyberattacks are constantly increasing, leading to ever-higher requirements for network security monitoring and prevention capabilities. Attack tracing is a crucial aspect of this work. However, as cyberattack techniques continue to improve, attacks are evolving from single attacks to organized and planned multi-step attacks, such as Advanced Persistent Threats (APTs). In recent years, utilizing big data analytics and machine learning to discover various malicious behaviors and their relationships within cyberattack chains, and gradually achieving attack chain recapture and tracing, has remained a challenging issue in the cybersecurity field. Currently, attack event descriptions primarily circulate within the cybersecurity industry in the form of text-based threat intelligence. The conversion of unstructured data, such as text-based threat intelligence, into attack chain information is typically done manually, resulting in low automation. Therefore, it is necessary to leverage natural language processing technology to automatically extract cybersecurity event attack chains from unstructured text information, such as text-based threat intelligence, thereby improving the efficiency of cybersecurity event analysis.

[0003] Traditional attack chain-based attribution analysis focuses on the attack chain of a single security incident, lacking horizontal comparison and correlation analysis with other security incidents, and also lacking correlation analysis with the attacking organizations behind the security incidents. However, there are many different ways to achieve the same attack target, and each attacking organization has its unique attack chain preferences, personnel, and arsenal. This characteristic information of attacking organizations is of great significance for attacking organization identification. Therefore, it is necessary to model known attacking organizations based on attack chain information, and use attacking organization models to predict the attribution of future security incident attack chains, providing a basis and support for cybersecurity attribution and accountability. Summary of the Invention

[0004] This invention proposes a method and system for attack chain construction and attack attribution based on text-based threat intelligence. Based on text-based threat intelligence provided by security organizations such as network security service providers, it extracts attack chains from security incidents and establishes attack path models for attack organizations, thereby enabling the attribution and prediction of attack chains for future security incidents.

[0005] To achieve the above objectives, this invention proposes a method for constructing attack chains and tracing attack sources based on text-based threat intelligence, comprising the following steps:

[0006] Collect text-based threat intelligence on attack events from known attack organizations, and perform data preprocessing on the collected text-based threat intelligence to make it meet the needs of subsequent analysis;

[0007] By utilizing the collected text-based threat intelligence, network security entities and relationships are extracted to form network security event description triples;

[0008] For the extracted network security event description triples, a security event attack chain is constructed based on the timeline and logical order;

[0009] We train an attack organization feature model using the attack chain of known attack organizations' security incidents, and then use the trained attack organization feature model to predict the source of attack events from unknown origins.

[0010] Furthermore, the data preprocessing operations include data completion, deduplication, and correction.

[0011] Furthermore, the process of constructing a network security event description triplet includes the following steps: based on the collected and processed text-based threat intelligence, using natural language processing technologies such as BERT and bidirectional long short-term neural networks, network security entities such as attack organizations, IPs, attack tools, and attack methods are extracted from the threat intelligence; and using remote supervised learning methods, the relationships between entities are extracted to form a network security event description triplet in the format of entity-relationship-entity.

[0012] Furthermore, the construction of the security event attack chain includes the following steps: based on manually defined attack sub-events, the network security event description triplet is divided into different network security sub-events, and based on the timeline and text description logic, the network security sub-events are arranged according to the actual occurrence order and causal logic to form an attack chain description of this network security event.

[0013] Furthermore, the step of training an attack organization feature model using the security event attack chain of a known attack organization includes the following steps: forming a Bayesian conditional probability network from network security sub-event categories, inputting the security event attack chain of a known attack organization, and training the conditional probabilities of each node as the attack feature profile of the attack organization, i.e., the attack organization feature model.

[0014] Furthermore, the source tracing and prediction of attack events from unknown sources based on the trained attack organization feature model includes: for text-based threat intelligence of attack events from unknown sources, converting it into a security event attack chain through the aforementioned steps, matching it with the attack feature profiles of known attack organizations, and selecting the attack organization with the highest confidence as the source tracing and prediction result.

[0015] Based on the same inventive concept, this invention also provides an attack chain construction and attack attribution system based on text-based threat intelligence, employing the above-mentioned method, comprising:

[0016] The data acquisition and preprocessing module is used to collect text-based threat intelligence from known attack organizations and to perform data preprocessing on the collected text-based threat intelligence to make it meet the needs of subsequent model analysis.

[0017] The entity and relation extraction module is used to extract network security entities and relations to form network security event description triples;

[0018] The attack chain generation module is used to summarize network security sub-events and construct security event attack chains based on timelines and logical order to describe the security event process.

[0019] The feature model training and source tracing prediction module is used to train the feature model of the attack organization based on the generated security event attack chain, and to perform source tracing prediction of network security events with unknown attack sources.

[0020] This invention starts with text-based threat intelligence, extracts the attack chain of cybersecurity incidents, describes the overall process and internal logic of security incidents, and combines it with known attack sources to characterize attack feature models for attacking organizations, effectively serving cybersecurity personnel in tracing and holding accountable cybersecurity incidents from unknown sources. Attached Figure Description

[0021] Figure 1 This is a flowchart of an attack chain construction and attack attribution method based on text-based threat intelligence according to the present invention. Detailed Implementation

[0022] To make the technical solution of the present invention more obvious and understandable, specific embodiments are provided and described in detail below with reference to the accompanying drawings.

[0023] This invention provides a method for constructing attack chains and tracing attack attribution based on text-based threat intelligence, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0024] Step 1: Data Acquisition and Preprocessing

[0025] Text-based threat intelligence for security incidents includes the sequence of actions, the impact on the attacked system, and indicators of damage (IOC). Among these, the description of the attacker's action sequence and targets in this particular attack is of greater value for subsequent analysis and modeling. The data preprocessing process removes irrelevant data, duplicate data, and anomalous symbols from the text-based threat intelligence. It also supplements incomplete information based on existing cybersecurity databases to address inconsistencies and improve the quality, completeness, and accuracy of the collected data. Data cleaning includes, but is not limited to, the following:

[0026] 1) It should support the handling of missing information in the collected data, such as information that is not detailed or marked as unknown in threat intelligence, but can be found in existing network security databases;

[0027] 2) It should support the handling of abnormal information in the collected data. Abnormal information includes duplicate information and error information. When merging text-based threat intelligence from multiple sources, duplicate information may occur. Error information may be due to the data collection module being inadequate, resulting in the acquisition of threat intelligence containing incorrect characters or garbled text.

[0028] 3) It should support the cleaning of non-required information in the collected data. In order to avoid the redundancy of information from interfering with the accuracy of the algorithm, non-required information in the collected data needs to be deleted.

[0029] Step 2: Extract network security incident description triples

[0030] This embodiment is based on relevant standards and specifications in the field of cybersecurity, such as the ISO / IEC 27000 information security management system standard family, the cybersecurity threat information format specification GB / T36643-2018, information security technical terminology GB / T25069-2019, information security technology network attack definition and description specification GB / T37027-2018, and some public security industry standards, thereby determining the core ontology, concepts and terminology in the field of cybersecurity, establishing a cybersecurity knowledge ontology model, and clarifying the attribute set.

[0031] For example:

[0032] (1) Network assets: including various hardware devices, software devices, network environments, virtual personnel, etc. in cyberspace.

[0033] (2) Vulnerability: This includes vulnerability vulnerability and weakness vulnerability, such as vulnerabilities, system configuration, and protection software.

[0034] (3) Cyberattacks: This includes attackers, attack methods, tools used, attack events, and attack consequences. Attackers include individuals, groups, or hacker organizations; attack methods include the means used in the attack, such as denial-of-service attacks, backdoor attacks, vulnerability exploits, network scanning and eavesdropping, phishing, interference incidents, advanced threat incidents, and other cyberattack incidents; tools used include normal software and malicious software.

[0035] The specific implementation methods for extracting network security incident description triples include:

[0036] 1) Network security entity extraction

[0037] First, the input text-based threat intelligence text is segmented into a character sequence S = {s1, s2, ..., s...}.n The BERT model is used for character vector embedding, with proper nouns and English words embedded as a single character. To obtain richer features, the input E of the BERT model... i This is the sum of the vectors of a character across the three dimensions of symbol embedding, fragment embedding, and position embedding. The model is pre-trained using a masked language model approach. During training, characters in the training sequence are replaced with masked markers with a 15% probability. The original word at the masked marker position is then predicted. Of the parts covered by the masked markers, 80% are replaced with the "[MASK]" marker, and 20% are replaced with random other characters. This ensures that BERT remains sensitive to all characters and improves vector embedding efficiency.

[0038] The next step is to process the character vector sequence X = {x1, x2, ..., x...} output by BERT. n The input X is used as the input to the BiLSTM neural network. BiLSTM consists of two LSTM networks, a forward LSTM network and a backward LSTM network. During feature extraction, X is simultaneously input into both the forward and backward LSTM networks, and the hidden layer vector sequence at the current time step is calculated and concatenated. Then, the formula o is applied to this vector. t =tanh(W h h t +b o ) Perform activation function calculation, where o t As the output at the current moment, h t Let W be the hidden layer vector at the current time step. t and b o These are the weight matrix and bias term of the output gate, respectively.

[0039] Finally, entity information in the natural language text is manually labeled, and the order of appearance of the labeled symbols is constrained by a conditional random field. The output of the BiLSTM neural network is used as input. For each predicted sequence of a character sequence, its score is equal to the sum of the predicted score of the character itself and the transition score between character sequences. Finally, the predicted sequence with the highest score is selected as the model output.

[0040] 2) Network security event relationship extraction

[0041] First, based on actual needs, define the types of relationships to be extracted and the definitions of head and tail entities. Then, construct a PCNN (Piecewise Convolutional Neural Network) relationship extraction model based on remote supervised learning to extract relationships from the entire sample. Specific implementation methods include:

[0042] First, a small number of samples are manually labeled. Then, for a large number of unlabeled samples, all labeled entity pairs are marked as manually labeled relations, and entity pairs with the same relation are placed into a bag of sentences. Subsequent model training uses the bag of sentences as the basic unit. Then, for each entity pair in the bag of sentences, the sentence is segmented into three segments at the positions of the two entities. Each segment is fed into a CNN network for training as a separate sentence. The input of the CNN network is the concatenation vector of the character embedding vector and the relative distance between the character and the two entities. The output is the confidence score of the sentence in each relation category. Finally, the sentence with the highest confidence score is selected as the feature of the bag of sentences, and this is used as the criterion for determining the relation between the entity pairs.

[0043] Step 3: Construct a security incident attack chain

[0044] A security incident attack chain consists of a series of attack behaviors, used to describe the sequence and logic of the attacker's actions in this attack event. This embodiment limits the attack behaviors to fixed types of network attack techniques, including unauthorized modification, brute-force attacks, remote control, data theft, DoS attacks, scanning and probing, website malware injection, etc. (these are network security sub-events). It also adds attack start and end markers as the head and tail of the attack chain to extract characteristic information of the initial attack and the final objective. Specific implementation methods include:

[0045] First, experts were organized to build an attack behavior determination rule base based on the attack behavior flow, and then an attack behavior determination model was constructed on this basis to achieve the abstract generalization and extraction of network security incident description triples. The method for "building the attack behavior determination rule base" is as follows: manually analyzing the attack behavior flow and constructing the correspondence between entity relationships and attack behaviors. The method for "building the attack behavior determination model" is as follows: using the constructed attack behavior determination rule base, retrieving entities and relationships that conform to the rules in the entity instance graph, and thus labeling them as the corresponding attack behaviors.

[0046] Then, the attack behaviors are sorted based on timeline and logical order. The specific method is as follows: for each identified attack behavior, the time information of the paragraph in which it occurs is retrieved and marked as the time of occurrence of the attack behavior, and sorted according to the time order; if there is no explicit time information, logical words (first, second, then, finally, etc.) in the natural language text are manually marked, and a logical order is manually set, and the attack behaviors are sorted based on this.

[0047] Step 4: Feature Model Training and Source Prediction

[0048] Based on the attack chain information of the known attack source (attack organization) obtained in step 3, it is input into the Bayesian network for training. Specifically, a matrix M is maintained.m*m Where m represents the known types of attack behaviors, and each element M in the matrix ij The probability that the next attack after attack action i will be attack action j is calculated as follows:

[0049]

[0050] For text-based threat intelligence regarding subsequent cybersecurity incidents from unknown attack sources, the process first goes through steps 1 to 3 to obtain the attack chain information X = {x1, x2, ..., x}. t Then, based on the existing Bayesian network, its confidence level for each attacking organization is calculated. The calculation method is as follows:

[0051]

[0052] Where t represents the attack chain length and k represents the iteration variable.

[0053] Finally, the attacking organization with the highest confidence level was selected as the source tracing prediction result.

[0054] Another embodiment of the present invention provides an attack chain construction and attack attribution system based on text-based threat intelligence using the above method, comprising:

[0055] The data acquisition and preprocessing module is used to collect text-based threat intelligence from known attack organizations and to perform data preprocessing on the collected text-based threat intelligence to make it meet the needs of subsequent model analysis.

[0056] The entity and relation extraction module is used to extract network security entities and relations to form network security event description triples;

[0057] The attack chain generation module is used to summarize network security sub-events and construct security event attack chains based on timelines and logical order to describe the security event process.

[0058] The feature model training and source tracing prediction module is used to train the feature model of the attack organization based on the generated security event attack chain, and to perform source tracing prediction of network security events with unknown attack sources.

[0059] For the specific implementation process of each module, please refer to the description of the method of the present invention above.

[0060] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0061] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0062] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A method for constructing attack chains and tracing attack origins based on text-based threat intelligence, characterized in that, This method is used to predict the origin of attack groups, and includes the following steps: Collect text-based threat intelligence on attack events from known attack organizations, and perform data preprocessing on the collected text-based threat intelligence to make it meet the needs of subsequent analysis; By utilizing the collected text-based threat intelligence, network security entities and relationships are extracted to form network security event description triples; For the extracted network security event description triples, a security event attack chain is constructed based on the timeline and logical order; The attack organization feature model is trained using the security event attack chain of known attack organizations, and the source tracing and prediction of attack events from unknown sources is performed based on the trained attack organization feature model. The steps for constructing a security incident attack chain include: first, organizing experts to construct an attack behavior judgment rule base based on the attack behavior process, and then constructing an attack behavior judgment model on this basis, thereby realizing the abstract generalization and extraction of the network security incident description triple, and then sorting the attack behaviors based on the timeline and logical order. The construction of the attack behavior determination rule base includes: manually analyzing the attack behavior process and constructing the correspondence between entity relationships and attack behaviors; the construction of the attack behavior determination model includes: using the constructed attack behavior determination rule base, retrieving entities and relationships that conform to the rules in the entity instance graph, and thus marking them as the corresponding attack behaviors. The method of sorting attack behaviors based on timeline and logical order includes: for each identified attack behavior, retrieving the time information of the paragraph in which it is located, marking it as the time of occurrence of the attack behavior, and sorting it according to the time order; if there is no explicit time information, then manually marking logical words in the natural language text, manually setting the logical order, and sorting the attack behaviors based on this. The method of training an attack organization feature model using the security event attack chain of a known attack organization includes: forming a Bayesian conditional probability network from network security sub-event categories, inputting the security event attack chain of a known attack organization, and training the conditional probability of each node as the attack feature profile of the attack organization, i.e., the attack organization feature model. The method of tracing and predicting the source of attacks from unknown sources based on the trained attack organization feature model includes: transforming text-based threat intelligence of attacks from unknown sources into security event attack chains, matching them with attack feature profiles of known attack organizations, and selecting the attack organization with the highest confidence as the source prediction result. The process of training an attack organization feature model using the security event attack chain of a known attack organization, and then performing source tracing and prediction of attack events from unknown sources based on the trained attack organization feature model, includes: Maintain a matrix Where m represents the known types of attack behaviors, and each element M in the matrix... ij The probability that the next attack after attack action i will be attack action j is calculated as follows: For text-based threat intelligence on cybersecurity incidents with unknown attack sources, the attack chain information X = {x1, x2, ..., x} for this attack event is first obtained. t Then, the confidence level of each attacking organization is calculated using the following method: Where t represents the attack chain length and k represents the iteration variable; Finally, the attacking organization with the highest confidence level was selected as the source tracing prediction result.

2. The method as described in claim 1, characterized in that, The text-based threat intelligence includes the sequence of actions, the impact on the attacked system, and the damage indicators. Among these, the description of the attacker's attack sequence and targets in this attack event is of greater value for subsequent analysis and modeling. The data preprocessing operation removes irrelevant data, duplicate data, and abnormal symbols from text-based threat intelligence, and completes some incomplete information based on existing network security databases to solve the problem of information inconsistency, thereby improving the quality of collected data and ensuring the integrity and accuracy of collected data.

3. The method as described in claim 1, characterized in that, The process of extracting cybersecurity entities and relationships from collected text-based threat intelligence to form cybersecurity event description triplets includes: extracting cybersecurity entities using natural language processing technology based on collected text-based threat intelligence, and extracting relationships between entities using remote supervised learning to form cybersecurity event description triplets in entity-relationship-entity format.

4. The method as described in claim 3, characterized in that, The extraction of network security entities using natural language processing technology involves employing a BERT-BiLSTM-CRF network, and its steps include: The input text-based threat intelligence text is segmented into character sequences S = {s1, s2, ..., s}. n The BERT model is used to embed characters into vectors, with proper nouns and English words being embedded as a single character. The character vector sequence X = {x1, x2, ..., x...} output by BERT is... n } as input to the BiLSTM neural network; Entity information in natural language text is manually labeled, and the order of appearance of the labeled symbols is constrained by a conditional random field. The output of a BiLSTM neural network is used as input. For each predicted sequence of a character sequence, its score is equal to the sum of the predicted score of the character itself and the transition score between character sequences. Finally, the predicted sequence with the highest score is selected as the model output.

5. The method as described in claim 3, characterized in that, The steps for extracting inter-entity relationships using remote supervised learning include: First, a small number of samples are manually labeled. Then, for a large number of unlabeled samples, all labeled entity pairs are labeled as manually labeled relations, and entity pairs with the same relations are put into a bag of sentences. Subsequent model training uses the bag of sentences as the basic unit. Then, for each entity pair in the bag of sentences, the sentence is divided into three segments at the positions of the two entities. Each segment is fed into a CNN network for training as a separate sentence. The input of the CNN network is the concatenation vector of the character embedding vector and the relative distance between the character and the two entities. The output is the confidence score of the sentence in each relation category. Finally, the sentence with the highest confidence score is selected as the feature of the bag of sentences and used as the criterion for determining entity relations.

6. A system for constructing attack chains and tracing attack sources based on text-based threat intelligence, employing the method described in any one of claims 1 to 5, characterized in that, include: The data acquisition and preprocessing module is used to collect text-based threat intelligence from known attack organizations and to perform data preprocessing on the collected text-based threat intelligence to make it meet the needs of subsequent model analysis. The entity and relation extraction module is used to extract network security entities and relations to form network security event description triples; The attack chain generation module is used to summarize network security sub-events and construct security event attack chains based on timelines and logical order to describe the security event process. The feature model training and source tracing prediction module is used to train the feature model of the attack organization based on the generated security event attack chain, and to perform source tracing prediction of network security events with unknown attack sources.

7. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method described in any one of claims 1 to 5.

Images