A method for detecting APT attack sequences based on subgraph matching

Abstract

The present invention discloses an APT attack sequence detection method based on subgraph matching, comprising the following steps: 101) a data preprocessing step, 102) a traceability graph construction step, 103) a model training sample acquisition step, and 104) a model forming and prediction step. The present invention provides an APT attack sequence detection method based on subgraph matching, which utilizes threat intelligence information and integrates log entries to improve prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the field of computer security, and more specifically, to an APT attack sequence detection method based on subgraph matching. Background Art

[0002] Advanced persistent threat (APT) is used to describe the attack activities of intruders who establish a long-term illegal presence on the network to mine highly sensitive data. With the continuous development of global network informatization, the threat posed by APT, a new type of complex targeted network attack, is becoming increasingly prominent. APT attacks can not only steal the target's data, but also remain hidden throughout the attack process.

[0003] my country is also one of the long-term victims of APT attacks. In 2008, the network dedicated to the Chinese military was attacked by hackers from the US Department of Defense. The attackers implanted backdoors into the network to steal intelligence. In recent years, these APT attacks have used 0-day vulnerabilities, automated attack frameworks, artificial intelligence and other technologies to enhance the efficiency and concealment of attacks, making detection, disposal and tracking more difficult. Therefore, research on defense work against APT attacks is very important.

[0004] At present, the detection ideas of APT attack detection methods are mainly based on host data and network data to identify and discover APT attack behaviors. Host-based detection methods mainly monitor system calls, network access, file reading and writing, process creation, memory modification and other behaviors, and use static analysis and dynamic analysis methods to detect malicious programs and prevent APT attacks. Early pattern matching-based methods can effectively identify known attacks, but the recognition rate of unknown attacks is low, and the rule base needs to be updated regularly. At present, more detection methods based on behavioral modeling are used. In recent years, with the rise of data mining and machine learning technologies, researchers have proposed a variety of detection technologies based on machine learning and data mining.

[0005] For example, the patent with publication number CN105262726A discloses an APT attack detection method based on big data behavior sequence analysis, including: detecting the user's operation behavior and building a user operation behavior sequence library; optimizing the user operation behavior sequence library; extracting the user operation behavior sequence based on the big data method; matching and analyzing the extracted user operation behavior sequence with the behavior sequence in the user operation behavior sequence library, and outputting the matching result; analyzing the matching result and outputting the threat behavior. The present invention can improve the accuracy of threat detection, thereby building a safe and efficient network active defense system, and effectively overcoming the threat posed by APT attacks to network security.

[0006] The patent with the publication number CN111447212A discloses a method for generating and detecting APT attack sequences based on GAN. This invention aims at the problems that there are still problems in the detection direction of APT attacks, such as the inability to detect long sequence attacks, poor historical information correlation, and poor unknown attack detection capabilities. In order to effectively detect APT attacks and expand the APT attack dataset, a method for generating and detecting APT attack sequences based on GAN is proposed. The present invention adopts the LSTM network in the form of a generative adversarial network to add steps to associate the context, memorize the historical records of network traffic, and has a good detection capability for time series data; through the generation model, random noise can be used to generate data that meets the characteristics of APT attacks, expanding the existing APT attack dataset.

[0007] The patent with publication number CN112333195A discloses an APT attack scene restoration detection method and system based on multi-source log association analysis. The detection method includes collecting multi-source logs of the host, setting new feature parameters, using relationship vectors to associate all log entries, treating all log entries as nodes in the network, treating the relationship between log entries as edges between nodes, constructing an undirected, weighted complex network graph, clustering using a label propagation algorithm, and identifying events; then arranging logs and events into long sequences in chronological order, mining the logical relationship and time relationship between events, generating an initial sub-partition graph and continuously optimizing it to obtain a scene graph; then learning the vector expression of the vertices and edges of the scene graph, clustering, detecting whether the new edges and vertices of the updated scene graph are abnormal, and after completing the detection, updating the clustering situation to prepare for subsequent detection. The present invention can comprehensively and accurately restore the attack scene, prevent high false alarm rates and missed fish, and efficiently detect APT attacks.

[0008] However, the technical application effect of the first patent is very dependent on the behavior sequence library, manual intervention and maintenance are time-consuming, and the detection effect of long-sequence attack behaviors is poor. The second patent uses GAN to expand APT attack data and uses LSTM to enhance the detection ability of long sequences, but does not use threat intelligence information. The third patent improves the effect of the detection model based on multi-source log correlation analysis and clustering methods, but also does not use threat intelligence information. Summary of the invention

[0009] The present invention overcomes the shortcomings of the prior art and provides an APT attack sequence detection method based on subgraph matching, which utilizes threat intelligence information and integrates log entries to improve prediction accuracy.

[0010] The technical solution of the present invention is as follows:

[0011] A method for detecting APT attack sequences based on subgraph matching includes the following steps:

[0012] 101) Data preprocessing step: collecting data such as system security alarm logs, browser logs, and DNS logs in the host, uniformly collecting and processing different types of log data, and sorting each record by time;

[0013] 102) Constructing a traceability graph step: constructing a traceability graph based on the data processed in step 101), and optimizing the structure of the traceability graph by graph pruning and isolated vertex deletion methods;

[0014] 103) Model training sample acquisition step: In the model training sample generation stage, the attack behavior query graph is constructed by extracting threat intelligence information, and the number of attack sequence samples is expanded by using methods such as subgraph matching and neighborhood graph extraction;

[0015] Among them, the attack sequence sample generation method based on subgraph matching uses a subgraph matching algorithm to find subgraph structures similar to the extracted attack subgraph in the traceability graph generated in step 102), and marks the key entities in these subgraph structures as attack entities;

[0016] Specifically, an attack sequence sample is defined as a series of attack behaviors containing attack entities sorted by time. The attack sequence generation process is as follows:

[0017] 301) obtaining a set of all attack entities from the traceability graph, and constructing an entity subset containing two or more entities, wherein the attack entity includes a simulated APT attack entity and an attack entity marked by subgraph matching;

[0018] 302) for each attack entity subset, extracting a neighborhood graph of the attack entity subset to capture all entities that have an association relationship with the attack entity;

[0019] 303) acquiring attack entities and behaviors sorted by timestamp from the constructed neighborhood graph, and if the source node or the target node is an attack entity, the behavior is marked as an attack behavior;

[0020] 304) converting the extracted entities and their behaviors into an attack sequence by sorting by timestamp;

[0021] 104) Model forming and prediction step: Based on the model training samples generated in step 103), a deep sequence network is used to train an APT attack sequence detection model, and indicators such as the model's precision, recall rate, and F1 value are evaluated; the trained model is used to predict whether there is APT attack activity in the log data.

[0022] Furthermore, the collection and processing are unified into data including the following information: ID, time, source IP, destination IP, source port, destination port, behavior category, PID, PPID, process name, object name, domain name and WEB object information.

[0023] Further, the subgraph structure similar to the extracted attack subgraph in step 103) specifically adopts the attack query graph extraction based on threat intelligence;

[0024] Threat intelligence uses text or structured / semi-structured language to describe the overall attack process and the relationship between the IOCs related to the attack; based on threat intelligence and using NLP-related technologies such as entity extraction, relationship extraction, and disambiguation, the query subgraph of the attack entity in the threat intelligence is automatically extracted to form an automatically updated threat subgraph library.

[0025] Furthermore, it also includes a non-attack sequence, which is defined as a series of attack behaviors containing non-attack entities sorted in time; the non-attack sequence also obtains all non-attack entities in the traceability graph, and then extracts their sequences based on all non-attack entities, and extracts a non-attack sequence by adding a non-attack entity in each attack subset.

[0026] Furthermore, the non-attack sequence includes a sample balancing method, which calculates the similarity between non-attack sequences to eliminate any one of the two sequences with higher similarity, and processes the non-attack sequence samples in such an under-sampling manner, so that the distribution of training samples is more balanced.

[0027] Furthermore, the detection process based on the APT attack sequence detection model in step 104) is as follows:

[0028] 401) Collect various log data in real time and process and optimize them to generate traceability diagrams;

[0029] 402) Based on the threat subgraph library and the security alarm log, the sequence to be identified of the suspected attack behavior is extracted by using the subgraph matching and domain graph extraction method;

[0030] 403) Based on the trained APT attack sequence detection model, identify whether there is APT attack activity in the sequence to be identified, and if so, issue an alarm in real time for security personnel to investigate.

[0031] The advantages of the present invention are:

[0032] The present invention mainly obtains relevant log data by simulating APT attacks on the host in the training phase, and trains an APT attack sequence detection model based on the log data; then in the prediction phase, the trained model is used to predict whether there is APT attack activity in the host log data.

[0033] Specifically, the present invention uniformly collects and processes system security alerts, browsers, DNS and other logs, builds a traceability graph based on the processed log data, and optimizes the structure of the traceability graph. In the model training sample generation stage, the attack behavior query graph is constructed by extracting threat intelligence information, and the number of attack samples is expanded by using methods such as subgraph matching and neighborhood graph extraction. Finally, the APT attack sequence detection model is trained based on the deep sequence network to solve the above technical problems and improve the APT attack behavior detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The figure is a flow chart of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Parts not specifically described in this solution can be implemented using conventional technical means.

[0036] like Figure 1 The method for detecting an APT attack sequence based on subgraph matching comprises the following steps:

[0037] 101) Data preprocessing step: Collect system security alarm logs, browser logs, DNS logs and other data in the host, collect and process different types of log data in a unified manner, and sort each record by time. Collect and process the data in a unified manner to include the following information: ID, time, source IP, destination IP, source port, destination port, behavior category, PID, PPID, process name, object name, domain name, and WEB object. Collect all log data and process them into a unified data format to facilitate the subsequent construction of a traceability diagram.

[0038] 102) Constructing the traceability graph step: Based on the data extracted after processing in step 101), the entities such as domain names, IP addresses, WEB objects and their associations in the generated log data are used as attributes of entities and edges to construct a directed traceability graph. The optimization of the traceability graph mainly reduces the number of some nodes and edges through methods such as graph pruning and isolated vertex deletion to optimize the structure of the traceability graph. Specifically, one graph pruning method is to eliminate all nodes and edges that cannot be reached from the attack node; another way is to optimize the graph structure with repeated edges.

[0039] 103) Model training sample acquisition steps: In the model training sample generation stage, the attack behavior query graph is constructed by extracting threat intelligence information, and the number of attack sequence samples is expanded by using methods such as subgraph matching and neighborhood graph extraction.

[0040] The extracted attack subgraph has a similar subgraph structure, which specifically uses the attack query graph extraction based on threat intelligence. That is, the attack query graph extraction based on threat intelligence uses NLP related technologies such as information extraction to extract the query subgraph of the attack entity in the threat intelligence.

[0041] Specific threat intelligence generally uses text or structured / semi-structured language to describe the overall attack process and the relationship between IOCs related to the attack. Based on threat intelligence, and using NLP related technologies such as entity extraction, relationship extraction, and disambiguation, the query subgraph of the attack entity in the threat intelligence is automatically extracted to form an automatically updated threat subgraph library.

[0042] Among them, the attack sequence sample generation method based on subgraph matching uses a subgraph matching algorithm to find subgraph structures similar to the extracted attack subgraph in the traceability graph generated in step 102), and marks the key entities in these subgraph structures as attack entities.

[0043] Specifically, an attack sequence sample is defined as a series of attack behaviors containing attack entities sorted by time. The attack sequence generation process is as follows:

[0044] 301) Obtain a set of all attack entities from the traceability graph, and construct an entity subset containing two or more entities, wherein the attack entity includes a simulated APT attack entity and an attack entity marked by subgraph matching.

[0045] 302) For each attack entity subset, extract a neighborhood graph of the attack entity subset to capture all entities that have an association relationship with the attack entity.

[0046] 303) Obtain attack entities and behaviors sorted by timestamp from the constructed neighborhood graph. If the source node or the target node is an attack entity, the behavior is marked as an attack behavior.

[0047] 304) The extracted entities and their behaviors are converted into an attack sequence sorted by timestamp.

[0048] It also includes non-attack sequences, which are defined as a series of attack behaviors containing non-attack entities sorted by time. The construction process of non-attack sequences is similar to that of attack sequences. Non-attack sequences also obtain all non-attack entities in the traceability graph, and then extract their sequences based on all non-attack entities. However, due to the huge number of non-attack entities, if this process is implemented according to the attack sequence generation method, the number of non-attack sequences obtained will far exceed the number of attack sequences, which will eventually mislead the training of the model. In order to enable the model to better distinguish the types of activity sequences, a non-attack sequence is extracted by adding a non-attack entity to each attack subset.

[0049] Among them, the non-attack sequence includes a sample balancing method, which calculates the similarity between non-attack sequences to eliminate any one of the two sequences with higher similarity. By processing the non-attack sequence samples in this under-sampling manner, the distribution of training samples is more balanced, thereby improving the effect of model reasoning.

[0050] 104) Model formation and prediction step: Based on the model training samples generated in step 103), a deep sequence network is used to train an APT attack sequence detection model, and the model's precision, recall, F1 value and other indicators are evaluated. The trained model is used to predict whether there is APT attack activity in the log data.

[0051] The detection process based on the APT attack sequence detection model is as follows:

[0052] 401) Collect various log data in real time and process and optimize them uniformly to generate a traceability graph.

[0053] 402) Based on the threat sub-graph library and security alarm log, the sub-graph matching and domain graph extraction methods are used to extract the sequences to be identified of suspected attack behaviors.

[0054] 403) Based on the trained APT attack sequence detection model, identify whether there is APT attack activity in the sequence to be identified, and if so, issue an alarm in real time for security personnel to investigate.

[0055] In summary, this solution combines threat intelligence information into the APT detection process by introducing subgraph matching technology, enabling the model to detect some of the latest APT attacks, which improves the generalization ability of the model to a certain extent; at the same time, this process also expands the number of training samples, thereby improving the recognition effect of the detection model.

[0056] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the concept of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for detecting APT attack sequences based on subgraph matching, characterized in that: The steps include: 101) Data preprocessing step: Collect system security alarm logs, browser logs, and DNS log data from the host, collect and process different types of log data in a unified manner, and sort each record by time; 102) Step of constructing a traceability graph: construct a traceability graph based on the data processed in step 101), and optimize the structure of the traceability graph by graph pruning and isolated vertex deletion methods; 103) Model training sample acquisition step: In the model training sample generation stage, the attack behavior query graph is constructed by extracting threat intelligence information, and the number of attack sequence samples is expanded by using subgraph matching and neighborhood graph extraction methods; The attack sequence sample generation method based on subgraph matching uses a subgraph matching algorithm to find subgraph structures similar to the extracted attack subgraph in the traceability graph generated in step 102), and marks key entities in these subgraph structures as attack entities; Specifically, an attack sequence sample is defined as a series of attack behaviors containing attack entities sorted by time. The attack sequence generation process is as follows: 301) Obtain a set of all attack entities from the traceability graph, and construct an entity subset containing two or more entities, where the attack entity includes a simulated APT attack entity and an attack entity annotated by subgraph matching; 302) For each attack entity subset, extract a neighborhood graph of the attack entity subset to capture all entities that have an association relationship with the attack entity; 303) Obtain attack entities and behaviors sorted by timestamp from the constructed neighborhood graph. If the source node or the target node is an attack entity, the behavior is marked as an attack behavior; 304) Convert the extracted entities and their behaviors into an attack sequence sorted by timestamp; 104) Model formation and prediction step: Based on the model training samples generated in step 103), a deep sequence network is used to train an APT attack sequence detection model, and the model is tested to evaluate the model's precision, recall, and F1 value indicators; the trained model is used to predict whether there is APT attack activity in the log data.

2. According to claim 1, a subgraph matching-based APT attack sequence detection method is characterized in that: The collected and processed information is unified to include the following information: ID, time, source IP, destination IP, source port, destination port, behavior category, PID, PPID, process name, object name, domain name and WEB object information data.

3. The APT attack sequence detection method based on subgraph matching according to claim 1 is characterized in that: The subgraph structure similar to the extracted attack subgraph in step 103) is specifically extracted by attack query graph based on threat intelligence; Threat intelligence uses text or structured / semi-structured language to describe the overall attack process and the relationship between IOCs related to the attack; Based on threat intelligence and using NLP related technologies such as entity extraction, relationship extraction, and disambiguation, the query subgraph of the attack entity in the threat intelligence is automatically extracted to form an automatically updated threat subgraph structure.

4. The APT attack sequence detection method based on subgraph matching according to claim 1 is characterized in that: It also includes non-attack sequences, which are defined as a series of attack behaviors containing non-attack entities sorted in time. The non-attack sequence also obtains all non-attack entities in the traceability graph, and then extracts their sequences based on all non-attack entities, and extracts a non-attack sequence by adding a non-attack entity in each attack subset.

5. The APT attack sequence detection method based on subgraph matching according to claim 4 is characterized in that: The non-attack sequence includes a sample balancing method, which calculates the similarity between non-attack sequences to eliminate any one of the two sequences with higher similarity. By processing the non-attack sequence samples in such an under-sampling manner, the distribution of training samples is more balanced.

6. The APT attack sequence detection method based on subgraph matching according to claim 1 is characterized in that: The detection process based on the APT attack sequence detection model in step 104) is as follows: 401) Collect various log data in real time and process and optimize them to generate traceability graphs; 402) Based on the threat subgraph structure and security alarm log, the subgraph matching and domain graph extraction methods are used to extract the sequences to be identified of suspected attack behaviors; 403) Based on the trained APT attack sequence detection model, identify whether there is APT attack activity in the sequence to be identified. If so, issue an alarm in real time for security personnel to investigate.

Images