APT anomaly detection method fusing dual-channel GNN thought

Abstract

The invention discloses an APT anomaly detection method fusing a dual-channel GNN thought, and belongs to the technical field of network security and machine learning. The method comprises the following steps: 1, data processing: converting an acquired benign system log into an origin graph; 2, feature extraction and fusion: extracting data features through a behavior sensing channel and a static sensing channel to obtain comprehensive feature representation of each node; and 3, anomaly detection: converting the fused features into anomaly scores by using MLP, and performing final classification decision. According to the method, static characteristics and dynamic behavior characteristics can be analyzed at the same time, and the benign behavior mode of the user can be more comprehensively understood; the graph neural network can capture complex interaction and communication modes among entities, and is crucial for identifying key behaviors and modes in a data set; the design allows real-time analysis of system behaviors and quick response to potential APT attacks; meanwhile, self-optimization and updating can be carried out according to new detection feedback, and the robustness and adaptability of detection are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security and machine learning, and specifically relates to an APT anomaly detection method integrating the dual-channel GNN idea. Background Art

[0002] In the field of network security, APT (Advanced Persistent Threat) attacks pose a severe challenge to the security protection of organizations due to their complexity, concealment and long-term latency. These attacks are often carried out by organized groups with the purpose of quietly invading critical networks or systems for a long time to steal sensitive data or cause damage. Because traditional security tools mainly focus on known threats and have limited ability to identify unknown or variant attack patterns, existing security measures are always difficult to detect APT attacks or are easily bypassed by hidden and complex APT attacks.

[0003] The difficulty in detecting APT attacks also leads to a scarcity of samples. Currently, the publicly available APT data samples are not sufficient to train a sufficiently accurate and robust learning model. Therefore, most researchers use benign data sets to perform anomaly detection, identify behavior patterns that are significantly different from normal behavior, and define them as abnormal intrusion behaviors.

[0004] Although anomaly detection technology has made progress driven by machine learning and artificial intelligence, especially graph neural networks (GNNs), which have shown great potential in analyzing complex relationships between entities, existing methods still have many limitations, including performance bottlenecks when processing big data, difficulties in feature extraction, and high false alarm rates. Many methods have high computational costs and lack of real-time performance when processing large-scale data sets. Feature engineering is crucial to detection performance, but manual feature selection is time-consuming and may miss key information. Existing anomaly detection systems may be overly sensitive to small changes in normal behavior, resulting in high false alarm rates.

[0005] In response to the above problems, how to design an automated feature extraction method to process the raw data and, on this basis, conduct real-time anomaly detection with lower false alarms and higher accuracy is an issue that needs to be addressed urgently. Summary of the invention

[0006] In view of the above problems existing in the prior art, the object of the present invention is to provide an APT anomaly detection method integrating the dual-channel GNN idea.

[0007] The present invention provides the following technical solution: an APT anomaly detection method integrating the dual-channel GNN concept, comprising the following steps:

[0008] (1) Data processing: define the system behaviors involved in the underlying system log data, collect benign data and convert it into a provenance graph;

[0009] (2) Feature extraction and fusion: The behavioral and static features of benign data are extracted through the behavioral perception channel and the static perception channel, and then fused to obtain the comprehensive feature representation of each node;

[0010] (3) Anomaly detection: Collect log data as test data, input the test data into the trained multi-layer perceptron MLP, and determine whether it is an abnormal node based on the loss between input and output.

[0011] Furthermore, the specific process of (1) is as follows:

[0012] (1-1) Node type definition: Node types are defined based on the system behaviors involved in the log data, and relationship types are defined based on the interaction behaviors between nodes. A is defined as the node type set and R is defined as the relationship type set.

[0013] (1-2) Data cleaning: Collect good log data from multiple sources and conduct data review on the log data to identify potential problems with the data; handle missing values ​​by filling, deleting, or predicting, and ensure that all records follow a unified time format and data type;

[0014] (1-3) Constructing the provenance graph: Mapping the actual log data to the type definition, constructing the system behavior provenance graph G = (V, E), where G is a directed graph, V is a node set, E is a relationship set, any node in V corresponds to a node type in A, and any relationship in E corresponds to a relationship type in R.

[0015] Furthermore, the specific process of (2) is as follows:

[0016] (2-1) Behavioral feature extraction: define key behavioral features related to APT attacks, discover behavioral groups through cluster analysis algorithms, and use sequence analysis methods to identify behavioral patterns;

[0017] (2-2) Static feature extraction: The TransR method is used to encode nodes and extract static features from the collected data.

[0018] (2-3) Feature fusion: First, feature alignment is performed to integrate data from different data sources according to entity identifiers and synchronize data according to timestamps. Then, different weights are assigned to different features according to their importance, and the two are fused using a weighted tie to obtain the final representation h of node i. i .

[0019] Furthermore, the specific process of (3) is as follows:

[0020] (3-1) Model training: The node feature representation obtained in step (2) is input into the MLP model for training application; the softmax layer performs the anomaly detection task based on the final feature representation, and introduces weighted cross entropy loss to solve the problem of unbalanced classification;

[0021] (3-2) Abnormality judgment: In the test phase, the test data is processed according to steps (1-3) and step (2); the feature vector of the processed test data is input into the trained MLP model to obtain the abnormality score of the node. If it exceeds the threshold σ, the node is considered to be abnormal;

[0022] (3-3) Optimization and updating: Regularly collect and analyze the latest detection feedback from multiple sources, and update the feature set of the model through feature engineering; use these updated features to perform incremental learning on the model, while using transfer learning to retain existing knowledge.

[0023] By adopting the above technology, compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1) The present invention can analyze static features and dynamic behavior features at the same time, and multi-dimensional feature analysis helps to more comprehensively understand the user's benign behavior patterns;

[0025] 2) The present invention uses graph neural networks to capture complex interactions and communication patterns between entities, which is crucial for identifying key behaviors and patterns in data sets;

[0026] 3) The design of the method of the present invention allows real-time analysis of system behavior and can quickly respond to potential APT attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of an APT anomaly detection method integrating the dual-channel GNN concept of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments of the specification. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0030] Reference Figure 1 , an APT anomaly detection method integrating the dual-channel GNN idea, including the following steps:

[0031] (1) Data processing: Define the system behaviors involved in the underlying system log data and collect enough benign data to convert it into a provenance graph.

[0032] (2) Feature extraction and fusion: The behavioral features and static features of the data are extracted through the behavioral perception channel and the static perception channel, and then fused to obtain the comprehensive feature representation of each node.

[0033] (3) Anomaly detection: The test data is input into the trained multi-layer perceptron MLP, and the loss between the input and output is used to determine whether it is an abnormal node.

[0034] In step (1), the detailed steps of data processing are as follows:

[0035] (1-1) Node type definition: First, node types are defined based on the system behaviors involved in the log data, including processes, files, networks, node attributes, etc. Then, relationship types are defined based on the interaction behaviors between nodes, including the derivation relationship between processes, the read relationship between processes and files, the creation relationship between processes and files, the access relationship between processes and networks, the inclusion relationship between files and file attributes, etc. Finally, A is defined as the node type set and R is defined as the relationship type set.

[0036] (1-2) Data cleaning: After collecting data such as benign network traffic logs, system logs, and user behavior logs from multiple sources, data review is required to identify potential problems with the data. Missing values ​​are handled by filling, deleting, or predicting, and all records follow a unified time format and data type. Removing duplicate records and filtering noise are important steps to ensure data quality. Standardization ensures that different features are on the same scale and removes irrelevant or redundant information.

[0037] (1-3) Constructing the provenance graph: Mapping the actual log data to the type definition, constructing the system behavior provenance graph G = (V, E), where G is a directed graph, V is a node set, E is a relationship set, any node in V corresponds to a node type in A, and any relationship in E corresponds to a relationship type in R.

[0038] In step (2), the detailed steps of feature extraction and fusion are as follows:

[0039] (2-1) Behavioral feature extraction: First, define the key behavioral features related to APT attacks, such as unusual login frequency, abnormal network connection patterns, or suspicious file access behaviors. Use cluster analysis algorithms to discover behavioral groups; and use sequence analysis methods to identify behavioral patterns.

[0040] Use the Louvain method to discover behavioral groups: assign weights to each edge according to the interaction intensity (frequency) between entities to build a weighted graph, and initialize each node as its own community. Calculate the modularity Q gain of each node moving to the community where its neighboring node is located. If the movement can increase the modularity Q, the node is moved to that community. Repeat the modularity optimization steps for all nodes until the modularity cannot be increased by moving the nodes. Treat each community as a new node and repeat the modularity optimization and community cohesion steps until it is condensed into an entire community and the modularity cannot be further increased by moving the nodes. Finally, the algorithm traces back to the iteration with the largest modularity Q value and uses the community division of that iteration as the final result. The modularity calculation formula is:

[0041]

[0042] Among them A ij is an element in the adjacency matrix. If there is an edge between nodes i and j, then A ij is 1, otherwise it is 0. k i and k j are the degrees of nodes i and j respectively. m is the sum of the weights of all edges in the graph. i , c j ) is an indicator function, which is 1 if nodes i and j belong to the same community, otherwise it is 0. For node i considered to move, calculate its connection weight s within the current community i (i.e., the sum of the weights of the edges connected to other nodes in the community) and the connection weight k outside the current community i -s i Assume that node i moves to the community to which node j belongs, and calculate the sum of the connection weights s between node i and other nodes in the new community. ij The modularity gain, i.e. the difference between the modularity contribution before and after the movement, is:

[0043]

[0044] If ΔQ>0, then the move is beneficial.

[0045] Use Markov chains to identify behavior patterns: Define the state space of behavior analysis as different categories of user behavior, such as login attempts, file access, network requests, etc. Calculate the transition probabilities between states based on the collected data and construct a transition probability matrix. Combine the learned initial probability distribution of each state to simulate the generation of state sequences. Identify different behavior patterns by observing the transition of state sequences.

[0046] For each identified behavior group, statistical analysis is performed on the behaviors within the group, statistical features are extracted (including but not limited to node degree, node clustering coefficient, node Betweenness centrality, community modularity, etc.), and the behavior patterns within the group are identified. The extracted features are converted into numerical vectors and standardized to obtain the behavior features h of node i i behavior .

[0047] (2-2) Static feature extraction: In order to extract static features from the collected data, the TransR method is used to encode the nodes. Two transformation matrices are initialized and learned for each relationship type: one is used to convert the entity into a representation from the perspective of the relationship, and the other is used for the reverse conversion. At the same time, the vector representation of the entity is randomly initialized. By optimizing the model parameters to capture the semantics between entities and relationships, the final vector representation of the entity will serve as its static feature. Finally, it is standardized like its behavioral features to obtain the static feature h i static .

[0048] (2-3) Feature fusion: In order to ensure that entities with different feature sets can correspond, feature alignment is first performed to integrate data from different data sources according to entity identifiers, and synchronize data according to timestamps. Since the extraction methods of behavioral features and static features are inconsistent, they cannot be simply spliced ​​together. Different weights are assigned to different features according to their importance, and the weighted average is used to fuse the two to obtain the final representation h of node i. i Among them, w behavior and w static are the weights of behavioral features and static features respectively, and b is the bias parameter.

[0049]

[0050] In step (3), the detailed steps of anomaly detection are as follows:

[0051] (3-1) Model training: The node feature representation obtained in step (2) is input into the MLP for training. The softmax layer is applied to perform the anomaly detection task based on the final feature representation, and the weighted cross entropy loss is introduced to solve the problem of possible imbalanced classification.

[0052] x i =softmax(w0·h i +b0) (4)

[0053]

[0054] Where D refers to the node set of the data, p iis the probability that the model predicts that sample i is benign, w0 is the weight parameter of the current node, and b0 is the bias parameter.

[0055] (3-2) Abnormality judgment: In the test phase, the test data is processed according to the previous steps. Of course, data such as the state transition probability in the Markov chain will be used as constants to participate in the feature extraction of the test data, thereby improving the processing efficiency and enhancing the real-time performance of the method. The feature vector of the processed test data is input into the trained MLP model to obtain the abnormality score of the node. If it exceeds the threshold σ, the node is considered to be abnormal.

[0056] (3-3) Optimization and updating: Regularly collect and analyze the latest detection feedback from multiple sources, and update the model’s feature set through feature engineering. Using these updated features, the model is incrementally learned, while using transfer learning to retain existing knowledge.

[0057] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An APT anomaly detection method integrating the dual-channel GNN concept, characterized in that: The following steps are involved: (1) Data processing: define the system behaviors involved in the underlying system log data, collect benign data and convert it into a provenance graph; (2) Feature extraction and fusion: The behavioral and static features of benign data are extracted through the behavioral perception channel and the static perception channel, and then fused to obtain the comprehensive feature representation of each node; (3) Anomaly detection: Collect log data as test data, input the test data into the trained multi-layer perceptron MLP, and determine whether it is an abnormal node based on the loss between input and output.

2. According to claim 1, the APT anomaly detection method integrating the dual-channel GNN concept is characterized in that: The specific process of (1) is as follows: (1-1) Node type definition: Node types are defined based on the system behaviors involved in the log data, and relationship types are defined based on the interaction behaviors between nodes. A is defined as the node type set and R is defined as the relationship type set. (1-2) Data cleaning: Collect good log data from multiple sources and conduct data review on the log data to identify potential problems with the data; handle missing values ​​by filling, deleting, or predicting, and ensure that all records follow a unified time format and data type; (1-3) Constructing the provenance graph: Mapping the actual log data to the type definition, constructing the system behavior provenance graph G = (V, E), where G is a directed graph, V is a node set, E is a relationship set, any node in V corresponds to a node type in A, and any relationship in E corresponds to a relationship type in R.

3. According to claim 2, the APT anomaly detection method integrating the dual-channel GNN concept is characterized in that: The specific process of (2) is as follows: (2-1) Behavioral feature extraction: define key behavioral features related to APT attacks, discover behavioral groups through cluster analysis algorithms, and use sequence analysis methods to identify behavioral patterns; (2-2) Static feature extraction: The TransR method is used to encode nodes and extract static features from the collected data. (2-3) Feature fusion: First, feature alignment is performed to integrate data from different data sources according to entity identifiers and synchronize data according to timestamps. Then, different weights are assigned to different features according to their importance, and the two are fused using a weighted tie to obtain the final representation h of node i. i .

4. According to claim 3, the APT anomaly detection method integrating the dual-channel GNN concept is characterized in that: The specific process of (3) is as follows: (3-1) Model training: The node feature representation obtained in step (2) is input into the MLP model for training application; the softmax layer performs the anomaly detection task based on the final feature representation, and introduces weighted cross entropy loss to solve the problem of unbalanced classification; (3-2) Abnormality judgment: In the test phase, the test data is processed according to steps (1-3) and the steps before step (2); the feature vector of the processed test data is input into the trained MLP model to obtain the abnormality score of the node. If it exceeds the threshold σ, the node is considered to be abnormal; (3-3) Optimization and updating: Regularly collect and analyze the latest detection feedback from multiple sources, and update the feature set of the model through feature engineering; use these updated features to perform incremental learning on the model, while using transfer learning to retain existing knowledge.

Images