Multi-source log data fusion subtree traceability graph attack detection method based on active defense

Abstract

The invention provides an active defense-based multi-source log data fusion sub-tree traceability graph attack detection method, which comprises the following steps of: constructing an attack behavior graph according to CTI (Computer Telephony Integration) data, fusing multi-source log data and constructing a traceability graph; constructing a node alignment rule, and performing node alignment on the attack behavior graph and the traceability graph; taking the attack behavior graph after node alignment as a query graph, taking the traceability graph after node alignment as a target graph, and respectively performing subtree construction on the query graph and the target graph; performing subtree isomorphic calculation and recursive updating on the constructed query graph subtree and target graph subtree to obtain a subtree isomorphic matrix, and calculating a final subtree isomorphic index according to the subtree isomorphic matrix; performing similarity calculation on the query graph and the target graph to obtain a generalized index matrix; and attack detection is carried out based on the generalized index matrix and the subtree isomorphic indexes. According to the method, the low time complexity can be guaranteed, meanwhile, the high recognition rate is maintained, and the APT attack detection efficiency and accuracy are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network attack detection, and in particular to an active defense-based multi-source log data fusion subtree provenance graph attack detection method. Background Art

[0002] Threat hunting is an active defense strategy that emphasizes actively searching for unknown threats and attacks in the network environment before or after a threat occurs. Unlike passive defense, which mainly relies on set rules and signatures to identify known malicious activities, threat hunting focuses more on exploratory security analysis work, aiming to discover security events or advanced persistent threats (APTs) that traditional defense mechanisms fail to capture. APT attacks are usually carried out by highly organized attackers who often use system and application vulnerabilities as a breakthrough point for intrusion and use complex techniques and strategies to conduct long-term and covert attacks aimed at stealing sensitive information, destroying systems, or conducting other malicious activities. The goal of threat hunting is to detect these threats early and reduce potential damage. However, current network attacks have gone beyond the scope of a single step. Single-step attack alerts often only touch the tip of the iceberg of a complex multi-step attack chain, making it difficult to see the attacker's true intentions. Complex multi-step attacks are carefully planned to gradually penetrate and ultimately achieve the attack target. Therefore, security analysis urgently needs to deeply correlate each step of the attack behavior and fuse multiple log data sets to reveal its complete intentions. The researchers found that relying on system monitoring log data to construct a traceability graph with strong abstract expression capabilities for causal analysis can effectively express the cause, attack path and impact of threat events, providing higher detection efficiency and robustness for threat discovery and forensic analysis.

[0003] Currently, APT attack detection based on traceability graphs is mainly divided into rule-driven methods, anomaly methods, and machine learning methods. 1) Rule-driven methods: rely on predefined rules and signatures to detect APT attacks. These rules are usually built based on known attack features and behavior patterns. When the attack behavior matches the known rules, this method can provide high-precision detection results, reduce false alarm rates, and have low resource consumption. 2) Anomaly detection methods: by establishing normal behavior baselines for systems and networks, and detecting abnormal activities that deviate from these baselines, this method can effectively identify new and unknown attack patterns and reduce false alarm rates by focusing on activities that are significantly different from normal behaviors. 3) Learning methods: using machine learning and deep learning technologies, learning models can continuously learn and train to adapt to new attack technologies and changes, thereby achieving high-accuracy APT attack identification. Compared with traditional methods, machine learning models can more accurately distinguish between true attack behaviors and false alarm events.

[0004] However, APT attack identification based on traceability graphs still faces many challenges, including the rapid update and concealment of APT attack methods. Current methods have the following disadvantages: 1) The predefined edge matching rules are too sensitive and ineffective when attackers change or update tools; 2) They are easily interfered by false signals or cannot distinguish between attacks and benign system activities (i.e., they cannot cope with the high concealment and rich detection evasion methods of APT); 3) Current learning-based algorithms have high time complexity.

[0005] The invention patent with publication number CN117596044A discloses an attack detection method based on a knowledge graph recommendation model, the steps of which include: system log collection; traceability graph construction, including system entity construction, directed acyclic graph construction, and graph compression; knowledge graph construction, including interaction extraction and entity alignment; design and training of recommendation models, including single-step information modeling and multi-step information modeling; and finally, calculation of the probability of interaction possibility of the data to be detected to complete the detection of potential attacks. The present invention uses a knowledge graph that integrates traceability graph data and interaction information to design a model input, and uses a recommendation algorithm based on a graph neural network to design an attack detection model. The present invention can effectively detect potential attacks by analyzing the context of system behavior and extracting the interaction relationship between system entities. However, the algorithm time complexity of this method is high. Summary of the invention

[0006] In view of the technical problem that the existing models of current threat hunting methods based on graph matching learning have high complexity, the present invention proposes a multi-source log data fusion subtree tracing graph attack detection method based on active defense, and proposes the SGM-Hunter (SubGraph Match Hunter, subtree matching hunting model) model, a platform-independent, main memory-based audit log data dependency graph abstract subtree matching model, which defines the problem as a subtree matching problem with linear complexity. By introducing advanced subtree matching strategies and graph deep learning models, the entire captured network security log tracing graph is matched with known attack behavior graphs for node pair matching to deduce network attack paths. This method can not only effectively deal with complex and changeable attack modes, but also maintain a high recognition rate while ensuring low time complexity, thereby significantly improving the efficiency and accuracy of APT attack detection.

[0007] In order to achieve the above object, the technical solution of the present invention is achieved as follows:

[0008] A multi-source log data fusion subtree tracing graph attack detection method based on active defense, comprising the steps of:

[0009] S1: Build an attack behavior graph based on CTI data, fuse multi-source log data and build a traceability graph;

[0010] S2: Construct node alignment rules to align nodes of the attack behavior graph and the traceability graph;

[0011] S3: The attack behavior graph after node alignment is used as the query graph, and the traceability graph after node alignment is used as the target graph. The Weisfeiler-Leman algorithm is used to construct subtrees for the query graph and the target graph respectively.

[0012] S4: perform subtree isomorphism calculation on the constructed query graph subtree and target graph subtree and recursively update them to obtain a subtree isomorphism matrix, and calculate the final subtree isomorphism index based on the subtree isomorphism matrix;

[0013] S5: Calculate the similarity between the query graph and the target graph to obtain a generalized indicator matrix; perform attack detection based on the generalized indicator matrix and the subtree isomorphism indicator.

[0014] The multi-source log data includes firewall log data, network flow log data and system audit log data;

[0015] The method of fusing multi-source log data and constructing a traceability graph is as follows:

[0016] Multi-source log data fusion: define the attributes of nodes and edges; perform log parsing and cleaning on firewall log data, network flow log data, and system audit log data; define event association rules, and use the 4W method to find the same nodes between firewall log data, network flow log data, and system audit log data according to the event association rules and integrate the node attributes;

[0017] Construct a traceability graph: Encode the attributes and labels of different types of nodes into the same feature vector to obtain the traceability graph node feature X T , encode the attributes of different types of edges into the same feature vector, and obtain the traceability graph edge feature E T , build a traceability graph through the gallery.

[0018] The event association rule is: association matching based on similar timestamps, the same process ID, the same name, the same file path, and the same network address and port;

[0019] The alignment rules are:

[0020] Categorize specific process names into generic process types;

[0021] Classify specific file names into common file types based on their extensions or uses;

[0022] Classify specific IP addresses and port numbers as internal or external network traffic.

[0023] The method of using the Weisfeiler-Leman algorithm to construct subtrees for the query graph and the target graph is as follows: in the query graph, construct an L-layer tree containing all query nodes v of the query graph. j The query graph tree structure is constructed by using the Weisfeiler-Leman subtree generation function at each layer of the query graph tree structure to generate query nodes v in each layer. j Generate query node v for the root node j For each query node v at each level j And query node v j Perform K DropEdge sampling on the child nodes to obtain K query nodes v j The sampling subtree of the target graph is i As the root node, the target node v is generated using the Weisfeiler-Leman subtree generation function. i The subtree of .

[0024] The method for performing subtree isomorphism calculation and recursive updating on the constructed query graph subtree and target graph subtree is as follows: calculating the subtree isomorphism value of the leaf node according to the leaf node subtree isomorphism calculation formula, calculating the subtree isomorphism value of the non-leaf node according to the non-leaf node isomorphism calculation formula, and then recursively processing all nodes from the leaf node upward to obtain the subtree isomorphism matrix.

[0025] The method for calculating the leaf node subtree isomorphism value according to the leaf node subtree isomorphism calculation formula is as follows:

[0026] According to the leaf node subtree isomorphism calculation formula, K query nodes v j The sampling subtree and target node v i The subtree of the query node v is calculated j and the target node v i The K subtree isomorphic values ​​are aggregated and the aggregated result is used as the query node v j and the target node v i The subtree isomorphism value of the leaf node; the calculation formula for the subtree isomorphism of the leaf node is:

[0027]

[0028] in, Indicates querying node v j Is the subtree rooted at the target node v i is the subtree isomorphic to the root, k represents the kth sampling, and l is the query node v j The layer number, lab() represents the labeling operation;

[0029] The method for calculating the isomorphic value of a non-leaf node subtree according to the non-leaf node isomorphic calculation formula is as follows: according to the non-leaf node subtree isomorphic calculation formula, K query nodes v j The sampling subtree and target node v i The subtree of the query node v is calculated j and the target node v i The K subtree isomorphic values ​​are aggregated and the aggregated result is used as the query node v j and the target node v i The subtree isomorphism value of non-leaf node subtree isomorphism calculation formula is:

[0030]

[0031] Among them, C(v i ) is the target node v i The child node set of C(v j ) is the query node v j The child node set of B (l) (C(v j ),C(v i ))=1 indicates querying node v j All child nodes and target node v i All child nodes of are isomorphic.

[0032] The formula for aggregating K subtree isomorphic values ​​is:

[0033]

[0034] Among them, ⊙ represents the dot product;

[0035] The subtree isomorphism matrix is: is the element in the subtree isomorphism matrix;

[0036] The calculation method of the subtree isomorphism index is as follows:

[0037]

[0038] Among them, z subtree is the subtree isomorphism index, σ is the activation function, is the number of query nodes corresponding to the query node set, and MLP represents the calculation function of the fully connected layer.

[0039] The method for calculating the similarity of the query graph and the target graph to obtain the generalized indicator matrix is ​​as follows: using the edge feature conversion function to convert the edge features in the query graph and the target graph into node feature increments respectively, and splicing the node feature increments with the node features to obtain the first-layer node representations of the query graph and the target graph respectively, connecting the previous-layer node representations of the query graph and the target graph with the subtree isomorphism matrix respectively, and using the random walk algorithm to embed the node neighbor features to obtain the next-layer node representations of the query graph and the target graph respectively, calculating the node and edge similarity matrices of each layer of the query graph and the target graph according to the node representations of each layer of the query graph and the target graph, splicing the node and edge similarity matrices of each layer to obtain a complete node and edge attribute similarity matrix, and calculating the generalized indicator matrix according to the complete node and edge attribute similarity matrix and the subtree isomorphism index.

[0040] The calculation formula for the first layer node representation is:

[0041]

[0042] in, is the first-layer node representation of the target graph, is the first-layer node representation of the query graph, f() represents the feature conversion function, A represents the adjacency matrix, and W (0) is the weight, f update represents the update function, concat() represents the concatenation operation, X Q is the query graph node feature, E Q To query graph edge features;

[0043] The method for obtaining the next layer node representation of the query graph and the target graph is:

[0044]

[0045] in, is the node representation of the lth layer of the target graph, l ranges from 0 to L, is the node representation of the lth layer of the query graph, W (l1) , W (l2) represents the weight, A T , A Q are adjacency matrices, Represents the node features of the lth layer.

[0046] The calculation method of the similarity matrix of nodes and edges of each layer is:

[0047]

[0048] in, is the similarity matrix of nodes and edges in the l+1th layer, W (l3)is the weight, NTN represents the calculation function of the neural tensor network NTN model;

[0049] The calculation formula for concatenating the node representations of each layer to obtain a complete node and edge attribute similarity matrix is:

[0050]

[0051] Among them, MLP represents the calculation function of the fully connected layer, z gl is the complete node and edge attribute similarity matrix;

[0052] The method for calculating the generalized indicator matrix is:

[0053] Z=MLP(z gl *z subtree )

[0054] Among them, Z is the generalized indicator matrix.

[0055] Beneficial effects of the present invention:

[0056] The method of the present invention proposes to fuse multivariate data to construct a traceability graph, which can obtain more complete attack information and thus reduce false positives and false negatives.

[0057] The present invention proposes a strategy based on subtree matching, which effectively improves the model running speed and reduces the time complexity from exponential level to linear level compared with the subgraph matching model.

[0058] The model of the present invention is superior to existing models in terms of accuracy, recall, F-score and running time, and can accurately locate log entities related to APT attacks.

[0059] The model of the present invention enriches the expression of the graph and integrates the node, edge and structural information on the traceability graph.

[0060] In addition, SGM-Hunter has been successfully applied to the continuous threat detection and tracing system. The service has been online for 4 months. It is based on multi-source data such as assets, attackers, threat events, protocol metadata, threat intelligence, etc., and is associated with attack chains and scenarios, restoring attack events, issuing timely alarms, tracing threats, and playing a key role in detecting and defending against APT attacks. It has provided efficient and stable security services to more than 200 industry customers, with an APT attack interception rate of more than 99.9%. It fully proves its important role in APT attack identification and defense. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 It is a flow chart of the present invention.

[0063] Figure 2 It is a schematic diagram of the SGM-Hunter model structure of the present invention.

[0064] Figure 3 Schematic diagram of the data fusion rule of the present invention.

[0065] Figure 4 A schematic diagram of generating a subtree structure for nodes in a query graph of the present invention.

[0066] Figure 5 It is a schematic diagram of the dynamic programming algorithm of the present invention.

[0067] Figure 6 This is the node matching situation of the target graph and the query graph in the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] A multi-source log data fusion subtree traceability graph attack detection method based on active defense, such as Figure 1 As shown, the steps include:

[0070] S1: Build an attack behavior graph based on CTI data and fuse multi-source log data to build a traceability graph.

[0071] Data preparation: Utilize the system behavior collection mechanisms of different platforms including Auditd, Dtrace, ETW and Splunk to capture firewall log data, network flow log data and system audit log data as multi-source log data; collect data released by MISP and STIX network threat intelligence platforms as CTI data.

[0072] Further, an attack behavior graph is constructed based on the CTI data: Natural language processing (NLP) technology is used to extract key entity information from the CTI report, such as processes, files, IP addresses, etc. These entities will serve as nodes in the attack behavior graph, and based on expert knowledge and previous research work, behaviors related to network attacks are identified and screened. This embodiment adopts the AttackG algorithm, and uses NLP technology to match the description in the CTI report with the technology and examples in the MITRE ATT&CK framework, so as to extract keywords and generate a graph structure with labels (attack types). The extracted keywords and the generated graph structure are classified and labeled according to the technical labels of the MITRE ATT&CK framework, so that each node not only contains entity information, but also is associated with specific attack technologies or stages. The intrusion indicators (IOC, Indicators of Compromise) information is integrated into the attack behavior graph as part of the node attributes, and the logical or temporal relationships between entities are converted into edges in the graph, and finally the attack behavior graph G is obtained. Q (X Q ,E Q ,S Q ), X Q is the node feature of the attack behavior graph (query graph), E Q is the edge feature of the attack behavior graph, where S Q Represents graph structure information, including attack behavior graph (query graph) node set V Q ={v1,v2,…,v j},v j represents the jth node. For the convenience of explanation, we record v j The query node.

[0073] Furthermore, the multi-source log data is fused and a traceability graph is constructed. The process is as follows.

[0074] 1. Attribute definition of nodes and edges:

[0075] Node classification: Nodes are divided into three categories, including subject objects that actively initiate behaviors (such as processes), object objects that passively accept system behaviors (such as processes, files, network flows), and events that connect subject objects and object objects;

[0076] Define node attributes: such as process attributes, file attributes, and network flow attributes, as shown in Table 1;

[0077] Define the attributes of the edge: for example, the edge of the process, the edge of the file, and the edge of the network flow, as shown in Table 1.

[0078] 2. Log parsing and cleaning:

[0079] A parallel log parsing strategy is adopted to divide large log files into multiple small blocks according to time. Each thread or process executes the same parsing logic to extract the required information, such as timestamp, log level, message content, etc. After all parallel tasks are completed, the parsing results are merged to form the final analysis report;

[0080] Data cleaning mainly involves cleaning the collected logs, removing duplicates, inconsistently formatted data, or invalid entries, and using regular expressions to extract attributes from log fields based on the nodes with defined attributes to form structured data. Use regular expressions to extract attributes from log fields.

[0081] For example, if the log format is 2024-09-20 12:00:00

[12345] INFO Some message, the regular expression for extracting the process number in the process attributes is \[(\d+)\], and the extracted process number is: 12345.

[0082] Perform timestamp calibration to ensure that the timestamps of all logs are accurate to support analysis in the time dimension.

[0083] 3. Event association: Figure 3 ,The specific content of the data set in the figure is only for illustration and is not used to illustrate the technical solution. According to the event association rules, the 4W method is used to find the same nodes between the firewall log data, network flow log data and system audit log data and integrate the node attributes; the event association rules are: association matching based on similar timestamps, the same process ID, the same name, the same file path and the same network address and port.

[0084] For example: network flow records and firewall logs usually contain source and destination IP addresses and their ports, which can be used to associate network traffic with specific firewall events; timestamps can be used to arrange log events from different sources in chronological order (for example, blocking activities recorded in firewall logs can be found in corresponding traffic in network flow records and compared through timestamps); blocked malicious IP addresses may be recorded in firewall logs, while login activities of corresponding users may be recorded in system audit logs.

[0085] 4. Construct a traceability graph: Encode the attributes and labels of different types of nodes into the same feature vector to obtain the node feature X T , encode the attributes of different types of edges into the same feature vector to obtain edge features E T , in order to convert heterogeneous graphs into homogeneous graphs, and construct the traceability graph G through the graph library T (X T ,E T ,S T ), where ST Represents graph structure information, including source graph (target graph) node set V T ={v1,v2,…,v i},v i Represents the i-th node. For the convenience of explanation, let v i is the target node.

[0086] Among them, word2vec and onehot encoding are used to encode the attributes of nodes and edges. Among them, the attributes of nodes, such as host name, file path, file name, source IP and destination IP, are encoded by word2vec, the attributes of other nodes are encoded by onehot encoding, and the attributes of edges are encoded by onehot encoding.

[0087] 5. Graph pruning: including deleting edge nodes (node ​​A is pruned), deleting duplicate edges (edge ​​1 and edge 2, only one of them is retained), and deleting graph structures with the same structure (as shown by nodes D and E, only one node is retained).

[0088] Table 1

[0089]

[0090] Among them, the timestamp is only used for the similarity judgment of the nodes in the log data set fusion, and is not used for the subsequent node attribute learning.

[0091] S2: Construct node alignment rules to align nodes of the attack behavior graph and the traceability graph.

[0092] Classify specific process names into common process types;

[0093] Classify specific file names into common file types based on their extensions or uses;

[0094] Classify specific IP addresses and port numbers as internal or external network traffic.

[0095] In this embodiment, the results before and after are shown.

[0096]

[0097]

[0098] S3: The attack behavior graph after node alignment is used as the query graph, and the traceability graph after node alignment is used as the target graph. The Weisfeiler-Leman algorithm is used to construct subtrees for the query graph and the target graph respectively.

[0099] Furthermore, the Weisfeiler-Leman algorithm is used to construct subtrees for the query graph and the target graph respectively:

[0100] In order to improve the computational efficiency, we use the idea of ​​local sampling and feature aggregation, and based on the Drop Edge operation, we construct an L-layer containing all query nodes v in the query graph. j The query graph tree structure is constructed by using the Weisfeiler-Leman subtree generation function at each layer of the query graph tree structure to generate query nodes v in each layer. j Generate query node v for the root node j For each query node v at each level j And query node v j The child nodes of the query node v are dropped K times (in the present invention, each sampling is deleted j An edge in the subtree of ) is sampled to obtain K query nodes v j The sampling subtree of the target graph is i As the root node, the target node v is generated using the Weisfeiler-Leman subtree generation function. i The subtree of Figure 4 .

[0101] Specifically, construct the query graph subtree

[0102]

[0103] Specifically, construct the target graph subtree

[0104]

[0105] Among them, v j is the query node, v i is the target node, L is the depth of the tree, and Respectively represent the target node v at depth l i and query node v j The generated subtree, v q Represents the query node v j The child node set of v t Represents the target node v i The sub-node set, WL-Subtree l () represents the Weisfeiler-Leman subtree generation function, ∪ represents the merge operation. The results at each depth level are different. Each time a layer of depth is added, the node is expanded, and the node is added one layer downward. To calculate the subtree of the final node, it is necessary to merge it with the previously calculated results.

[0106] Determine whether there is an APT attack in the network security log, that is, verify whether the query graph belongs to the target graph. This problem can be transformed into whether there is a query graph subtree and the target graph subtree Matching, that is, subtree isomorphism.

[0107] There exists a target graph subtree with a depth of l and query graph subtree isomorphic and can be formulated as:

[0108]

[0109] Only when When querying node v j The generated subtree may match the target node v i The resulting subtree.

[0110] Further, subtree isomorphism calculation: define the leaf node subtree isomorphism calculation formula and non-leaf node isomorphism calculation formula.

[0111] S4: Perform subtree isomorphism calculation on the constructed query graph subtree and target graph subtree and recursively update them to obtain a subtree isomorphism matrix, and calculate the final subtree isomorphism index based on the subtree isomorphism matrix.

[0112] In a tree structure, nodes without child nodes are leaf nodes, and nodes with child nodes are non-leaf nodes.

[0113] Further, for the processing of leaf nodes: query node v j and the target node v i The condition for isomorphism is that the labels are the same. The calculation formula for isomorphism of leaf node subtrees is:

[0114]

[0115] in, Indicates querying node v j Is the subtree rooted at the target node v i is the subtree isomorphic to the root, l is the query node v j The layer number, lab() represents the label retrieval operation.

[0116] Further, for the processing of non-leaf nodes: query node v j and the target node v i The condition for isomorphism is that the labels are equal and the sub-node sets are isomorphic, which simplifies the isomorphism problem of sub-node sets to the problem of isomorphism detection of sub-nodes one by one.

[0117] Define C(v i ) is the target node v iThe child node set of C(v j ) is the query node v j The child node set of non-leaf nodes isomorphic calculation formula is:

[0118]

[0119] Among them, B (l) (C(v j ),C(v i ))=1 indicates querying node v j All child nodes and target node v i All child nodes of are isomorphic.

[0120] Furthermore, in order to obtain the dynamic programming table (i.e., the subtree isomorphism matrix), the subtree isomorphism value of the leaf node is calculated according to the leaf node subtree isomorphism calculation formula, and the subtree isomorphism value of the non-leaf node isomorphism calculation formula is calculated, and all nodes are recursively processed from the leaf node upward to obtain the subtree isomorphism matrix. Figure 5 In actual calculation, for the query graph, we construct an L-layer containing all query nodes v in the query graph. j The query graph tree structure is first calculated as the subtree isomorphism value of the subtree with the leaf node (such as T3, T5) as the root node, and then after obtaining the two subtree isomorphism values, the subtree isomorphism value of the subtree with the non-leaf node (such as T2) as the root node is calculated. The calculation is recursively calculated upward until all nodes are calculated to obtain the dynamic programming table (i.e., subtree isomorphism matrix). Dynamic programming recursively aggregates the labels of nodes and their neighborhoods, and then hashes the aggregated results into unique new labels. The process of filling the dynamic programming table is a systematic calculation process. It solves complex problems by gradually filling the table. The core is to find the recursive relationship of the problem and effectively store the intermediate results to avoid repeated calculations, thereby significantly improving the efficiency of the algorithm.

[0121] The method for calculating the leaf node subtree isomorphism value according to the leaf node subtree isomorphism calculation formula is as follows: according to the leaf node subtree isomorphism calculation formula, K query nodes v j The sampling subtree and target node v i The subtree of the query node v is calculated j and the target node v i The K subtree isomorphic values ​​are aggregated and the aggregated result is used as the query node v j and the target node v i The subtree isomorphic value of .

[0122] The method for calculating the isomorphic value of a non-leaf node subtree according to the non-leaf node isomorphic calculation formula is as follows: according to the non-leaf node subtree isomorphic calculation formula, K query nodes v j The sampling subtree and target node vi The subtree of the query node v is calculated j and the target node v i The K subtree isomorphic values ​​are aggregated and the aggregated result is used as the query node v j and the target node v i The subtree isomorphic value of .

[0123] The formula for aggregating K subtree isomorphic values ​​is:

[0124]

[0125] Here, ⊙ represents the dot product.

[0126] The subtree isomorphism matrix is: is the element in the subtree isomorphism matrix.

[0127] The calculation method of the subtree isomorphism index is as follows:

[0128]

[0129] Among them, z subtree is the subtree isomorphism index, σ is the activation function, is the number of query nodes corresponding to the query node set, and MLP represents the calculation function of the fully connected layer

[0130] Considering that in APT attacks, multiple attacks may appear in the log data within a period of time, and multiple attacks may occur. j The corresponding target node v i There may be multiple subtrees in the target graph, so a dynamic programming algorithm is used to deal with the subtree isomorphism problem.

[0131] S5: Calculate the similarity between the query graph and the target graph to obtain a generalized indicator matrix; perform attack detection based on the generalized indicator matrix and the subtree isomorphism indicator.

[0132] Most models do not consider the attributes of edges when performing graph matching. Since the number of edges in the traceability graph is much larger than the number of nodes, directly training the features of nodes and edges separately will increase the time complexity. The present invention adopts an edge feature conversion function to simplify the attributes of edges.

[0133] Specifically, the edge feature conversion function is used to convert the edge features in the query graph and the target graph into node feature increments, and the node feature increments are concatenated with the node features to obtain the first-layer node representations of the query graph and the target graph, respectively:

[0134]

[0135] in, is the first-layer node representation of the target graph, is the first-layer node representation of the query graph, f() represents the feature conversion function, A represents the adjacency matrix, and W (0) is the weight, f update Represents the update function, and concat() represents the concatenation operation.

[0136] Furthermore, the previous layer node representations of the query graph and the target graph are connected to the subtree isomorphism matrix respectively, and the random walk algorithm is used to embed the node neighbor features to obtain the next layer node representations of the query graph and the target graph respectively:

[0137]

[0138] in, is the node representation of the lth layer of the target graph, l ranges from 0 to L, is the node representation of the lth layer of the query graph, W (l1) , W (l2) represents the weight, A T , A Q are adjacency matrices, Represents the node features of the lth layer in the corresponding subtree.

[0139] Furthermore, the similarity matrix of nodes and edges of each layer of the query graph and the target graph is calculated according to the node representation of each layer of the query graph and the target graph:

[0140]

[0141] in, is the similarity matrix of nodes and edges in the l+1th layer, W (l3) is the weight, and NTN represents the calculation function of the neural tensor network NTN model.

[0142] Furthermore, the similarity matrix of nodes and edges in each layer is concatenated to obtain the complete attribute similarity matrix of nodes and edges:

[0143]

[0144] Among them, MLP() represents the calculation function of the fully connected layer, z gl is the complete node and edge attribute similarity matrix.

[0145] Furthermore, the generalized indicator matrix is ​​calculated based on the complete node and edge attribute similarity matrix and subtree isomorphism index:

[0146] Z=MLP(z gl *z subtree )

[0147] Among them, Z is the generalized indicator matrix. It represents the matching between node pairs. When |Z| takes 1, it proves that There is some kind of APT attack, such as Figure 6 Schematic diagram of the matching between the final target graph and the query graph.

[0148] The present invention also sequentially constructs a graph construction module, a node alignment module, a subtree matching module and a graph learning module; Figure 2 As shown, the FCs layer represents the fully connected layer. represents matrix dot product, Represents matrix merging (concat), and the figure shows: judging the matching status of traceability graph nodes T2, T3, T4 and query graph Q2, Q3, Q4 at the first depth level.

[0149] The graph construction module is used to construct an attack behavior graph based on CTI data, fuse multi-source log data and construct a tracing graph.

[0150] The node alignment module is used to construct node alignment rules and perform node alignment on the attack behavior graph and traceability graph.

[0151] The subtree matching module is used to use the attack behavior graph after node alignment as the query graph and the traceability graph after node alignment as the target graph, and use the Weisfeiler-Leman algorithm to construct subtrees for the query graph and the target graph respectively; the subtree isomorphism calculation is performed on the constructed query graph subtree and the target graph subtree and recursively updated to obtain the subtree isomorphism matrix, and the final subtree isomorphism index is calculated based on the subtree isomorphism matrix.

[0152] The graph learning module is used to calculate the similarity between the query graph and the target graph to obtain a generalized indicator matrix; attack detection is performed based on the generalized indicator matrix and subtree isomorphism indicators.

[0153] The model prediction result of the present invention is composed of a subtree module and a graph learning module. Directly calculating the final similarity score will result in loss deviation. For example, when fitting negative samples, the subtree structures between nodes are not the same, which makes the subtree similarity calculation result infinitely close to zero, thereby forcing the overall gradient to be zero, which hinders the training of the graph learning module. Therefore, different loss functions are used for the subtree module and the graph learning module to achieve their respective training goals. The loss function of the model is constructed according to the different goals of the subtree module and the graph learning module. For the subtree module, the subtree module designed to learn isomorphic relationships, the result should be 0 or 1, indicating mismatch or match. Therefore, the mean absolute error (MAE) loss function is used to force the result to be 0 or 1. For the graph learning module, the goal is to encourage the output of the graph learning module to capture the similarity between nodes. Therefore, the mean square error (MSE) loss function is used to measure the difference between the output of the GNN and the true label. y is the label of the node pair, and the loss function is:

[0154] Loss = MSE(zgl ,y)+MAE(z subtree ,y).

[0155] This invention innovatively proposes a multi-source log data fusion subtree traceability graph attack detection method based on active defense - SGM-Hunter, which aims to actively find unknown threats and attacks in the network environment. SGM-Hunter uses traceability graph technology to deeply mine the causal chain between multi-source log entities, refine CTI data and traceability graph structure, and then accurately align the attack behavior graph and the corresponding nodes in the traceability graph through node attributes. The core of the model lies in the subtree matching module, which significantly reduces the computational complexity to a linear level by efficiently matching node pairs. At the same time, the SGM-Hunter model constructs a multi-dimensional analysis framework, comprehensively considers node attributes, edge attributes and graph structure information, and greatly improves the accuracy.

[0156] The present invention uses three sets of real-world network attack data sets for verification. Table 4 compares the performance of the SGM-Hunter model with the APT attack detection algorithm.

[0157] Table 4

[0158]

[0159] Table 4 shows the performance comparison between different algorithms of three data sets. Among them, the Cadets data set and the Clearscope data set are derived from the DARPA data set created or supported by the Defense Advanced Research Projects Agency (DARPA) of the United States Department of Defense. Another group of DayDayPoc data sets are actual enterprise collection data sets, which contain a large number of attacks by external hosts on enterprise-related servers and internal users of enterprises. The present invention actively collects log data and utilizes system behavior collection mechanisms of different platforms including Auditd, Dtrace, ETW and Splunk to achieve respective collection and labeling. It can be seen that the model of the present invention is optimal in all evaluation indicators. Rule-based APT detection relies on a predefined set of rules to detect and identify potential threat behaviors. For known attacks, the accuracy is high and the recognition efficiency is high, but when a new attack type appears, the accuracy is greatly reduced when there is a lack of effective recognition strategy. This is also the reason why the accuracy of the SLEUTH algorithm on the data set is low, especially in the data set collected autonomously in the present invention, there are more new attack behaviors, resulting in a lower accuracy of the algorithm, only 88.37%. The POIROT algorithm builds an attack behavior graph to match APT attacks based on a threshold matching traceability graph. This method has relatively high accuracy and recall, which are 95.37% and 97.22% respectively on the Cadets dataset. The performance is relatively stable, but the graph matching speed is too slow and the time efficiency is low. And as the dataset increases, its accuracy also decreases. The anomaly-based detection algorithm first establishes the outline of normal network behavior, and then determines the behavior of abnormal activities as abnormal. Unlike rule-based detection methods that require prior knowledge of specific rules of malicious attacks, anomaly-based methods are less dependent on specific malicious samples or feature libraries, and can therefore cope with zero-day attacks. This method is reasonable in most cases, but sometimes legitimate activities may show patterns similar to anomalies, which may lead to false positives and require further manual analysis and adjustment of the anomaly detection model. Overall, THREATRACE and Unicorn algorithms achieved good results. Their accuracy, recall and F-scores are better than our algorithm. THREATRACE has higher accuracy and F-scores. Due to the matching rules of its algorithm, its performance is stable, but it also faces a certain degree of decline in accuracy when the data set increases. Learning-based APT (Advanced Persistent Threat) detection is a method that uses machine learning and data analysis to detect and identify potential threat behaviors.These methods generally capture the structural information and node attribute information in the traceability graph, but the accuracy of these two algorithms is not high. One reason is that there is too much noise in the information aggregation in the graph. On the other hand, SimGNN believes that if the sizes of the two input graphs are different, then the two graphs are not similar. This rule leads to the highest accuracy of SimGNN not exceeding 96.26%. In contrast, the model of the present invention provides an applicable APT attack detection model that can still maintain high accuracy and stability under large-scale data, with accuracy (Accuracy), recall (Recall) and F-score (F-score) as high as 98.26%, 98.53% and 97.70%.

[0160] The results show that the SGM-Hunter model has a minimum accuracy of 97.57%, and in the optimal scenario, the processing time is reduced to less than 1 second, which far exceeds the rule-based, anomaly detection and existing machine learning methods, demonstrating its excellent performance advantages. This result will greatly promote organizations to formulate more accurate and effective prevention and response strategies, and fundamentally improve the overall network security defense level.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-source log data fusion subtree traceability graph attack detection method based on active defense, characterized in that: Includes steps: S1: Build an attack behavior graph based on CTI data, fuse multi-source log data and build a traceability graph; S2: Construct node alignment rules to align nodes of the attack behavior graph and the traceability graph; S3: The attack behavior graph after node alignment is used as the query graph, and the traceability graph after node alignment is used as the target graph. The Weisfeiler-Leman algorithm is used to construct subtrees for the query graph and the target graph respectively. S4: perform subtree isomorphism calculation on the constructed query graph subtree and target graph subtree and recursively update them to obtain a subtree isomorphism matrix, and calculate the final subtree isomorphism index based on the subtree isomorphism matrix; S5: Calculate the similarity between the query graph and the target graph to obtain a generalized indicator matrix; Attack detection based on generalized indicator matrix and subtree isomorphism indicators.

2. According to the active defense-based multi-source log data fusion subtree traceability graph attack detection method of claim 1, it is characterized in that: The multi-source log data includes firewall log data, network flow log data and system audit log data; The method of fusing multi-source log data and constructing a traceability graph is as follows: Multi-source log data fusion: define the attributes of nodes and edges; Perform log parsing and cleaning on firewall log data, network flow log data and system audit log data; define event correlation rules, and use the 4W method to find the same nodes between firewall log data, network flow log data and system audit log data according to the event correlation rules and integrate the node attributes; Construct a traceability graph: Encode the attributes and labels of different types of nodes into the same feature vector to obtain the traceability graph node feature X T , encode the attributes of different types of edges into the same feature vector, and obtain the traceability graph edge feature E T , build a traceability graph through the gallery.

3. The multi-source log data fusion subtree traceability graph attack detection method based on active defense according to claim 2 is characterized in that: The event association rule is: association matching based on similar timestamps, the same process ID, the same name, the same file path, and the same network address and port; The alignment rules are: Categorize specific process names into generic process types; Classify specific file names into common file types based on their extensions or uses; Classify specific IP addresses and port numbers as internal or external network traffic.

4. The multi-source log data fusion subtree traceability graph attack detection method based on active defense according to claim 3 is characterized in that: The method of using the Weisfeiler-Leman algorithm to construct subtrees for the query graph and the target graph is as follows: in the query graph, construct an L-layer tree containing all query nodes v of the query graph. j The query graph tree structure is constructed by using the Weisfeiler-Leman subtree generation function at each layer of the query graph tree structure to generate query nodes v in each layer. j Generate query node v for the root node j For each query node v at each level j And query node v j Perform K DropEdge sampling on the child nodes to obtain K query nodes v j The sampling subtree of the target graph is i As the root node, the target node v is generated using the Weisfeiler-Leman subtree generation function. i The subtree of .

5. The multi-source log data fusion subtree provenance graph attack detection method based on active defense according to any one of claims 1 to 4, characterized in that: The method for performing subtree isomorphism calculation and recursive updating on the constructed query graph subtree and target graph subtree is as follows: calculating the subtree isomorphism value of the leaf node according to the leaf node subtree isomorphism calculation formula, calculating the subtree isomorphism value of the non-leaf node according to the non-leaf node isomorphism calculation formula, and then recursively processing all nodes from the leaf node upward to obtain the subtree isomorphism matrix.

6. The multi-source log data fusion subtree traceability graph attack detection method based on active defense according to claim 5 is characterized in that: The method for calculating the leaf node subtree isomorphism value according to the leaf node subtree isomorphism calculation formula is as follows: according to the leaf node subtree isomorphism calculation formula, K query nodes v j The sampling subtree and target node v i The subtree of the query node v is calculated j and the target node v i The K subtree isomorphic values ​​are aggregated and the aggregated result is used as the query node v j and the target node v i The subtree isomorphism value of the leaf node; the calculation formula for the subtree isomorphism of the leaf node is: in, Indicates querying node v j Is the subtree rooted at the target node v i is the subtree isomorphic to the root, k represents the kth sampling, and l is the query node v j The layer number, lab() represents the labeling operation; The method for calculating the isomorphic value of a non-leaf node subtree according to the non-leaf node isomorphic calculation formula is as follows: according to the non-leaf node subtree isomorphic calculation formula, K query nodes v j The sampling subtree and target node v i The subtree of the query node v is calculated j and the target node v i The K subtree isomorphic values ​​are aggregated and the aggregated result is used as the query node v j and the target node v i The subtree isomorphism value of non-leaf node subtree isomorphism calculation formula is: Among them, C(v i ) is the target node v i The child node set of C(v j ) is the query node v j The collection of child nodes of B (l) (C(v j ),C(v i ))=1 indicates querying node v j All child nodes and target node v i All child nodes of are isomorphic.

7. The multi-source log data fusion subtree traceability graph attack detection method based on active defense according to claim 6 is characterized in that: The formula for aggregating K subtree isomorphic values ​​is: Among them, ⊙ represents the dot product; The subtree isomorphism matrix is: is the element in the subtree isomorphism matrix; The calculation method of the subtree isomorphism index is as follows: Among them, z subtree is the subtree isomorphism index, σ is the activation function, |V Q | is the number of query nodes corresponding to the query node set, MLP represents the computation function of the fully connected layer.

8. The multi-source log data fusion subtree traceability graph attack detection method based on active defense according to claim 7 is characterized in that: The method for calculating the similarity of the query graph and the target graph to obtain the generalized indicator matrix is ​​as follows: using the edge feature conversion function to convert the edge features in the query graph and the target graph into node feature increments respectively, and splicing the node feature increments with the node features to obtain the first-layer node representations of the query graph and the target graph respectively, connecting the previous-layer node representations of the query graph and the target graph with the subtree isomorphism matrix respectively, and using the random walk algorithm to embed the node neighbor features to obtain the next-layer node representations of the query graph and the target graph respectively, calculating the node and edge similarity matrices of each layer of the query graph and the target graph according to the node representations of each layer of the query graph and the target graph, splicing the node and edge similarity matrices of each layer to obtain a complete node and edge attribute similarity matrix, and calculating the generalized indicator matrix according to the complete node and edge attribute similarity matrix and the subtree isomorphism index.

9. The multi-source log data fusion subtree traceability graph attack detection method based on active defense according to claim 8 is characterized in that: The calculation formula for the first layer node representation is: in, is the first-layer node representation of the target graph, is the first-layer node representation of the query graph, f() represents the feature conversion function, A represents the adjacency matrix, and W (0) is the weight, f update represents the update function, concat() represents the concatenation operation, X Q is the query graph node feature, E Q To query graph edge features; The method for obtaining the next layer node representation of the query graph and the target graph is: in, is the node representation of the lth layer of the target graph, l ranges from 0 to L, is the node representation of the lth layer of the query graph, W (l1) , W (l2) represents the weight, A T , A Q are adjacency matrices, Represents the node features of the lth layer.

10. The multi-source log data fusion subtree traceability graph attack detection method based on active defense according to claim 9 is characterized in that: The calculation method of the similarity matrix of nodes and edges of each layer is: in, is the similarity matrix of nodes and edges in the l+1th layer, W (l3) is the weight, NTN represents the calculation function of the neural tensor network NTN model; The calculation formula for concatenating the node representations of each layer to obtain a complete node and edge attribute similarity matrix is: Among them, MLP represents the calculation function of the fully connected layer, z gl is the complete node and edge attribute similarity matrix; The method for calculating the generalized indicator matrix is: Z=MLP(z gl *With subtree ) Among them, Z is the generalized indicator matrix.

Images