Intrusion detection method based on traceability graph

Abstract

An intrusion detection method based on a traceability graph comprises the following steps that S1, an original log data set not containing hostile attacks serves as input, logs are analyzed, and an initial full-system traceability graph is constructed; s2, deleting redundant nodes and edges on the initial full-system traceability graph; s3, specific domain knowledge is fused in the node feature construction and graph representation process, and initial embedding is generated for nodes on the traceability graph; s4, constructing a GraphSAGE-based neural network model, performing node classification on the traceability graphs after training is finished, storing sub-models obtained by training each time, and jointly using the sub-models for abnormal node detection; s5, the log data containing attack traces serve as input, a traceability graph is constructed in real time, and the traceability graph constructed in real time is detected; writing a single abnormal node on the identification traceability graph into an alarm file; and S6, carrying out attack scene reconstruction by taking the abnormal node alarm as an entry event of attack investigation. The false alarm rate can be reduced, and the detection performance of the intrusion detection method can be improved.

Description

Technical Field

[0001] This invention belongs to the field of network traffic anomaly detection technology, specifically relating to an intrusion detection method based on source graphs. Background Technology

[0002] Intrusion detection systems (IDS) serve as a crucial line of defense in cybersecurity, enabling early identification and response to attacks. This significantly helps cybersecurity analysts discover threats and mitigate losses as early as possible.

[0003] For network flow-based intrusion detection methods, Gu Zhaoquan et al. from Harbin Institute of Technology invented a "network intrusion detection method, system and medium based on graph anomaly detection algorithm". This method proposes a specific approach to model and analyze network flows. It aggregates the source IP address, source port number, destination IP address, destination port number and transport layer protocol in network data traffic into a five-tuple network flow graph. It then uses a similarity calculation method to reduce the size of the network flow graph. Finally, it uses a graph autoencoder to encode and decode network flow graphs with the same source and the same destination. Malicious traffic is distinguished based on the obtained anomaly score.

[0004] Compared to network traffic-based intrusion detection methods, host-based intrusion detection methods are not limited to the network layer, can analyze host internal behavior in a more granular way, and facilitate source tracing analysis of attack behaviors.

[0005] For host-based intrusion detection methods, Chen Bing et al. from Nanjing University of Aeronautics and Astronautics invented "Host Intrusion Detection Method and Device, Storage Medium and Electronic Device". The method describes the process of constructing computer host audit logs into a directed source graph, and proposes a graph encoder for graph-level embedding and a dual autoencoder for predicting graph embedding.

[0006] Xie Yulai and colleagues from Huazhong University of Science and Technology invented "An Intrusion Detection Method Based on Source Graphs." This method divides intrusion detection into two stages: rapid judgment and precise judgment. It also proposes a method for calculating the importance of nodes in the source graph. In the rapid judgment stage, nodes with high importance and their neighboring nodes are selected and converted into a first neighborhood matrix. In the precise judgment stage, the size of the central node and its neighboring nodes is expanded, and the source graph is deeply mined to construct a second neighborhood matrix. The differences between the first and second neighborhood matrices and normal regular behavior are calculated, and different thresholds are set to determine abnormal behavior.

[0007] Similar to the host-based intrusion detection methods mentioned above, previous anomaly detection methods are not applicable to anomaly detection in large source maps.

[0008] In summary, the shortcomings of existing technologies are:

[0009] 1. Traditional network traffic-based anomaly detection systems can only analyze statistical characteristics during network transmission, such as IP address, port number, and packet size. They cannot obtain specific operational information of user-related processes within the host, making it difficult to distinguish the essential differences between normal and malicious traffic. Furthermore, relying on thresholds to detect malicious traffic is more prone to problems such as high false negative and false positive rates.

[0010] 2. The data source for intrusion detection systems based on source maps is computer audit logs. These logs provide long-term, detailed records of various system operations, including process communication, network connections, user actions, and file access. Long-running computers generate massive amounts of log data, and processing such large volumes significantly increases system complexity and response time.

[0011] 3. Existing methods of using graph neural networks to learn source graphs can better learn the topological relationships between nodes and capture the behavioral patterns of source graphs. However, using graph neural networks often tends to ignore the semantic information of the nodes themselves and focuses only on spatial information, making it difficult to distinguish between normal and abnormal nodes.

[0012] 4. Anomaly detection, as the finest-grained detection level, boasts the highest detection accuracy. Traditional anomaly detection tasks treat distinguishing anomaly nodes as supervised binary classification learning, using MLP networks to output binary classification results. However, this approach heavily relies on expert knowledge, requiring prior knowledge of attack patterns to label attack attribution maps. Furthermore, manual anomaly labeling is highly inefficient. Therefore, many studies only consider benign data, learning normal baseline models. However, these anomaly-based intrusion detection systems all face the problem of high false positive rates, exacerbated by the lack of attack data and the imbalance between benign and malicious data. Summary of the Invention

[0013] To overcome the shortcomings of existing technologies, this invention aims to provide an intrusion detection method based on a source graph. By employing an efficient neighbor feature information aggregation method and applying GraphSAGE to sample and aggregate nodes, the computational burden of large graphs can be effectively reduced. GraphSAGE supports multiple aggregation methods, enabling better capture of dependencies between nodes. Furthermore, this invention overcomes the problem of a single model detecting all node features through incremental learning. Specifically, it trains multiple sub-models using different data samples, and during detection, these trained sub-models work together to detect abnormal nodes, reducing false positive rates and improving the detection performance of the intrusion detection method.

[0014] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0015] An intrusion detection method based on a source graph includes the following steps:

[0016] S1: Source graph construction. The original log dataset, which does not contain malicious attacks, is used as input. The logs are parsed and an initial source graph of the entire system is constructed.

[0017] S2: Source graph reduction, which removes redundant nodes and edges from the initial source graph of the entire system; reduces the size of the source graph; facilitates the subsequent extraction of node features from the source graph, and accelerates the training efficiency of the model;

[0018] S3: Node feature extraction incorporates specific domain knowledge into the node feature construction and graph representation process, including node semantic information, spatial structure information, and potential vulnerability information. Starting from multiple dimensions, it generates initial embeddings for nodes on the traceability graph, which serve as inputs for subsequent neural network models. Multi-dimensional node feature extraction can prevent attackers from easily tampering with node feature information and increase the stability of the model.

[0019] S4: Construct a neural network model based on GraphSAGE and train it using incremental learning. First, divide the nodes into different types such as processes, files, and networks, which serve as labels for node classification. The number of training rounds during incremental learning is not fixed. Before each round of training, a source graph of a certain size is loaded. After training, the nodes are classified on these source graphs. Nodes that are not correctly classified on the source graph are repeatedly trained, i.e., loaded into the next round of training graphs, while correctly classified nodes are not trained again. After multiple rounds of training, all nodes are correctly classified. Save the sub-models obtained from each training and use the sub-models together for abnormal node detection.

[0020] S5: In the abnormal node identification stage, log data containing attack traces is used as input to construct a source map in real time. Multiple sub-models trained by the neural network model based on GraphSAGE in S4 are then loaded to detect the source map constructed in real time. Individual abnormal nodes on the source map are identified and written into the alarm file to facilitate subsequent attack forensics analysis by network security analysts.

[0021] S6: Use abnormal node alerts as entry events for attack investigation to reconstruct the attack scenario.

[0022] Preferably, in S1, the data source for constructing the source graph is computer audit logs. Audit logs under different operating systems have different formats. For unstructured logs, the entities appearing in the logs are abstracted into three types: processes, files, and network streams, which are used as nodes on the source graph. The abstracted event types are used as the features of the edges between nodes, and finally the log text data is constructed into a system-wide source graph.

[0023] In the S1 stage of constructing the source graph, the size of the collected source graph is directly proportional to the number of hosts and the collection time. Since APT attacks can last from several days to several months, the amount of data can quickly reach TB levels, causing storage and computational burdens. To achieve high-performance source graph learning, it is necessary to reduce the source graph. Preferably, in S2, several strategies are proposed for reduction. First, structural reduction eliminates isolated nodes in the initial system-wide source graph, which is crucial for large-scale log analysis. Second, semantic reduction merges similar operations between two nodes, eliminating redundant edges and further making the source graph more compact. Finally, based on specific domain knowledge, a whitelist process is set to filter out highly trustworthy and known-safe nodes from the source graph, such as scheduled task nodes and nodes connected by benign IP addresses. These different strategies result in a compact, reduced source graph, facilitating the subsequent learning of benign source graph patterns.

[0024] Preferably, in S3, three different levels of node features are considered, including the executable permissions of process nodes, the spatial information of nodes, and the semantic features of nodes.

[0025] When managing users and system resources, the operating system uses a permission model to ensure data security, sets different access control permissions for different users and processes, and encodes the vulnerability information of the process into a 16-dimensional multi-hot vector, which is then added to the embedding vector of the process node.

[0026] The spatial information of a node refers to the connections between the node and its neighbors. These connections reflect the node's location within the system and its interaction with other nodes. In graph data structures, the in-degree and out-degree of a node are important indicators of its importance and connectivity. The spatial characteristics of a node can be effectively represented by statistically analyzing its in-degree and out-degree information across the entire system's origin graph.

[0027] Log analysis revealed that different types of entities each have their own absolute paths, and this absolute path information reflects the entity's origin, permissions, and behavioral patterns. For example, in a Linux system, ` / usr / bin / ` typically stores user-executable programs, most of which do not require administrator privileges, while ` / usr / sbin / ` stores system administration data, which usually requires root privileges to execute. ` / usr / local / ` stores software installed locally by users and is not part of the system's default programs. These different path information are reflected in the source graph as the topology between nodes. By extracting the absolute path information of different types of entities, the root directory attribute of the path is added to the node type.

[0028] Preferably, in S4, an unsupervised network model based on GraphSAGE is constructed, consisting of two parts: a GraphSAGE neighbor aggregation module and an incremental training module.

[0029] The GraphSAGE neighbor aggregation module is used to aggregate neighbor features for each node in the source graph, and efficiently generate node embedding representations with contextual semantics through message passing and neighbor sampling.

[0030] The incremental training module is used to improve learning efficiency by learning large source graphs in multiple stages to solve the problem of excessive memory consumption during one-time learning. At the same time, it uses multiple sub-model detection to overcome the problem that a single model cannot accurately detect all nodes.

[0031] When using GraphSAGE to aggregate the neighbors of a central node, k aggregators are set to aggregate the neighbor features of the central node, and the axis is the node's initialization vector x. v Assign to the first hidden layer vector Iterate through the k-hop neighbors and use an aggregator function to aggregate neighbor information to update the embedding of each neighbor node at each level:

[0032]

[0033] Where N(v) represents all neighboring nodes of v, and AGGREGATE represents an aggregator function. During training, various aggregators are selected to embed the updated nodes from the k-th layer. In addition to the self-embedding vector before the update Then pass it through a fully connected layer with a non-linear activation function σ:

[0034]

[0035] The above steps aggregate k-order neighbors. Each aggregation represents a parameter weight update, and a total of k-1 updates are performed to obtain the final output feature z. v ,in Then, aggregate the results z from the last time for each central node. v After softmax normalization, the calculation formula is as follows:

[0036]

[0037] Where M is the number of node categories.

[0038] The probability of obtaining the category Next, the CrossEntroyLoss function is used to calculate the loss between the expected label and the actual label of the node, as follows:

[0039]

[0040] Where n is the total number of activity nodes in a batch, y ic It is the true category; when the node category equals c, y ic If the loss value is 1, then the weight matrix is ​​updated by gradient descent, otherwise 0. This allows us to learn different types of benign nodes.

[0041] The incremental training module divides the training graph into multiple subgraphs for training, setting a maximum number of active nodes per batch to ensure scalability. Furthermore, considering the imbalance of node types, learning all node features using a single model is extremely difficult. Besides type imbalance, the ratio of benign to malicious nodes within the same type is also unbalanced. Therefore, this invention designs an incremental learning model that obtains a set of sub-models through multiple training iterations. When identifying abnormal nodes, all sub-models must be used. The incremental training steps are as follows:

[0042] First, maintain an empty list X in memory to store nodes that have not yet been correctly classified. Initialize the untrained nodes from a batch into the active list X. After k aggregations using GraphSAGE, obtain the first sub-model M1, which detects all nodes in X. During detection, set the confidence level to R. If the maximum predicted probability exceeds the second-largest predicted probability R, the classification is considered correct.

[0043]

[0044] Where C(v) is the final label of node v, L(v) is the predicted label of node v, and M... p(v) M is the maximum probability of the predicted node in model M. p′(v) This is the second-highest probability predicted by model M for a node. Correctly classified nodes are removed from X, and the remaining nodes in X are added to the next batch of training graphs.

[0045] Continuously add untrained nodes to X, repeating the above steps until the list X is empty. This indicates that each node has been learned and correctly classified. Nodes that are not correctly classified during training are repeatedly reinforced, and a sub-model is obtained at the end of each training batch, ultimately resulting in a set of sub-models M. s .

[0046] During the testing phase, similar to the training phase, an empty subgraph is maintained, and newly arriving nodes and edges are continuously added. When the test graph size equals s, it is processed as a batch and sequentially passed through the sub-model for detection. An anomalous node cannot be confidently classified into a category in every sub-model; therefore, a detection confidence level R is set here. tIf the probability predicted by all sub-models does not exceed this threshold, the node is determined to be an anomalous node. In other words, if a node is correctly classified in any sub-model, then the node is a benign node. This detection method greatly reduces the number of false positives and decreases the false alarm rate.

[0047] Preferably, in the anomaly detection phase of S5, to identify a single anomalous node, the malicious nodes in the training dataset need to be expanded. A k-order expansion method is used to expand the malicious nodes. A threshold k is set, which considers all neighbors within k hops of an anomalous node to be suspicious nodes. A breadth-first search k-order expansion algorithm is used to expand the initial malicious nodes. For the source graph G = (V, E), where V is the node set and E is the edge set, the initial set of anomalous nodes in the groundtruth is S = {s1, s2, ... s}. n},s j For a node ∈ V, its neighboring nodes N(s) = {t, (s,t) ∈ E}. First, calculate the set of first-order suspicious nodes. The formula is as follows:

[0048]

[0049] Then continue to extend to the kth order:

[0050]

[0051] Finally, a set of abnormal nodes is obtained.

[0052] Preferably, in S6, an attack scenario reconstruction method based on POI (Point of Interest) event propagation is used;

[0053] First, analyze the alarm files generated after S5 ends, select one POI event, such as the modification of a sensitive file, the creation of a suspicious file, or access by an abnormal IP, and generate an initial dependency graph related to this POI event. At this time, the dependency graph will be very large. Next, edge aggregation operation needs to be performed on the dependency graph to eliminate duplicate events. Finally, starting from the POI event, trace the source in both the forward and reverse directions on the simplified dependency graph to obtain the complete attack scenario.

[0054] The method is applied to host-based intrusion detection systems and is suitable for environments that require processing large amounts of log information. It is particularly suitable for host intrusion detection and abnormal node identification based on source graphs.

[0055] The beneficial effects of this invention are:

[0056] This invention offers a richer advantage in extracting node features from the entire computer system's source graph. By combining the learning model with graph neural networks during training, it can uncover the potential relationships between entities running within the system, solving the problem of insufficient node feature extraction in existing anomaly detection methods. Furthermore, it employs a GraphSAGE-based incremental learning approach to obtain a large number of sub-models. During anomaly detection, a single anomaly node is not immediately reported; instead, multiple sub-models obtained through incremental training detect it simultaneously. Compared to traditional source graph-based intrusion detection systems, using all sub-models for anomaly detection significantly reduces the high false positive rate faced by node-level intrusion detection systems. The proposed network also exhibits higher learning efficiency and superior generalization ability. The attack scenario reconstruction method introduces forward and reverse source tracing mechanisms based on POI events, visualizing key attack scenarios and resolving the issue of a large number of discrete alerts being unable to correlate. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of an abnormal node detection method based on GraphSAGE provided in an embodiment of the present invention.

[0058] Figure 2 This is a simplified schematic diagram of the traceability graph provided in an embodiment of the present invention.

[0059] Figure 3 This is a flowchart of the incremental learning process proposed in this embodiment of the invention.

[0060] Figure 4 This is a flowchart illustrating the attack scenario reconstruction process after abnormal node detection in an embodiment of the present invention. Detailed Implementation

[0061] The present invention will now be described in further detail with reference to the accompanying drawings.

[0062] Example 1:

[0063] Figure 1 This is a schematic diagram of the abnormal node detection method based on GraphSAGE provided in an embodiment of the present invention. The specific steps are as follows:

[0064] S1: In the source graph construction phase, the audit logs used in this embodiment of the invention originate from a project of the U.S. Defense Advanced Research Projects Agency (DARPA). This project aims to address the persistent advanced threats (APTs) caused by the lack of transparency in modern computing systems, which often remain hidden and undetectable. By leveraging transparent computing environment data, it ensures the visibility of interactions between all components of the computer system and dynamically tracks the causal dependencies between components. The provided dataset is one of the standard datasets in the field of network security. The source graph is constructed using this dataset, with various runtime entities contained in the logs forming nodes, and the interactions between entities, i.e., system call relationships, forming edges. Node types generally include processes, files, sockets, network connections, etc., while edge types generally include system calls such as read, write, and execute. The basic meta-model of the source graph constructed using the above method is shown in Table 1 below.

[0065] Table 1. Description of the Source Tracing Primitive Model Construction

[0066]

[0067]

[0068] S2: The source graph reduction phase includes three reduction strategies. The first strategy is to eliminate orphaned nodes. Due to different log system configurations, the collected audit logs are diverse and numerous. These logs can be used to construct a large-scale system-wide source graph, which often contains many orphaned nodes. These orphaned nodes have no connection with other nodes and do not significantly help in modeling normal behavior or detecting abnormal behavior. Retaining these orphaned nodes can actually affect the message passing mechanism of the graph neural network, leading to poorer algorithm results. Therefore, it is necessary to eliminate orphaned nodes to ensure the effectiveness of the source graph.

[0069] The second strategy is to eliminate redundant edges. Computer audit logs are operating system-level logs, and a single user-level operation may generate multiple audit logs for low-level system calls. When we modeled the initial logs as a system-wide source graph, we did not remove these redundant system call logs, resulting in a large number of redundant edges. These redundant edges need to be removed during the graph reduction process.

[0070] The third strategy is to use a whitelist of filtering nodes. When simplifying the source graph, some known safe and trusted nodes can be excluded from the entire system source graph. These nodes can be IP addresses, applications, or scheduled tasks.

[0071] In this embodiment, the above three strategies can preserve the semantics of the source graph to the greatest extent while reducing its size. Figure 2This is a simplified source graph diagram of this embodiment. Rectangles represent processes, ellipses represent files, P0 is the top-level parent process node, P1 and P1' are child processes with the same name as P0, and P2 is another child process. Outside this connected graph, there is an isolated process P8, and several smaller source graphs containing only two process nodes. The right side of the diagram shows the simplified source graph. It can be seen that the latest source graph does not contain the isolated process P8, nor does it contain P9 and P10 with a large number of repetitive fixed patterns, because they have been added to the whitelist. Only one interaction edge is retained between processes and files.

[0072] S3: Node feature extraction module. In this embodiment of the invention, three types of node features are considered to enhance the graph neural network model’s ability to represent graphs: the executable permissions of process nodes, the spatial information of nodes, and the semantic information of nodes.

[0073] 1) This embodiment of the invention uses the list of privileges that can be escalated by attackers when common processes are exploited, provided by the security expert tool GTFObins. This includes the legitimate functions of some binary files and how these legitimate functions are abused. GTFObins contains a total of 320 binary lists: awk, bash, nc, nmap, ping, etc., and 16 potential privilege escalation methods: shell, sudo, file-read, file-write, etc. Taking awk as an example, awk is a text processing tool. In the Linux operating system, the binary executable file is stored in the / usr / bin / directory. Attackers can inject commands into awk to perform arbitrary operations. A common method is to execute awk'BEGIN{system(" / bin / sh")}' to open a shell and gain system access. This embodiment encodes the 16 privilege information in GTFObins using the function f. λ The vector is mapped to a 16-dimensional multi-hot vector and added to the embedding vector of the process node.

[0074] 2) Node spatial information can represent the potential behavior of nodes. Attacks are often accompanied by entity activities that deviate from normal patterns, and the resulting source graphs are often significantly different from those of normal hosts. Therefore, by learning the behavioral patterns of various types of entities under normal host activity, a baseline model of normal behavior can be established, and deviations from abnormal entities can be learned. Common brute-force attack techniques involve using automated tools to continuously attempt to crack simple passwords on the SSH service, or attempting to bypass SSH login authentication through dictionary attacks. This generates a large number of identical operation logs, which appear as a very large out-degree of the SSH service node on the source graph. Similar to the abnormal source graph caused by SSH brute-force attacks, common attack techniques such as DDoS attacks, malicious code injection, and SQL injection also cause abnormal in-degrees and out-degrees of different node types.

[0075] In this embodiment, the categories (read, write, join, etc.) of interactions between nodes on the source graph are first calculated. e Because the entity interaction relationships in real computer operating systems are complex, the same user action may generate multiple different system call logs, containing a large amount of redundant information. Therefore, this paper summarizes different event types into abstract event types and removes redundant information. To encode the spatial information of node v, a mapping function f needs to be learned. ω Map node v to 0 to 2·N e The multi-hot vector on, the first N e A dimensional vector represents the position of node v in N. e in-degree above, after N e Dimension represents node v in N e The degree of departure above.

[0076] 3) Node semantic information is the absolute path information of a node. In the Linux system, user-level programs and system-level programs are stored in different paths, and therefore have different interaction behaviors with other nodes, which is reflected in the source graph as different graph topologies.

[0077] In this embodiment of the invention, node types are classified according to path information, and the node category with root attribute is used as the fine-grained label N of the node. v Three abstract categories (process, file, network stream) are used as coarse-grained labels N for nodes. v '. Constructor f φ Map nodes to 0-N v Where v∈V, f φ = [a0, a1, 0, 0, ..., 0], where a0 and a1 represent the abstract node type (if a0 is 1, it represents a process type node; if a1 is 1, it represents a file type node; if both are 0, it represents other types of nodes), and the remaining dimensions are of size N. vThe vector represents the specific node category.

[0078] The initial node embedding is obtained after applying the above three mapping functions.

[0079]

[0080] S4: The audit log data input from S1, S2, and S3 are all benign data. In this embodiment, an unsupervised network model based on GraphSAGE is constructed to learn benign behavior patterns that do not contain attacks. The specific steps are as follows:

[0081] 1) Set the sample size. In order to balance the training graph size and computational performance overhead, a certain size of training graph is loaded each time. In this embodiment, the maximum number of nodes in the source graph for training at one time is set to 5000.

[0082] 2) The inductive graph neural network GraphSAGE is used to aggregate all neighbors of the central node in each training graph. Nodes are randomly selected and a sampling neighbor hop count k is set. In this embodiment, k is set to 2. All neighbors in each layer are sampled, and the remaining nodes are repeatedly randomly sampled to ensure that the input training samples reach a certain size each time, ensuring that all nodes are sampled. Simultaneously, self-loops are set for each node in the source graph, i.e., nodes are connected to themselves, to avoid information loss during message transmission. In this embodiment, two GraphSAGE convolutional layers are set, and each layer samples and aggregates all nodes.

[0083] 3) Fully connected layer: embeds the center node of the aggregated training graph into a vector. Add the self-embedding vector before the update By using a fully connected layer with a non-linear activation function σ, local features are integrated into global features of nodes, thereby enhancing the network's expressive power and complexity by introducing a non-linear activation function.

[0084] 4) Calculate the multi-class cross-entropy loss function:

[0085]

[0086] Where n is the total number of nodes in a batch of activities, and in this embodiment, the total number of nodes is 5000. ic This is the true category. In this embodiment, as described in S3, the node granularity is divided according to the absolute path root attribute information of the node. When the node category equals c, y... ic If the loss value is 1, then the weight matrix is ​​updated by gradient descent, otherwise 0. This allows us to learn different types of benign nodes.

[0087] 5) Save the trained model for subsequent detection of abnormal nodes.

[0088] The above five steps constitute one training iteration of the model. Compared to traditional neural network training, this invention argues that it is extremely difficult for a single neural network model to detect all anomalous node features. Therefore, it proposes an incremental training method. Figure 3 This is a schematic diagram of the incremental training process in an example of the present invention. The specific steps are as follows:

[0089] 1) First, maintain an empty list X in memory to store nodes that have not yet been correctly classified. In this embodiment of the invention, all untrained nodes in a batch are first initialized into the active list X.

[0090] 2) After the active list is aggregated k times by GraphSAGE, the first sub-model M1 is obtained. For details, please refer to the five detailed steps of the training process of the above model.

[0091] 3) Detect all nodes in X using sub-model M1. Set the confidence level to R during detection. If the maximum predicted probability exceeds the second largest predicted probability R, the classification is considered correct, as shown in the following formula.

[0092]

[0093] Where C(v) is the final label of node v, and y is the value used in the CrossEntropyLoss function above. ic Similarly, L(v) is the predicted label of node v, and M... p(v) It is the maximum probability value of the predicted node in model M, M p′(v) This is the second-highest probability value predicted by model M for a node. Nodes exceeding the confidence level R are correctly classified and are removed from X. The remaining nodes in X are then added to the next batch of training graphs. In this embodiment, the confidence level varies for different training datasets. The model trained with a confidence level of R=1 on the DARPA TC dataset has higher detection accuracy.

[0094] 4) Continuously add new training images to X. At this point, X contains both nodes that were misclassified in the previous step and new, untrained nodes. During training, incorrectly classified nodes are repeatedly reinforced. Each training iteration reduces the number of incorrectly classified nodes in the X list. This process is repeated until the X list is empty, indicating that each node has been learned and correctly classified. At the end of each training batch, a sub-model is obtained. All sub-models from the training process are saved, resulting in a sub-model set M. s .

[0095] 5) Using log data containing malicious attacks as input, a test graph is obtained after source graph construction and feature extraction in steps S1, S2, and S3. Detection is then performed on the test graph. Similar to the training phase, an empty subgraph is maintained initially, and newly arriving nodes and edges are continuously added. When the test graph size equals s, it is processed as a batch and sequentially through the sub-models for detection. An abnormal node cannot be confidently classified into a category in every sub-model; therefore, a detection confidence level R is set here. t If the probability predicted by all sub-models does not exceed this threshold, the node is determined to be an anomalous node. In other words, if a node is correctly classified in any sub-model, then the node is a benign node. This detection method greatly reduces the number of false positives and decreases the false alarm rate.

[0096] S5: To identify individual anomalous nodes, the malicious nodes in the training dataset need to be expanded. This invention uses a k-order expansion method to expand malicious nodes, setting a threshold k such that all neighbors within k hops of an anomalous node are considered suspicious nodes. This embodiment uses a breadth-first search k-order expansion algorithm to expand the initial malicious nodes. For the source graph G = (V, E), where V is the node set and E is the edge set, the initial set of anomalous nodes S = {s1, s2, ... s} n},s j For a node ∈ V, its neighboring nodes N(s) = {t, (s,t) ∈ E}. First, calculate the set of first-order suspicious nodes.

[0097]

[0098] Then continue to extend to the kth order:

[0099]

[0100] Finally, a set of abnormal nodes is obtained.

[0101] Write the abnormal nodes detected on the test graph in S4 to the alarm file, and combine them with... Accuracy, Precision, Recall, and False Positive Rate (FPR) are calculated together.

[0102]

[0103] FP represents false positive data, i.e., benign nodes on the source tracing graph (not included in the data). (In the middle) is detected as anomaly by the model. TN represents true negative data, meaning benign nodes on the source tracing graph are detected as normal by the model. TP represents true positive data, meaning malicious nodes on the source tracing graph (included in...) (In the middle) is detected as abnormal by the model. FN indicates false negative data, that is, malicious nodes on the source graph are detected as normal by the model.

[0104] The table below shows the test results for different DARPA TC datasets in this embodiment:

[0105] Table 2 DARPA TC Dataset Test Results

[0106]

[0107]

[0108] S6: Attack scenario reconstruction is performed using a method based on POI (Point of Interest) event propagation. Figure 4 The specific steps are shown.

[0109] First, we analyze the alarm file generated after S5 ends. The alarm file contains multiple sets of abnormal nodes. We select one POI event, which could be the modification of a sensitive file, the creation of a suspicious file, or access from an abnormal IP. A POI event is an alarm in the form of a 7-tuple (sourceIP, sourcePort, targetIP, targetPort, TTP, timestamp, priority). Here, sourceIP and sourcePort represent the suspicious source IP and port number, targetIP represents the target IP and port number, TTP represents the attack method, timestamp represents the timestamp, and priority represents the priority. The higher the value, the higher the alarm priority. The value range is 1-10. Based on the IP address and port number in an alarm event, we can uniquely match the network connection node v in the source graph. In the source graph, a network connection node is connected to a unique Subject entity, and its associated event is the POI event in the form of a four-tuple (source, target, timestamp, type), which serves as the entry point for attack investigation.

[0110] After uniquely identifying the entry point on the source graph based on the POI event, an initial dependency graph related to that node is generated. At this point, the dependency graph will be very large, requiring edge aggregation to eliminate duplicate events. This embodiment merges different versions of process entities on the dependency graph and distinguishes different events by marking timestamps on the edges, making the dependencies between nodes clearer.

[0111] After completing the dependency graph aggregation operation, the entire attack scenario is reconstructed starting from the entry point. This embodiment uses a label propagation method, first marking POI events as suspicious events, and then propagating the suspicious labels from both the reverse and forward directions to generate an attack dependency graph G.Dep Before backpropagation, all events on the dependency graph need to be extracted in reverse chronological order. Starting from the latest event, events directly related to POI events are continuously marked as suspicious events and added to the POI event set. Forward propagation follows the same process, searching for other events related to POI events in the forward-ordered event set to construct the final attack dependency graph, thus reconstructing the entire attack scenario.

[0112] The proposed anomaly node detection method based on source graph does not rely on prior attack knowledge or predefined rule bases. It learns benign data in an unsupervised manner and has significant advantages under the condition that attack data is very difficult to obtain.

[0113] This invention proposes a novel node feature encoding method that considers both the semantic and structural information of nodes and effectively aggregates neighbor node features through GraphSAGE. GraphSAGE is a type of inductive graph neural network. Compared to traditional transductive graph neural networks, such as GCN and GAT, inductive graph neural networks solve the problem of inefficient model learning caused by the massive amount of data in source graphs.

[0114] This invention proposes an attack scenario reconstruction method based on POI event propagation. By tracing POI events from both forward and reverse directions, the method reconstructs key attack scenarios and transforms discrete attack alarm nodes into a connected attack scenario graph. This allows security analysts to proactively defend against attack behaviors through visualization.

[0115] This invention considers both the structural information of the source graph and the semantic information of the nodes on the source graph, and proposes a novel node feature encoding method. This encoding method for different dimensions of node information can help graph neural networks to deeply understand the internal operating characteristics of the system.

[0116] This invention innovatively proposes an intrusion detection algorithm based on GraphSAGE: Previous inventions mostly utilize transductive graph neural networks to learn source graphs, such as GCN and GAT, neglecting the inefficiency caused by the massive amount of data in the source graph. This invention applies GraphSAGE to source graph learning. GraphSAGE is an inductive graph neural network that uses neighbor sampling for batch learning, improving the model's learning efficiency. By employing incremental learning to obtain a large number of sub-models and using all sub-models for anomaly node detection, the high false positive rate faced by node-level intrusion detection systems can be significantly reduced.

[0117] Finally, it should be noted that the above specific embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail in the specific embodiments, those skilled in the art can easily understand that they can still modify or make equivalent substitutions to the technical solutions proposed in the present invention, and such modifications or equivalent substitutions should all be included within the protection scope of the present invention.

Claims

1. An intrusion detection method based on source graphs, characterized in that, Includes the following steps: S1: Take the raw log dataset that does not contain malicious attacks as input, parse the logs and construct an initial system-wide source graph; S2: Delete redundant nodes and edges on the initial system-wide source graph; reduce the size of the source graph; S3: Integrate specific domain knowledge into the node feature construction and graph representation process, and generate initial embeddings for nodes on the traceability graph from multiple dimensions, which serve as inputs for subsequent neural network models; S4: Construct a neural network model based on GraphSAGE and train it using incremental learning; After training, nodes are classified on the source graph. Nodes that are not correctly classified on the source graph are repeatedly trained, i.e., loaded into the next round of training graph, while correctly classified nodes are not trained again. After multiple rounds of training, all nodes are correctly classified. The sub-models obtained from each training are saved and used together for abnormal node detection. S5: Using log data containing attack traces as input, construct a source graph in real time, continue to load multiple sub-models trained by the GraphSAGE-based neural network model in S4, detect the source graph constructed in real time, and write the identified single abnormal node on the source graph into the alarm file; S6: Use abnormal node alerts as entry events for attack investigation to reconstruct the attack scenario.

2. The intrusion detection method based on a source map according to claim 1, characterized in that, In S1, the data source for constructing the source graph is computer audit logs. For unstructured logs, the entities appearing in the logs are abstracted into three types: processes, files, and network streams, which are used as nodes on the source graph. The abstracted event types are used as the features of the edges between nodes, and finally the log text data is used to construct a source graph for the entire system.

3. The intrusion detection method based on a source map according to claim 2, characterized in that, In S2, the first step is to reduce the structure and eliminate isolated nodes in the initial system-wide source graph; Secondly, starting from semantic reduction, similar operations between two nodes are merged to eliminate redundant edges in the source graph, making the source graph more compact. Finally, based on specific domain knowledge, a whitelist process is set up to filter out nodes with high credibility and known security from the source map.

4. The intrusion detection method based on a source map according to claim 3, characterized in that, In S3, three different levels of node features are considered, including the executable permissions of process nodes, the spatial information of nodes, and the semantic features of nodes. The vulnerability information of the process is encoded into a 16-dimensional multi-hot vector and added to the embedding vector of the process node; The spatial information of a node is the connection relationship between the node and its neighbors. These connections reflect the node's position in the system and its interaction with other nodes. In the graph data structure, the spatial characteristics of a node are effectively represented by statistically analyzing the in-degree and out-degree information of the node on the entire system's source graph. By extracting the absolute path information of different types of entities, the root directory attribute of the path is added to the node type.

5. The intrusion detection method based on a source map according to claim 4, characterized in that, In S4, an unsupervised network model based on GraphSAGE is constructed. The model consists of two parts: a GraphSAGE neighbor aggregation module and an incremental training module. The GraphSAGE neighbor aggregation module is used to aggregate neighbor features for each node in the source graph, and efficiently generate node embedding representations with contextual semantics through message passing and neighbor sampling. The incremental training module is used to improve learning efficiency by learning the large source graph multiple times and using multiple sub-models for detection.

6. The intrusion detection method based on a source map according to claim 5, characterized in that, When using GraphSAGE to aggregate the neighbors of a central node, k aggregators are set to aggregate the neighbor features of the central node, and the axis is the node's initialization vector x. v Assign to the first hidden layer vector Iterate through the k-hop neighbors and use an aggregator function to aggregate neighbor information to update the embedding of each neighbor node at each level: Where N(v) represents all neighboring nodes of v, and AGGREGATE represents an aggregator function. During training, various aggregators are selected to embed the updated nodes from the k-th layer. In addition to the self-embedding vector before the update Then pass it through a fully connected layer with a non-linear activation function σ: The above steps aggregate k-order neighbors. Each aggregation represents a parameter weight update, and a total of k-1 updates are performed to obtain the final output feature z. v ,in Then, aggregate the results z from the last time for each central node. v After softmax normalization, the calculation formula is as follows: Where M is the number of node categories; The probability of obtaining the category Next, the CrossEntroyLoss function is used to calculate the loss between the expected label and the actual label of the node, as follows: Where n is the total number of activity nodes in a batch, y ic It is the true category; when the node category equals c, y ic If the loss value is 1, then the weight matrix is ​​updated by gradient descent, otherwise 0. This allows us to learn different types of benign nodes.

7. The intrusion detection method based on a source map according to claim 6, characterized in that, The incremental training module divides the training graph into multiple subgraphs for training, sets a maximum number of active nodes per batch to ensure scalability, and obtains a set of sub-models through multiple training iterations. When identifying abnormal nodes, all sub-models must be used. The incremental training steps are as follows: First, maintain an empty list X in memory to store nodes that have not yet been correctly classified. Initialize untrained nodes from a batch into the active list X. After k aggregations by GraphSAGE, obtain the first sub-model M1, which detects all nodes in X. During detection, set the confidence level to R. If the maximum predicted probability exceeds the second-largest predicted probability R, the classification is considered correct. Where C(v) is the final label of node v, L(v) is the predicted label of node v, and M... p(v) M is the maximum probability of the predicted node in model M. p′(v) The second highest probability predicted by model M is to remove correctly classified nodes from X. The remaining nodes in X will then be added to the next batch of training graphs. Continuously add untrained nodes to X, repeating the above steps until the list of X is empty, which means that each node has been learned and correctly classified. Nodes that are not correctly classified during training will be repeatedly reinforced, and a sub-model will be obtained at the end of each training batch, ultimately resulting in a set of sub-models M. s ; During the testing phase, an empty subgraph is maintained, and newly arrived nodes and edges are continuously added. When the size of the test graph equals s, it is treated as a batch and sequentially passed through the sub-model for testing. A detection confidence level R is set. t If the probability predicted by all sub-models does not exceed this threshold, the node is judged as an abnormal node. If a node is correctly classified in any sub-model, the node is a benign node.

8. The intrusion detection method based on a source map according to claim 7, characterized in that, In the anomaly detection phase of S5, a k-order expansion method is used to expand malicious nodes. A threshold k is set, which considers all neighbors within k hops of an abnormal node to be suspicious nodes. A breadth-first search k-order expansion algorithm is used to expand the initial malicious nodes. For the source graph G = (V, E), where V is the node set and E is the edge set, the initial set of abnormal ground truth nodes S = {s1, s2, ... s} is given. n },s j For a node ∈ V, its neighboring nodes N(s) = {t, (s,t) ∈ E}. First, calculate the set of first-order suspicious nodes. The formula is as follows: Then continue to extend to the kth order: Finally, a set of abnormal nodes is obtained.

9. The intrusion detection method based on a source map according to claim 8, characterized in that, S6 uses a POI event propagation-based method to reconstruct attack scenarios; First, analyze the alarm files generated after S5 ends, select one POI event, such as the modification of a sensitive file, the creation of a suspicious file, or access by an abnormal IP, and generate an initial dependency graph related to this POI event. Next, perform edge aggregation on the dependency graph to eliminate duplicate events. Finally, starting from the POI event, trace the source in both the forward and reverse directions on the simplified dependency graph to obtain the complete attack scenario.

10. The application of the method according to any one of claims 1-9, characterized in that, It is applied to host-based intrusion detection systems and is suitable for environments that require processing large amounts of log information or for host intrusion detection and abnormal node identification based on source graphs.

Images