APT attack detection method and system based on mask strategy and graph auto-encoder

Abstract

The invention discloses an APT (Advanced Persistent Threat) attack detection method based on a mask strategy and a graph auto-encoder. The method comprises the following steps: 1, constructing a log traceability graph, and carrying out noise reduction on the log traceability graph; 2, nodes needing to be masked in the log traceability graph are masked through a mask node selector; step 3, training a mask graph auto-encoder, and representing features through a corresponding benign graph obtained by the graph auto-encoder; 4, constructing a corresponding to-be-detected log traceability graph, and performing noise reduction on the log traceability graph; establishing a mapping relation between the traceability graph node and the system log entity; 5, obtaining graph representation features of the nodes to be detected; 6, calculating an abnormal score, and judging whether a node corresponding to the graph representation feature is abnormal or not according to whether the score exceeds an abnormal threshold or not; and step 7, obtaining abnormal entity entries which are entity entries detected by the method and related to the APT attack. And a better APT attack detection effect is obtained.

Description

Technical Field

[0001] This invention belongs to the fields of network security and deep learning technology, specifically relating to an APT attack detection method and system based on masking strategy and graph autoencoder. Background Technology

[0002] With the development of the internet and technology, cybersecurity issues have become increasingly significant, and the detection of cyberattacks has become a crucial topic in the cybersecurity field. APT (Advanced Persistent Threat) attacks, also known as advanced persistent threats, are highly stealthy, lurking in target systems for extended periods. They exhibit greater persistence than ordinary cyberattacks and employ more advanced techniques, making them difficult for traditional intrusion detection and prevention systems to effectively detect. Early APT attack detection methods summarized the technical methods and attack processes of APT attacks, constructing corresponding knowledge bases and frameworks to detect potential APT attacks in systems.

[0003] With the development of machine learning and deep learning technologies, methods for detecting APT attacks based on system logs and employing machine learning or deep learning techniques have emerged. System log-based methods detect APT attack behavior in a system by extracting information from the system logs, thus obtaining more information. Methods like DeepLog directly detect APT attacks based on system logs and have good detection results. However, due to the lack of causal analysis, they do not fully utilize the effective information in the system logs.

[0004] With the development of graph neural network technologies, some APT attack detection methods with better detection performance have emerged, such as the Unicorn method. These methods obtain log source graphs by parsing system logs and perform APT attack detection based on these graphs. They fully utilize the effective information in the log source graphs while considering the causal relationships between entities in the system logs. These methods generally use graph representation learning methods to represent nodes in the log source graph as graph representation features, and then use anomaly detection methods to detect anomalies in the graph representation features to determine whether a node is related to an APT attack. However, these methods rarely perform fine-grained APT attack detection. Fine-grained APT attack detection methods, such as the MAGIC method based on masked graph autoencoders, use a combination of random masks and graph autoencoders in their graph representation model. However, in APT attack detection, they do not consider the uncertainties introduced by random masks and the uniformity of node feature distribution, and therefore still have room for improvement. In addition, some graph representation learning methods with masking strategies have emerged in current graph representation learning research, such as the AUG-MAE method. While incorporating a masking strategy into the graph autoencoder, it can consider the uniformity of node feature distribution and has the potential to be applied in APT attack detection.

[0005] Currently, log source graph-based APT attack detection methods rarely perform fine-grained APT attack detection, and the graph representation methods used in common fine-grained detection methods still have room for improvement. To address these issues, how to adopt a more effective graph representation model for log source graph-based APT attack detection methods and perform fine-grained APT attack detection is a problem that needs to be solved. Summary of the Invention

[0006] To overcome the shortcomings of the existing technology, the present invention aims to provide an APT attack detection method and system based on masking strategy and graph autoencoder. This method employs a more effective graph representation method for fine-grained APT attack detection. It obtains the graph representation features of log source graph nodes based on a graph autoencoder with masking strategy, while considering the influence of structural information and node feature uniformity. Furthermore, it performs anomaly detection on the graph representation vector based on common anomaly detection models such as the K-nearest neighbor model, thereby determining whether nodes in the log source graph are abnormal. Finally, it detects abnormal entity entries in the system log based on the mapping relationship between nodes and log entities, achieving better APT attack detection results.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] An APT attack detection method based on masking strategy and graph autoencoder includes the following steps;

[0009] Step 1: Obtain healthy system log data and extract entity and event attributes from it, construct a log source graph, and reduce noise in the log source graph, such as merging edges of the same type, to reduce the complexity of the log source graph;

[0010] Step 2: Based on the denoised log source map obtained in Step 1, select the nodes that need to be masked using a masking strategy; and mask the nodes that need to be masked in the log source map using a mask node selector.

[0011] Step 3: Input the masked benign log source graph obtained in Step 2 into the masked graph autoencoder to train the masked graph autoencoder. During the training process, update the parameters of the mask node selector in Step 2. After training, save the trained masked graph autoencoder model parameters and the benign graph representation features corresponding to the benign source graph nodes obtained by the graph autoencoder from the benign source graph nodes in the training data.

[0012] Step 4: Extract the system log data to be detected. Extract the entity attributes and event attributes in a similar way to Step 1, construct the corresponding log source graph to be detected, and process it in the same way as in Step 1. At the same time, establish the mapping relationship between the log source graph nodes and the system log entities.

[0013] Step 5: Input the source graph of the log to be detected into the trained masked graph autoencoder model to obtain the graph representation features of the node to be detected;

[0014] Step 6: Input the benign graph representation features obtained in Step 3 into the anomaly detection model as training data for the anomaly detection model, and input the graph representation features of the node to be detected obtained in Step 5 into the anomaly detection model. Calculate its anomaly score and determine whether the node corresponding to the graph representation feature is anomaly based on whether the score exceeds the anomaly threshold. Nodes corresponding to graph representation features with anomaly scores exceeding the anomaly threshold are considered as anomaly nodes.

[0015] Step 7: Based on the mapping relationship between log source graph nodes and system log entities obtained in Step 4, and combined with the abnormal nodes identified in Step 6, obtain the abnormal log entities corresponding to the abnormal nodes, and return the entity log entries corresponding to the abnormal entities. The obtained abnormal entity entries are the entity entries related to APT attacks detected by this method, which is the final result obtained by this method. Analysts can further determine the type and purpose of APT attacks based on these entries.

[0016] Furthermore, in step 1, the construction of the log source graph includes three parts: log feature extraction, source graph construction, and source graph feature encoding.

[0017] The specific operation of log feature extraction is as follows: first, distinguish each entity and event in the system log data, and then use the log parsing program to extract the unique identifier number, category, IP address and timestamp information corresponding to the entry as the features of the log entry;

[0018] The specific operations for constructing the source graph are as follows: entity entries in the system log are constructed into nodes, and the node features are entity log features extracted from log feature extraction; event entries in the system log are constructed into edges, and the edge features are event log features extracted from log feature extraction.

[0019] The specific operation of source graph feature encoding is as follows: the information corresponding to nodes and edges is initially encoded by one-hot encoding, and then the source graph is denoised. The denoising process is an operation performed on the edges, including deleting redundant edges of the same type between the same node pairs and merging edges of different types between the same node pairs.

[0020] Furthermore, in step 2, the masking strategy for log source graph nodes is a combination of random masking and learning-based masking. The random masking method selects nodes to be masked by random selection, while the learning-based masking method selects nodes to be masked by a mask node selector with learnable parameters.

[0021] The method based on random masking involves randomly selecting nodes from the source graph at a certain ratio k, and forming a random mask node set. Learning-based masking uses a mask node selector. Φ The selection of mask nodes is controlled, and the mask nodes selected in this way form a learning-based mask node set.

[0022] The learning-based masking method is as follows:

[0023] Learning-based masking methods select mask nodes using a learnable mask node selector. Φ Composed of a graph neural network and fully connected layers, it can obtain the probability (Prob) that each node i is masked after inputting the log source graph. i The probability is used to determine whether each node should be masked, and nodes that need to be masked are added to the learning-based mask set. In Chinese, the formula is as follows:

[0024]

[0025] In the formula, mask i mask is either 0 or 1. i A value of 1 indicates that node i needs to be added to the set of learning-based mask nodes. In the middle, otherwise not required, depending on the mask. i It can determine whether node i needs to be masked, ∈0, ∈1 are random noise sampled from Gumbel(0,1), τ is a hyperparameter, and σ is the Sigmoid function.

[0026] Furthermore, in step 2, the masking strategy for the log source graph nodes is a combination of random masking and learning-based masking, specifically:

[0027] The set of nodes obtained by using a random mask. The learning-based mask node set obtained by the learning-based masking method They need to be merged into a mask node set in the following way. Random mask node set and learning-based mask node set The number of nodes in the mask node selector is determined by the number of training iterations t, as shown in the following formula:

[0028]

[0029] In the formula, α lr (t) represents the mask weights calculated by the mask graph autoencoder in the t-th training epoch, T represents the total number of training epochs of the mask graph autoencoder, ω is a hyperparameter used to control the mask weight growth rate, and α0 and α T A real number between 0 and 1, indicating that the mask weight gradually increases from α0 to α T ;

[0030] As the number of training iterations increases, α lr (t) continuously increases from α0 to α T , The number of nodes gradually increases, and the final mask node set in each training round... middle, Randomly select from The final set of random mask nodes is composed of these nodes. express The number of elements in; Randomly select one of them The final learning-based mask node set is composed of these nodes. The set of mask nodes in the t-th training round

[0031] Obtain the set of mask nodes Then, the log source graph is compared with the mask node set. The initial features of the nodes corresponding to the nodes in the graph are masked. After the masking is completed, the masked log source graph can be obtained.

[0032] Furthermore, in step 3, the mask image autoencoder and its training process, as well as the parameter update of the mask node selector, are detailed as follows:

[0033] The masked graph autoencoder consists of a graph encoder and a graph decoder. During the training process of the masked graph autoencoder, the feature uniformity loss and structural reconstruction loss of the unmasked nodes obtained by the graph encoder are calculated. The features of the masked nodes are reconstructed by the graph decoder, and the feature reconstruction loss of the masked nodes is calculated. The masked graph autoencoder updates its parameters based on the feature uniformity loss of the unmasked nodes, the structural reconstruction loss, and the feature reconstruction loss of the masked nodes. The masked node selector in step 2 updates its parameters based on the feature reconstruction loss of the masked nodes. After training is completed, the model parameters are saved, and the graph representation features of the benign nodes obtained by the masked graph autoencoder are also saved.

[0034] The graph encoder encodes the masked log source graph data using a graph neural network to obtain the graph representation features h of the log source graph nodes. v The graph decoder reconstructs the reconstructed features of the mask nodes by reconstructing the graph representation features of the mask nodes.

[0035] Feature reconstruction loss L of mask nodes feature The initial feature x of the mask node v v With reconstruction features The calculation yielded:

[0036]

[0037] In the formula Let γ be the set of mask nodes, and γ be a hyperparameter.

[0038] The update of the mask node selector parameters specifically refers to the mask node selector. Φ The update of parameter Φ needs to be based on the mask loss L mask Perform, mask loss L mask The loss L is reconstructed from the features of the masked nodes. feature The calculation is as follows:

[0039]

[0040] In the formula, λ1 is the weight used to control the proportion of mask nodes, and N is the number of nodes in the log source graph;

[0041] The uniformity loss of unmasked nodes is derived from the graph representation feature set of unmasked nodes obtained by the graph encoder. The calculation yielded:

[0042]

[0043] In the formula, λ is the uniformity loss weight of the node graph representation feature, and α lr is the mask weight obtained in step 2, and n represents the number of nodes in X.

[0044] In step 3, the structural reconstruction loss of the unmasked nodes is the graph representation feature set of the unmasked nodes obtained by the graph encoder. The results were obtained through structural reconstruction based on sample pairs of sampling nodes, as detailed below:

[0045] Structural reconstruction based on sampled pairs requires random sampling of positive and negative sample pairs. Node pairs with edges in the log source graph are called positive sample pairs, denoted as (v, v). + ), and vice versa, are called negative sample pairs, denoted as (v, v). - For sample pairs (v, v) * The graph representation features h are obtained by using a multilayer perceptron (MLP). v and The probability of an edge existing between sample pairs is predicted, and the structural reconstruction loss between sample pairs is calculated using the cross-entropy loss function, as shown in the following formula:

[0046]

[0047] In the formula, This represents the set of unmasked nodes, where [a,b] represents the concatenation of a and b.

[0048] The mask image autoencoder is based on the aforementioned uniformity loss L uniform Structural reconstruction loss L struct and feature reconstruction loss L feature The parameters are updated by summing the results, and the final loss L is calculated using the following formula:

[0049] L = L struct +L uniform +L feature .

[0050] Furthermore, in step 3, the specific details after the mask image autoencoder model training is completed are as follows:

[0051] After training, the masked graph autoencoder model saves its model parameters for subsequent steps. Furthermore, the training data obtained in step 1, i.e., the benign log source graph, is directly converted into graph representation features by the graph encoder of the trained masked graph autoencoder model without undergoing the masking operation in step 2, and these graph representation features are saved.

[0052] In step 4, the method of constructing and denoising the log source graph is the same as in step 1. The only difference is that in step 4, the log data to be detected is constructed into a log source graph, while in step 1, the benign log data is constructed into a log source graph.

[0053] In step 4, a mapping table between log source graph nodes and system log entities is constructed based on the unique identifiers of entities in the system log. This facilitates querying the mapping relationship between log source graph nodes and system log entities, thereby identifying abnormal log entity entries from abnormal log source graph nodes and achieving fine-grained APT attack detection.

[0054] In step 5, the log source map to be detected does not need to be masked. It is directly input into the trained masked graph autoencoder model saved in step 3. The graph encoder encodes the log source map to be detected to obtain the graph representation features of each node in the log source map to be detected.

[0055] In step 6, the anomaly detection model has several options. One option is provided below, but it is not intended as an additional limitation on the overall approach described above. The anomaly detection model uses the K-nearest neighbor model to detect anomalies in the graph representation features. The specific details are as follows:

[0056] In step 6, when using the K-nearest neighbor model as the anomaly detection model, the step of inputting the graph representation features corresponding to the benign source tracing graph nodes into the anomaly detection model as training data specifically involves:

[0057] The benign node graph representation features obtained in step 3 are stored in a KD-tree as training data for K-nearest neighbor anomaly detection. This data is used to search for the K nearest neighbor features of the feature corresponding to the node to be detected, and the average distance d between each benign feature h∈H and its K nearest neighbor features in the KD-tree is calculated. h and d of all benign characteristics h By averaging, a distance variable is obtained that summarizes the range of benign characteristics. The calculation formula is as follows:

[0058]

[0059] When using the K-nearest neighbor model as the anomaly detection model, the anomaly score is calculated based on the graph representation features of the node to be detected, specifically as follows:

[0060] The nodes in the source-tracing graph to be detected are input into the K-nearest neighbor algorithm, and each node v is searched among all benign features of the KD tree. i The features are the K nearest neighbor benign features in the KD-tree and form a set. And calculate Features and nodes v i Features Calculate the distance between them and obtain the average value. The formula is as follows:

[0061]

[0062] The anomaly detection model will perform anomaly detection on each node v i Calculate abnormal scores The formula is as follows:

[0063]

[0064] When using the K-nearest neighbors model as the anomaly detection model, nodes corresponding to graph representation features whose anomaly scores exceed an anomaly threshold are considered anomalous nodes. Specifically, an anomaly threshold θ is determined, and if a node v i abnormal scores If the value is greater than the abnormal threshold θ, then the node v i It will be considered an abnormal node.

[0065] If there are too many nodes to be detected, in order to speed up the detection speed of the method, a portion of the benign node graph representation features obtained in step 3 are selected to train the anomaly detection model instead of all the benign node graph representation features. The detection speed is accelerated by reducing the number of K-nearest neighbor searches, but this method may reduce the detection effect.

[0066] In step 7, the mapping relationship between nodes and log entities is as follows: the unique identifier of the log source graph node in step 4 is the same as the unique identifier in the log entity attribute. The log source graph node and the log entity can be matched through the unique identifier.

[0067] Another objective of this invention is to provide an APT attack detection system based on masking strategies and graph autoencoders, comprising:

[0068] Memory is used to store system log data and instructions executed by the processor;

[0069] A processor for executing the instructions to implement the method described above.

[0070] The beneficial effects of this invention are:

[0071] (1) This invention proposes an APT attack detection method based on masking strategy and graph autoencoder. The log source graph nodes are masked by two methods: random masking and learning-based masking, in order to train the graph autoencoder. After the graph autoencoder is trained, an anomaly detection model is used to detect anomalies in the graph representation features. Compared with the existing APT attack detection method based on masked graph autoencoder, this method reduces the randomness brought by simple random masking, and allows the masking method to be adjusted by the graph autoencoder during training, which can achieve better APT attack detection results.

[0072] (2) In the training process of the mask graph autoencoder, this invention makes full use of the structural and attribute information of the log source graph and considers the uniformity of the graph representation feature distribution of the log source graph nodes. This makes the graph representation features of benign nodes more uniformly distributed because the mask graph autoencoder model introduces uniformity loss during its training process and the training data of the mask graph autoencoder only uses benign source graph data. This will reduce the differences between benign node features and decrease the probability of benign node features being falsely detected as anomalies, which is beneficial for the model to judge the anomalous features of APT attacks. Attached Figure Description

[0073] Figure 1 This is a schematic diagram illustrating the workflow of the APT attack detection method based on masking strategy and graph autoencoder according to an embodiment of the present invention.

[0074] Figure 2 This is a schematic diagram of the mask node selector structure used in the APT attack detection method based on masking strategy and graph autoencoder in an embodiment of the present invention.

[0075] Figure 3 This is a schematic diagram illustrating the basic framework and workflow of the graph representation learning model used in the APT attack detection method based on masking strategy and graph autoencoder in this embodiment of the invention.

[0076] Figure 4 This is a schematic diagram of the modules of the APT attack detection system based on masking strategy and graph autoencoder according to an embodiment of the present invention. Detailed Implementation

[0077] The present invention will now be described in further detail with reference to the accompanying drawings.

[0078] An APT attack detection method based on masking strategy and graph autoencoder, such as Figure 1 As shown, the specific steps include:

[0079] Step 1: Obtain healthy system log data and extract the attributes of entities and events, construct a log source graph, and perform noise reduction on the log source graph in some ways, such as merging edges of the same type, to reduce the complexity of the log source graph.

[0080] Step 1 specifically includes the following steps:

[0081] Step 1-1: Extract entity and event attributes from system logs. When extracting entity and event attributes from system logs, entity attributes include the entity type, the entity's unique identifier (UUID), and the entity's name. Common entity types include processes, files, and network stream objects. The entity's unique identifier is used to construct the mapping relationship between log entities and traceability graph nodes. Event attributes include the event type, the event's originating entity (source), the event's target entity (target), and the event's timestamp.

[0082] Step 1-2: Construct a log source graph based on the entity and event attributes extracted in Step 1-1. When constructing the source graph, first build the nodes based on the entity information. Then, based on the unique identifiers of the originating and target entities of the events, construct edges to connect the nodes in the source graph according to the event relationships in the system logs, thereby constructing the log source graph.

[0083] Steps 1-3: Perform one-hot encoding and noise reduction on the log source graph. After constructing the source graph, it is necessary to perform one-hot encoding on the source graph according to the types of nodes and edges in the source graph to obtain the initial features x of the nodes and edges in the log source graph. n and e n This facilitates subsequent masking and reconstruction of node features. After one-hot encoding, some noise reduction processing is needed on the source graph, mainly involving operations on edges, such as deleting redundant edges of the same type between the same node pairs and merging edges of different types between the same node pairs. When merging edges of different types between the same node pairs, the encoding corresponding to the edges is merged by averaging.

[0084] Step 2: Based on the log source graph obtained in Step 1, mask nodes are selected by combining random masking and learning-based masking strategies, and the nodes in the log source graph that need to be masked are masked.

[0085] Step 2 specifically includes the following steps:

[0086] Step 2-1: Obtain a randomized masked node set using a random masking method. The random masking method involves randomly selecting nodes from the node set at a certain ratio k. In this embodiment, k = 0.5. The set of nodes obtained through the random masking method is represented as follows:

[0087] Step 2-2: Obtain a learning-based mask node set using a learning-based masking method. The learning-based masking method uses a mask node selector with a learnable parameter Φ. Φ Select the mask node. A schematic diagram of the mask node selector is shown below. Figure 2 As shown, this embodiment uses a graph attention network, which ultimately obtains the probability Prob that each node i is masked. i The probability is used in conjunction with the following formula to determine whether each node needs to be masked, and nodes that need to be masked are added to the learning-based mask set. middle.

[0088]

[0089] In the formula, mask i mask is either 0 or 1. i A value of 1 indicates that node i needs to be added to the set of learning-based mask nodes. In the middle, otherwise not needed. (mask) i It can determine whether node i needs to be masked, ∈0, ∈1 are random noise sampled from Gumbel(0,1), τ is a hyperparameter, and σ is the Sigmoid function.

[0090] Steps 2-3: Using a random mask node set and learning-based mask node set Obtain the mask node set for each training round. The set of nodes obtained by using a random mask. The learning-based mask node set obtained by the learning-based masking method They need to be merged into a mask node set in the following way.

[0091] The merging method requires calculating the mask weight α. lr This weight is related to, for example Figure 3 The diagram shown is related to the training epoch t of the model. In this embodiment, α0 and α T A real number between 0 and 1, indicating that the mask weight gradually increases from α0 to α T .

[0092]

[0093] In the formula, α lr (t) indicates that, as Figure 3The diagram shown represents the mask weights calculated by the model in the t-th training epoch, where T represents the total number of training epochs and ω is a hyperparameter used to control the growth rate of the mask weights.

[0094] Random mask node set and The number of nodes in the system is determined by, for example Figure 3 The diagram shows that the number of training iterations for the model is fixed, and as the number of training iterations increases, α... lr (t) continuously increases from α0 to α T , The number of nodes gradually increases, and the final mask node set in each training round... middle, Randomly select one of them The final set of random mask nodes is composed of these nodes. Randomly select one of them The final learning-based mask node set is composed of these nodes. The set of mask nodes in the t-th training round

[0095] Steps 2-4: Mask the log source graph based on the contents of the mask node set. Obtain the mask node set. Then, the source map is compared with the mask node set. The initial features of the nodes corresponding to the nodes in the graph are masked. In this embodiment, the masking method is to set all feature values ​​to 0. After the masking is completed, the masked log source graph can be obtained.

[0096] Step 3: Input the masked benign log source graph obtained in Step 2 as follows: Figure 3 In the masked graph autoencoder of the graph representation learning model shown, the parameters of the mask node selector are updated during training. After training, the relevant model parameters are saved, and the benign source graph obtained in step 1 is input into the encoder to obtain the corresponding graph representation features, and these graph representation features are saved.

[0097] Step 3 specifically includes the following steps:

[0098] Step 3-1: Masking the Log Source Graph Input. Obtain the graph representation features of the nodes from the encoder of the masked graph autoencoder. Input the masked log source graph obtained in Step 2 into the following... Figure 3 The diagram illustrates a masked graph autoencoder in a graph representation learning model, which consists of an encoder and a decoder. First, a node v in the masked log source graph is encoded into a graph representation feature h by the encoder. vIn this embodiment, the encoder uses a graph attention network for encoding.

[0099] Step 3-2: The graph representation features of the masked nodes are obtained by decoding the input masked graph autoencoder. The reconstructed node features are then processed by the decoder of the input masked graph autoencoder. v After remasking, the data is input into the decoder, which decodes the graph representation features into the initial features x corresponding to node v. v Reconstruction features with the same structure In this embodiment, the decoder uses a graph attention network for decoding.

[0100] Step 3-3: Calculate the reconstruction loss L of node attributes feature And update the parameters of the mask node selector. Obtain the node features reconstructed by the decoder. Afterwards, the feature reconstruction loss L of the mask node feature The initial feature x of the mask node v v With reconstruction features The calculation yielded:

[0101]

[0102] In the formula Let be the set of mask nodes, and γ be a hyperparameter.

[0103] like Figure 3 As shown, the mask node selector. Φ The update of the intermediate parameter Φ requires reconstruction of the loss L based on the features of the mask nodes. feature Calculate the mask loss L mask Proceed, L mask The calculation formula is as follows:

[0104]

[0105] In the formula, λ1 is the weight used to control the proportion of mask nodes, and N is the number of nodes in the log source graph.

[0106] Steps 3-4: Calculate the uniformity loss of unmasked nodes. The uniformity loss of unmasked nodes is calculated from the graph representation feature set of unmasked nodes obtained by the graph encoder. The calculation yielded:

[0107]

[0108] In the formula, λ is the uniformity loss weight of the node graph representation feature, and α lr is the mask weight in step 2, and n represents the number of nodes in X.

[0109] Steps 3-5: Calculate the structural reconstruction loss of unmasked nodes. The structural reconstruction loss of unmasked nodes is calculated from the graph representation feature set of unmasked nodes obtained by the graph encoder. The following was obtained through structural reconstruction calculation based on sample pairs of sampling nodes:

[0110] Structural reconstruction based on sampled pairs requires random sampling of positive and negative sample pairs. Node pairs with edges in the log source graph are called positive sample pairs, denoted as (v, v). + ), and vice versa, are called negative sample pairs, denoted as (v, v). - For sample pairs (v, v) * The graph representation features h are obtained by using a multilayer perceptron (MLP). v and The probability of an edge existing between sample pairs is predicted, and the structural reconstruction loss between sample pairs is calculated using the cross-entropy loss function, as shown in the following formula:

[0111]

[0112] In the formula, This represents the set of non-masked nodes, where [a,b] represents the concatenation of a and b.

[0113] Steps 3-6: Calculate the training loss of the masked graph autoencoder model and update the parameters in the masked graph autoencoder. The masked graph autoencoder is based on the uniformity loss L described above. uniform Structural reconstruction loss struct and feature reconstruction loss L feature The sum of these parameters is used to update the parameters. The formula for calculating the training loss L of the mask image autoencoder model is as follows:

[0114] L = L struct +L uniform +L feature

[0115] Steps 3-7: After the masked graph autoencoder model is trained, save the model parameters and the corresponding graph representation features obtained by the graph autoencoder for the benign log source graph nodes in the training data. After training, save the model parameters for subsequent steps. Furthermore, the training data obtained in Step 1, i.e., the benign log source graph, is directly converted into graph representation features by the graph encoder of the trained masked graph autoencoder model without undergoing the masking operation in Step 2, and these graph representation features are saved.

[0116] Step 4: Extract the entity and event attributes from the system log data to be tested in a manner similar to Step 1, construct the corresponding log source graph, perform the same processing as in Step 1, and establish the mapping relationship between source graph nodes and system log entities.

[0117] Step 4 specifically includes the following steps:

[0118] Step 4-1: Extract the attributes of entities and events from the log data of the system to be tested, construct a log source graph in the manner of Step 1, and perform one-hot encoding and noise reduction on the nodes and edges in the source graph.

[0119] Step 4-2: Based on the unique identifier of the system log entity, and corresponding to the unique identifier of the node in the log source graph, construct a mapping relationship between source graph nodes and system log entities. This mapping relationship is used to achieve fine-grained APT attack detection; the abnormal log entity entry is determined by the abnormal log source graph node.

[0120] Step 5: Input the log source graph to be detected obtained in Step 4 into the trained masked graph autoencoder model obtained in Step 3 to obtain the graph representation features corresponding to each node to be detected. For example... Figure 3 As shown, in the model detection part, the log source graph to be detected does not need to go through the masking process. It is directly input into the encoder of the mask graph autoencoder to obtain the graph representation features corresponding to the nodes of the log source graph to be detected.

[0121] Step 6: Train an anomaly detection model based on the benign node graph representation features obtained in Step 3. In this embodiment, the K-nearest neighbor model is used to detect anomalies in the graph representation features of the node to be detected. The graph representation features of the node to be detected obtained in Step 5 are input into the anomaly detection model to calculate its anomaly score. The node corresponding to the graph representation feature is judged to be abnormal based on whether the score exceeds the anomaly threshold. The node corresponding to the graph representation feature with an anomaly score exceeding the anomaly threshold is regarded as an abnormal node.

[0122] Step 6 specifically includes the following steps:

[0123] Step 6-1: Store the graph representation features of the benign nodes obtained in Step 3 in a KD-tree, and calculate the distance variable that can summarize the range of features of the benign nodes. The benign node graph representation features obtained in step 3 are stored in a KD-tree as training data for K-nearest neighbor anomaly detection. This data is used to search for the K nearest neighbor features of the feature corresponding to the node to be detected, and the average distance d between each benign node feature h∈H and its K nearest neighbor features in the KD-tree is calculated. h and d of all benign characteristics h By averaging, a distance variable is obtained that summarizes the range of characteristics of benign nodes. The calculation formula in this embodiment is as follows:

[0124]

[0125] In addition, to speed up the detection process, a subset of the benign node graph representation features obtained in step 3 can be used to train the anomaly detection model instead of all the benign node graph representation features. This can speed up the detection process by reducing the number of K-nearest neighbor searches, but this approach may reduce the detection performance.

[0126] Step 6-2: Input the source map to be detected into the K-Nearest Neighbors algorithm and calculate the average distance between each node and its K-nearest neighbor features. Input the source map to be detected into the K-Nearest Neighbors algorithm and search for each node v among all benign features of the KD-tree. i The features are identified by the K nearest neighbor benign features in the KD-tree and formed into a set H. The relationship between each feature in H and node v is then calculated. i Features Calculate the distance between them and obtain the average value. The formula is as follows:

[0127]

[0128] Step 6-3: Calculate the anomaly score for each node in the source graph to be detected. For each node v in the source graph to be detected... i The anomaly detection model will calculate anomaly scores for it. The formula is as follows:

[0129]

[0130] Step 6-4: Determine the anomaly threshold. Based on whether the anomaly score of a node is greater than the anomaly threshold, determine whether the node is an anomaly node. In this embodiment, the optimal anomaly threshold θ is determined by the ROC (Receiver Operating Characteristic) curve. If the anomaly score of a node is greater than the anomaly threshold, then the node will be considered an anomaly node.

[0131] Step 7: Based on the mapping relationship between nodes and log entities obtained in Step 4, and combined with the abnormal nodes identified in Step 6, obtain the abnormal log entity corresponding to the abnormal node according to the mapping relationship, and return the entity log entry corresponding to the abnormal entity.

[0132] This invention also provides an APT attack detection system based on masking strategies and graph autoencoders, comprising: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the method described above.

[0133] An APT attack detection system based on masking strategy and graph autoencoder, such as Figure 4 As shown, it consists of four modules, including a log source graph construction module, a graph representation learning model training module, an anomaly detection module, and a log feedback module.

[0134] The log source graph construction module is responsible for the corresponding functions in steps 1 and 4. By extracting the attribute information of entities and events in the logs, it constructs a benign log source graph and a log source graph to be tested, and constructs the mapping relationship between nodes and log entities in the log source graph to be tested.

[0135] The graph representation learning model training module is responsible for the functions in steps 2 and 3. It obtains a set of masked nodes by combining random masking and learning-based masking, and masks the log source graph obtained in step 1. Then, it trains the masked graph autoencoder and saves the trained masked graph autoencoder and the corresponding graph representation features obtained by benign nodes through the graph autoencoder.

[0136] The anomaly detection module is responsible for the functions in steps 5 and 6. It trains anomaly detection models such as the K-nearest neighbor model using the benign node graph representation features obtained in step 3, and inputs the log source graph to be detected obtained in step 4 into the anomaly detection model for detection. It calculates the anomaly score of each node to be detected, and nodes corresponding to graph representation features that exceed the anomaly threshold will be regarded as anomaly nodes.

[0137] The log feedback module is responsible for the function in step 7. Based on the mapping relationship between the nodes in the traceability graph to be detected and the log entities in step 4, it obtains the abnormal log entities corresponding to the abnormal nodes and returns the entity log entries corresponding to the abnormal entities.

[0138] The aforementioned APT attack detection system based on masking strategy and graph autoencoder can be implemented as a computer program, stored on a hard disk, and loaded into a processor for execution to implement the method of the present invention.

[0139] The APT attack detection method proposed in this invention, based on masking strategy and graph autoencoder, trains the graph representation learning model by combining masking strategy and graph autoencoder. It takes into account the structural information of the log source graph and the uniform distribution of benign node features, thus ensuring the efficiency and accuracy of anomaly detection.

[0140] The above embodiments of the APT attack detection method based on masking strategy and graph autoencoder proposed in this invention were used to detect APT attacks on a subset of the DARPA-E3 Theia dataset. Multiple detection results show that the F1 score of this invention can reach 99.3%, the precision can reach 98.6%, and the recall can reach 99.9%. The detection results show that this invention has good detection performance in APT attack detection tasks.

[0141] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0142] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0143] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An APT attack detection method based on masking strategy and graph autoencoder, characterized in that, Includes the following steps; Step 1: Obtain healthy system log data and extract entity and event attributes from it, construct a log source graph, and reduce noise in the log source graph; Step 2: Based on the denoised log source map obtained in Step 1, select the nodes that need to be masked using a masking strategy; Mask the nodes that need to be masked in the log source graph using the mask node selector; Step 3: Train a masked graph autoencoder using the masked benign log source graph. During training, update the parameters of the masked node selector. After training, save the benign graph representation features corresponding to the nodes of the benign source graph obtained by the graph autoencoder. Step 4: Extract the entity attributes and event attributes of the system log data to be detected, construct the corresponding log source graph to be detected, and perform noise reduction; at the same time, establish the mapping relationship between the log source graph nodes and the system log entities; Step 5: Input the source graph of the log to be detected into the trained masked graph autoencoder model to obtain the graph representation features of the node to be detected; Step 6: Use the benign graph representation features as training data for the anomaly detection model, and input the graph representation features of the node to be detected into the anomaly detection model to calculate its anomaly score. Determine whether the corresponding node is abnormal based on whether the score exceeds the anomaly threshold. Nodes corresponding to graph representation features with anomaly scores exceeding the anomaly threshold are considered as abnormal nodes. Step 7: Based on the mapping relationship between the log source graph nodes and system log entities, and the abnormal nodes, obtain the abnormal log entities corresponding to the abnormal nodes, and return the entity log entries corresponding to the abnormal entities. The obtained abnormal entity entries are the entity entries detected that are related to APT attacks.

2. The APT attack detection method based on masking strategy and graph autoencoder according to claim 1, characterized in that, In step 1, the construction of the log source graph includes three parts: log feature extraction, source graph construction, and source graph feature encoding. The specific operation of log feature extraction is as follows: first, distinguish each entity and event in the system log data, and then use the log parsing program to extract the unique identifier number, category, IP address and timestamp information corresponding to the entry as the features of the log entry; The specific operations for constructing the source graph are as follows: entity entries in the system log are constructed into nodes, and the node features are entity log features extracted from log feature extraction; event entries in the system log are constructed into edges, and the edge features are event log features extracted from log feature extraction. The specific operation of source graph feature encoding is to initially encode the information corresponding to nodes and edges through one-hot encoding, and then perform noise reduction processing on the source graph. Noise reduction is an operation performed on edges, including deleting redundant edges of the same type between the same node pairs and merging edges of different types between the same node pairs.

3. The APT attack detection method based on masking strategy and graph autoencoder according to claim 2, characterized in that, In step 2, the masking strategy for log source graph nodes is a combination of random masking and learning-based masking. The random masking method selects nodes to be masked by random selection, while the learning-based masking method selects nodes to be masked by a mask node selector with learnable parameters. The method based on random masking involves randomly selecting nodes from the source graph at a certain ratio k, and forming a random mask node set. Learning-based masking uses a mask node selector. Φ The selection of mask nodes is controlled, and the mask nodes selected in this way form a learning-based mask node set. Learning-based masking selectors use a learned mask node selector to select mask nodes. Φ Composed of a graph neural network and fully connected layers, it can obtain the probability (Prob) that each node i is masked after inputting the log source graph. i The probability is used to determine whether each node should be masked, and nodes that need to be masked are added to the learning-based mask set. In Chinese, the formula is as follows: In the formula, mask i mask is either 0 or 1. i A value of 1 indicates that node i needs to be added to the set of learning-based mask nodes. In the middle, otherwise not required, depending on the mask. i It can determine whether node i needs to be masked, ∈0, ∈1 are random noise sampled from Gumbel(0,1), τ is a hyperparameter, and σ is the Sigmoid function.

4. The APT attack detection method based on masking strategy and graph autoencoder according to claim 3, characterized in that, The set of nodes obtained by using a random mask. The learning-based mask node set obtained by the learning-based masking method They need to be merged into a mask node set in the following way. Random mask node set and learning-based mask node set The number of nodes in the mask node selector is determined by the number of training iterations t, as shown in the following formula: In the formula, α lr (t) represents the mask weights calculated by the mask graph autoencoder in the t-th training epoch, T represents the total number of training epochs of the mask graph autoencoder, ω is a hyperparameter used to control the mask weight growth rate, and α0 and α T A real number between 0 and 1, indicating that the mask weight gradually increases from α0 to α T ; As the number of training iterations increases, α lr (t) continuously increases from α0 to α T , The number of nodes gradually increases, and the final mask node set in each training round... middle, Randomly select one of them The final set of random mask nodes is composed of these nodes. express The number of elements in; Randomly select one of them The final learning-based mask node set is composed of these nodes. The set of mask nodes in the t-th training round Obtain the set of mask nodes Then, the log source graph is compared with the mask node set. The initial features of the nodes corresponding to the nodes in the graph are masked. After the masking is completed, the masked log source graph can be obtained.

5. The APT attack detection method based on masking strategy and graph autoencoder according to claim 4, characterized in that, In step 3, the mask image autoencoder and its training process, as well as the parameter update of the mask node selector, are detailed below: The masked graph autoencoder consists of a graph encoder and a graph decoder. During the training process of the masked graph autoencoder, the feature uniformity loss and structural reconstruction loss of the unmasked nodes obtained by the graph encoder are calculated, and the features of the masked nodes are reconstructed by the graph decoder. The feature reconstruction loss of the masked nodes is calculated. The masked graph autoencoder updates its parameters based on the feature uniformity loss of the unmasked nodes, the structural reconstruction loss, and the feature reconstruction loss of the masked nodes. The graph encoder encodes the masked log source graph data using a graph neural network to obtain the graph representation features h of the log source graph nodes. v The graph decoder reconstructs the reconstructed features of the mask nodes by reconstructing the graph representation features of the mask nodes. Feature reconstruction loss L of mask nodes feature The initial feature x of the mask node v v With reconstruction features The calculation yielded: In the formula Let γ be the set of mask nodes, and γ be a hyperparameter. The update of the mask node selector parameters specifically refers to the mask node selector. Φ The update of parameter Φ needs to be based on the mask loss L mask Perform, mask loss L mask The loss L is reconstructed from the features of the masked nodes. feature The calculation is as follows: In the formula, λ1 is the weight used to control the proportion of mask nodes, and N is the number of nodes in the log source graph; The uniformity loss of unmasked nodes is derived from the graph representation feature set of unmasked nodes obtained by the graph encoder. The calculation yielded: In the formula, λ is the uniformity loss weight of the node graph representation feature, and α lr is the mask weight obtained in step 2, and n represents the number of nodes in X.

6. The APT attack detection method based on masking strategy and graph autoencoder according to claim 5, characterized in that, In step 3, the structural reconstruction loss of the unmasked nodes is the graph representation feature set of the unmasked nodes obtained by the graph encoder. The results were obtained through structural reconstruction based on sample pairs of sampling nodes, as detailed below: Structural reconstruction based on sampled pairs requires random sampling of positive and negative sample pairs. Node pairs with edges in the log source graph are called positive sample pairs, denoted as (v, v). + ), and vice versa, are called negative sample pairs, denoted as (v, v). - For sample pair (v,v) * The graph representation features h are obtained by using a multilayer perceptron (MLP). v and The probability of an edge existing between sample pairs is predicted, and the structural reconstruction loss between sample pairs is calculated using the cross-entropy loss function, as shown in the following formula: In the formula, This represents the set of unmasked nodes, where [a,b] represents the concatenation of a and b. The mask image autoencoder is based on the aforementioned uniformity loss L uniform Structural reconstruction loss L struct and feature reconstruction loss L feature The parameters are updated by summing the results, and the final loss L is calculated using the following formula: L=L struct +L uniform +L feature 。 7. The APT attack detection method based on masking strategy and graph autoencoder according to claim 6, characterized in that, Specifically, step 4 involves: constructing a mapping table between log source graph nodes and system log entities based on the unique identifiers of entities in the system logs. This table is used to query the mapping relationship between log source graph nodes and system log entities. Abnormal log entity entries are determined by abnormal log source graph nodes, thereby achieving fine-grained APT attack detection. In step 5, the log source map to be detected does not need to be masked. It is directly input into the trained masked graph autoencoder model saved in step 3. The graph encoder encodes the log source map to be detected to obtain the graph representation features of each node in the log source map to be detected.

8. The APT attack detection method based on masking strategy and graph autoencoder according to claim 7, characterized in that, In step 6, the anomaly detection model uses the K-nearest neighbor model to perform anomaly detection on the graph representation features, as detailed below: In step 6, when using the K-nearest neighbor model as the anomaly detection model, the step of inputting the graph representation features corresponding to the benign source tracing graph nodes into the anomaly detection model as training data specifically involves: The benign node graph representation features obtained in step 3 are stored in a KD-tree as training data for K-nearest neighbor anomaly detection. This data is used to search for the K nearest neighbor features of the feature corresponding to the node to be detected, and the average distance d between each benign feature h∈H and its K nearest neighbor features in the KD-tree is calculated. h and d of all benign characteristics h By averaging, a distance variable is obtained that summarizes the range of benign characteristics. The calculation formula is as follows: When using the K-nearest neighbor model as the anomaly detection model, the anomaly score is calculated based on the graph representation features of the node to be detected, specifically as follows: The nodes in the source-tracing graph to be detected are input into the K-nearest neighbor algorithm, and each node v is searched among all benign features of the KD tree. i The features are the K nearest neighbor benign features in the KD-tree and form a set. And calculate Features and nodes v i Features Calculate the distance between them and obtain the average value. The formula is as follows: The anomaly detection model will perform anomaly detection on each node v i Calculate abnormal scores The formula is as follows: When using the K-nearest neighbors model as the anomaly detection model, nodes corresponding to graph representation features whose anomaly scores exceed an anomaly threshold are considered anomalous nodes. An anomaly threshold θ is determined; if a node v i abnormal scores If the value is greater than the abnormal threshold θ, then the node v i This will be considered an abnormal node; If there are too many nodes to be detected, select a portion of the benign node graph representation features obtained in step 3 to train the anomaly detection model, and speed up the detection speed by reducing the number of K-nearest neighbor searches.

9. The APT attack detection method based on masking strategy and graph autoencoder according to claim 8, characterized in that, In step 7, the mapping relationship between nodes and log entities is as follows: the unique identifier of the log source graph node in step 4 is the same as the unique identifier in the log entity attribute. The log source graph node and the log entity can be matched through the unique identifier.

10. An APT attack detection system based on masking strategy and graph autoencoder, characterized in that, include: Memory is used to store system log data and instructions executed by the processor; A processor for executing the instructions to implement the method according to any one of claims 1-9.

Images