APT attack detection method fusing adversarial adaptation and graph domain adaptation

Abstract

The invention discloses an APT attack detection method fusing adversarial adaptation and graph domain self-adaptation, and belongs to the technical field of network attack detection and deep learning. The method comprises the following steps: (1) collecting system kernel log data, and constructing a traceability graph; (2) taking an existing APT attack data set as a source domain, taking a real unlabeled data set as a target domain, performing feature extraction and processing on nodes on the traceability graph, and training a classifier model on the source domain; (3) designing a feature generator and a domain discriminator to carry out adversarial training, combining the adversarial training and an information bottleneck method to jointly optimize a feature extractor, and carrying out pseudo-label optimization through meta-learning to realize unsupervised learning; finally, the model can detect potential APT attack behaviors in real log data. According to the method, the accuracy and robustness of the model are improved by combining the two domain adaptation methods of adversarial domain adaptation and graph domain adaptation, and the actual application requirements under different platforms and environments are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of network attack detection and deep learning, and specifically relates to an APT attack detection method integrating adversarial adaptation and graph domain adaptation. Background Art

[0002] The development of Internet information technology has also spawned many internal and external security risks and threats. In recent years, advanced persistent threats (APT) have posed a major risk to the data security and economic stability of countries, enterprises and other institutions. APT attacks are usually highly concealed, persistent and highly destructive. Attackers use a variety of means to infiltrate target systems, steal sensitive information or destroy system functions, which are difficult to detect by traditional security methods. In order to effectively detect and defend against APT attacks, researchers proposed a method for constructing a traceability graph based on system kernel logs. This method can extract rich contextual information from audit logs, providing a perfect data support for fine-grained causal analysis and APT attack detection.

[0003] Early APT attack detection work built rules based on typical or specified APT patterns and matched audit logs according to these rules to detect potential APTs. Some recent studies have attempted to use deep learning-based methods to detect APT attacks. They use various deep learning techniques to model APT attack patterns or system behaviors and perform APT attack detection in a classification or anomaly detection manner. However, the former is highly dependent on predefined rules or patterns for matching abnormal or malicious behaviors, and it is difficult to discover new attack behavior patterns; while the latter can accurately identify some hidden attack behaviors, but due to the characteristics of APT attacks that are difficult to reproduce and label, the number and diversity of relevant training samples are extremely low. At the same time, real log data often comes from different platforms and environments, and attackers will continue to evolve new attack behaviors. Therefore, the real collected data may be significantly different from the data distribution during training. If the data is collected and classified again for a new environment or attack pattern, it will be time-consuming and expensive, and the lack of effective labeled data will make the model extremely vulnerable in the real attack scene. On the other hand, domain adaptation is an important research direction in machine learning, which aims to solve the problem of data distribution differences between the source domain and the target domain. Adversarial adaptation enables the model to generate consistent features in the source domain and the target domain through adversarial training, thereby improving the generalization ability of the model under different data distributions. The graph information bottleneck method in graph domain adaptation can extract features that are useful for the task and domain-invariant, while removing irrelevant noise and redundant information. Summary of the invention

[0004] In view of the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide an APT attack detection method that integrates adversarial adaptation and graph domain adaptation. By combining the two domain adaptation methods, adversarial domain adaptation and graph domain adaptation, the accuracy and robustness of the model are improved to adapt to the actual application needs in different platforms and environments.

[0005] The present invention provides the following technical solution: an APT attack detection method integrating adversarial adaptation and graph domain adaptation, comprising the following steps:

[0006] (1) Construction of a traceability graph based on system kernel logs: Collect and compress kernel log data from multiple platforms, define the system entities and interaction behaviors involved in the log data, and construct a traceability graph that can represent the entities in the system and their interaction relationships;

[0007] (2) Feature extraction and classifier training: Using the existing APT attack dataset as the source domain and the real unlabeled dataset as the target domain, we extract and process the features of the nodes on the provenance graph and train a classifier model on the source domain.

[0008] (3) Model optimization and detection: Design feature generators and domain discriminators for adversarial training, combine adversarial training and the information bottleneck method to jointly optimize the feature extractor, and perform pseudo-label optimization through meta-learning to achieve unsupervised learning; ultimately, the model can detect potential APT attack behaviors in real log data.

[0009] Furthermore, the specific process of (1) is as follows:

[0010] (1-1) Log data collection: On each operating system, select the corresponding kernel log data collection tool to collect and save kernel logs, remove noise and redundant information, and perform data compression to ensure data quality;

[0011] (1-2) Construct a traceability graph: Analyze and extract the data collected by the log collection tool, and establish an entity cause-and-effect relationship graph based on the data to construct a traceability graph; in the traceability graph, each node represents a system entity of the system, and the edge represents the interaction relationship between them. Based on the node type and edge type, construct a traceability graph P = (V, R), where V is the node type set and R is the edge type set.

[0012] Furthermore, the specific process of (2) is as follows:

[0013] (2-1) Feature extraction: Use the existing APT attack dataset as the source domain P s , taking the real unlabeled dataset as the target domain P t ; Use GraphSAGE neural network to extract and process the features of nodes in the source domain traceability graph; the details are as follows:

[0014] (2-1-1) Assign an initial feature vector to each node based on the basic attributes of the node;

[0015] (2-1-1) Through neighbor sampling and feature aggregation, the aggregated neighbor features are combined with the current node features to generate a high-dimensional feature representation G(x s ), and standardize and normalize the extracted node features to ensure the consistency of feature scales;

[0016] (2-2) Train the classifier as follows:

[0017] A classifier model is trained on labeled data in the source domain, using the high-dimensional features of the nodes to represent G(x s ) as features, multi-layer perceptron as classifier, cross entropy loss function is used to measure the prediction error of the model, and the weights are updated through the back propagation algorithm, finally forming a classifier model that can accurately classify benign nodes and malicious nodes on the traceability graph.

[0018] Furthermore, the specific process of (3) is as follows:

[0019] (3-1) Adversarial adaptation: Design feature generators and domain discriminators for adversarial training, so that the target domain features are as close as possible to the source domain features;

[0020] (3-2) Graph domain adaptation based on information bottleneck: Use the graph information bottleneck method to extract domain-invariant feature representations that can effectively distinguish normal behavior from abnormal behavior to optimize the feature generator so that the generated features have good performance in both the source domain and the target domain;

[0021] (3-3) Unsupervised pseudo-label optimization: Use the model trained on the source domain to generate initial pseudo-labels for the target domain data, train the target domain data through pseudo-labels, and use meta-learning strategies to dynamically adjust the pseudo-label generation process based on the performance of the model on the source domain; (3-4) Threat detection: Input the unlabeled data of the target domain into the trained and optimized model, and the model will analyze and classify the target domain data based on the features and patterns learned in the source domain. Finally, potential APT attack threats are identified and corresponding detection results are generated.

[0022] Furthermore, in (3-3), during the training process, the pseudo-labels are iteratively optimized using Bayesian optimization, and the accuracy and diversity of the pseudo-labels are gradually improved through multiple iterations to ensure that the pseudo-labels can accurately reflect the true distribution of the target domain data; finally, through the unsupervised meta-pseudo-label optimization method, the model can generate high-quality pseudo-labels in the target domain and perform effective training on unlabeled data, thereby improving the detection accuracy and generalization ability of the overall model.

[0023] By adopting the above technology, compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1) By combining the data of the source domain and the target domain, the model can learn more diverse and rich features, better adapt to unknown attack forms and enhance the robustness against different types of attacks;

[0025] 2) Through adversarial training, the present invention makes the target domain features as close as possible to the source domain features, reduces the distribution difference between the source domain and the target domain, and ensures that the classifier can ensure effective classification in both the source domain and the target domain;

[0026] 3) The present invention further extracts features that are useful for the task and domain-invariant through the graph information bottleneck method, while removing irrelevant noise and redundant information to improve the model efficiency;

[0027] 4) The present invention utilizes a meta-learning pseudo-label strategy to perform unsupervised adaptation on the target domain, enabling it to maintain efficient detection capabilities even in the absence of labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of the method of the present invention;

[0029] Figure 2 This is a schematic diagram of domain adaptation of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments of the specification. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0031] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0032] Reference Figure 1-2 , an APT attack detection method integrating adversarial adaptation and graph domain adaptation, comprising the following steps:

[0033] (1) Construction of a traceability graph based on system kernel logs: Collect system kernel log data on multiple platforms and perform data compression, define the system entities and interaction behaviors involved in the log data, and construct a traceability graph that can represent the entities in the system and their interaction relationships.

[0034] (2) Feature extraction and classifier training: Using the existing APT attack dataset as the source domain and the real unlabeled dataset as the target domain, we extract and process the features of the nodes on the provenance graph and train a classifier model on the source domain.

[0035] (3) Model optimization and detection: Design feature generators and domain discriminators for adversarial training, combine adversarial training and information bottleneck methods to jointly optimize feature extractors, and optimize pseudo labels through meta-learning to achieve unsupervised learning. Ultimately, the model can detect potential APT attack behaviors in real log data.

[0036] In step (1), the detailed steps for constructing the traceability graph based on the system kernel log are as follows:

[0037] (1-1) Log data collection: On each operating system, select an appropriate kernel log data collection tool to collect and save kernel logs, including process information, file information, and event information, remove noise and redundant information, and perform data compression to ensure data quality.

[0038] (1-2) Construct a traceability graph: Analyze and extract the data collected by the log collection tool, and use this data to build an entity cause-and-effect relationship graph to construct a traceability graph. In the traceability graph, each node represents a system entity of the system, such as a process, file, or socket, and the edge represents the interaction relationship between them, such as the derivation relationship between processes, the creation relationship between processes and files, and the connection relationship between processes and sockets. Based on the node type and edge type, construct a traceability graph P = (V, R), where V is the node type set and R is the edge type set.

[0039] In step (2), the detailed steps of feature extraction and classifier training are as follows:

[0040] (2-1) Feature extraction: Use the existing APT attack dataset as the source domain P s , taking the real unlabeled dataset as the target domain P t . The GraphSAGE neural network is used to extract and process the features of the nodes in the source domain traceability graph. This method is more efficient when processing large-scale graphs and can capture important information in the graph structure. First, an initial feature vector is assigned to each node based on the basic attributes of the node. Then, through neighbor sampling and feature aggregation, the aggregated neighbor features are combined with the current node features to generate a high-dimensional feature representation G(x s ). The extracted node features are standardized and normalized to ensure the consistency of feature scales.

[0041] (2-2) Training the classifier: First, a classifier model is trained on labeled data in the source domain, using the high-dimensional feature representation of the node G(x s ) as features, multi-layer perceptron as classifier, cross entropy loss function is used to measure the prediction error of the model, and the weights are updated through the back propagation algorithm, finally forming a classifier model that can accurately classify benign nodes and malicious nodes on the traceability graph.

[0042] In step (3), the detailed steps of model optimization and testing are as follows:

[0043] (3-1) Adversarial adaptation: Design a feature generator and a domain discriminator for adversarial training, so that the target domain features are as close as possible to the source domain features. The feature generator receives the target domain data x t Generate its feature representation G(x) through the feature generator network t ), whose goal is to minimize the loss of the domain discriminator, that is, to reduce the distribution difference between the source domain and the target domain so that the generated features are indistinguishable from the perspective of the domain discriminator.

[0044] The domain discriminator receives the source domain features G(x s ) and target domain features G(x t ), the output is converted into a probability distribution through the softmax function, and the cross entropy loss is calculated to try to distinguish whether these features are from the source domain or the target domain. Its goal is to maximize the ability to distinguish the features of the source domain and the target domain. The final joint loss is the weighted sum of the adversarial loss and the discriminant loss, and its optimization goal is to simultaneously minimize the loss of the classifier and maximize the loss of the domain discriminator. Ultimately, the data distribution of the source domain and the target domain is as close as possible in the feature space, so that the model trained on the source domain can effectively classify on the target domain.

[0045] (3-2) Graph domain adaptation based on information bottleneck: Use the graph information bottleneck method to extract feature representations that are useful for the task and domain-invariant to optimize the feature generator so that the features it generates have good performance in both the source domain and the target domain. Mutual information measures the amount of information shared by the feature representation and the classification label. By maximizing the mutual information between the feature representation and the target variable, it ensures that the feature representation contains information that is useful for the task. By minimizing the mutual information between the feature representation and the input data, it removes irrelevant noise and redundant information. This can improve the generalization ability of the model in different domains while reducing overhead and improving efficiency. Combine adversarial training and the information bottleneck method to jointly optimize the feature extractor. Adversarial training ensures that the features generated by the feature extractor have good performance in both the source domain and the target domain, while the information bottleneck principle ensures that the features generated by the feature extractor are concise and effective.

[0046] (3-3) Unsupervised meta-pseudo-label optimization: Use the model trained on the source domain to generate initial pseudo-labels for the target domain data. These pseudo-labels represent the model’s preliminary classification results for the target domain data. Use these pseudo-labels to train the target domain data. Since the initial pseudo-labels may contain certain errors, in order to further improve the quality of the pseudo-labels, a meta-learning strategy is adopted to dynamically adjust the pseudo-label generation process based on the performance of the model on the source domain. During the training process, Bayesian optimization is used to iteratively optimize the pseudo-labels. Through multiple iterations, the accuracy and diversity of the pseudo-labels are gradually improved to ensure that the pseudo-labels can accurately reflect the true distribution of the target domain data. Finally, through this unsupervised meta-pseudo-label optimization method, the model can generate high-quality pseudo-labels on the target domain and perform effective training on unlabeled data, thereby improving the detection accuracy and generalization ability of the overall model.

[0047] (3-4) Threat Detection: The unlabeled data of the target domain is input into the trained and optimized model. The model will analyze and classify the target domain data based on the features and patterns learned in the source domain. Finally, potential APT attack threats are identified and corresponding detection results are generated.

[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting APT attacks integrating adversarial adaptation and graph domain adaptation, characterized in that: The following steps are involved: (1) Construction of a traceability graph based on system kernel logs: Collect and compress kernel log data from multiple platforms, define the system entities and interaction behaviors involved in the log data, and construct a traceability graph that can represent the entities in the system and their interaction relationships; (2) Feature extraction and classifier training: Using the existing APT attack dataset as the source domain and the real unlabeled dataset as the target domain, we extract and process the features of the nodes on the provenance graph and train a classifier model on the source domain. (3) Model optimization and detection: Design feature generators and domain discriminators for adversarial training, combine adversarial training and the information bottleneck method to jointly optimize the feature extractor, and perform pseudo-label optimization through meta-learning to achieve unsupervised learning; ultimately, the model can detect potential APT attack behaviors in real log data.

2. According to claim 1, the APT attack detection method integrating adversarial adaptation and graph domain adaptation is characterized in that: The specific process of (1) is as follows: (1-1) Log data collection: On each operating system, select the corresponding kernel log data collection tool to collect and save kernel logs, remove noise and redundant information, and perform data compression to ensure data quality; (1-2) Construct a traceability graph: Analyze and extract the data collected by the log collection tool, and establish an entity cause-and-effect relationship graph based on the data to construct a traceability graph; in the traceability graph, each node represents a system entity of the system, and the edge represents the interaction relationship between them. Based on the node type and edge type, construct a traceability graph P = (V, R), where V is the node type set and R is the edge type set.

3. According to claim 2, the APT attack detection method integrating adversarial adaptation and graph domain adaptation is characterized in that: The specific process of (2) is as follows: (2-1) Feature extraction: Use the existing APT attack dataset as the source domain P s , taking the real unlabeled dataset as the target domain P t ; Use GraphSAGE neural network to extract and process the features of nodes in the source domain traceability graph; the details are as follows: (2-1-1) Assign an initial feature vector to each node based on the basic attributes of the node; (2-1-1) Through neighbor sampling and feature aggregation, the aggregated neighbor features are combined with the current node features to generate a high-dimensional feature representation G(x s ), and standardize and normalize the extracted node features to ensure the consistency of feature scales; (2-2) Train the classifier as follows: A classifier model is trained on labeled data in the source domain, using the high-dimensional features of the nodes to represent G(x s ) as features, multi-layer perceptron as classifier, cross entropy loss function is used to measure the prediction error of the model, and the weights are updated through the back propagation algorithm, finally forming a classifier model that can accurately classify benign nodes and malicious nodes on the traceability graph.

4. According to claim 3, the APT attack detection method integrating adversarial adaptation and graph domain adaptation is characterized in that: The specific process of (3) is as follows: (3-1) Adversarial adaptation: Design feature generators and domain discriminators for adversarial training, so that the target domain features are as close as possible to the source domain features; (3-2) Graph domain adaptation based on information bottleneck: Use the graph information bottleneck method to extract domain-invariant feature representations that can effectively distinguish normal behavior from abnormal behavior to optimize the feature generator so that the generated features have good performance in both the source domain and the target domain; (3-3) Unsupervised pseudo-label optimization: Use the model trained on the source domain to generate initial pseudo-labels for the target domain data, train the target domain data through pseudo-labels, and use meta-learning strategies to dynamically adjust the pseudo-label generation process based on the performance of the model on the source domain; (3-4) Threat detection: Input the unlabeled data of the target domain into the trained and optimized model, and the model will analyze and classify the target domain data based on the features and patterns learned in the source domain. Finally, potential APT attack threats are identified and corresponding detection results are generated.

5. According to claim 4, the APT attack detection method integrating adversarial adaptation and graph domain adaptation is characterized in that: In (3-3), during the training process, the pseudo-labels are iteratively optimized using Bayesian optimization, and the accuracy and diversity of the pseudo-labels are gradually improved through multiple iterations to ensure that the pseudo-labels can accurately reflect the true distribution of the target domain data; Finally, through the unsupervised meta-pseudo-label optimization method, the model is able to generate high-quality pseudo-labels on the target domain and effectively train on unlabeled data, thereby improving the detection accuracy and generalization ability of the overall model.

Images