An early warning method and system for identifying an APT attack based on a heterogeneous graph

Abstract

This invention discloses a method and system for early identification and warning of APT attacks based on heterogeneous graphs. The method comprises four steps: threat intelligence gathering, IOCS extraction and attribute heterogeneous graph modeling, upstream contrastive pre-training, and downstream threat LOCS identification. By establishing a graph contrastive learning model and introducing information theory principles to maximize the mutual information between positive and negative samples, the method reduces the need for labels and mitigates the impact of sample noise. This method exhibits faster convergence speed and superior threat identification capabilities, effectively reducing losses caused by untimely identification during APT attacks.

Description

Technical Field

[0001] This invention belongs to the technical field of network security, and mainly relates to a method and system for early identification and warning of APT attacks based on heterogeneous graphs. Background Technology

[0002] With the rapid expansion of data assets and the continuous advancement of cyberattack techniques, the challenges facing cybersecurity are becoming increasingly severe. Advanced Persistent Threat (APT) attacks, in particular, have become one of the most destructive problems in the current cybersecurity field. APT attacks are typically launched by hacker groups possessing advanced attack tools, sophisticated tactical skills, and covert infrastructure. These groups specialize in long-term, covert network intrusions targeting specific high-value objectives. Utilizing zero-day vulnerabilities, phishing attacks, and spear-phishing emails, they not only remain lurking in target networks for extended periods to steal information but also employ techniques such as code obfuscation to conceal their tracks, making traditional security measures ineffective. Therefore, developing methods that can accurately and efficiently provide early warnings of such complex cyberattacks has become an urgent task in the cybersecurity field.

[0003] While signature-based malware detection methods have some practical value, their reliance on predefined malware signature libraries makes them ineffective against newly emerging or variant malware. A limitation of this approach is that traditional signature-based detection may fail to identify malware if the author modifies certain code characteristics. In contrast, network traffic analysis-based detection strategies identify potential threats by monitoring and analyzing traffic patterns during data transmission. This method can detect anomalous traffic behavior to some extent, but it is often ineffective against advanced persistent threats (APTs) that do not produce obvious traffic anomalies. These attacks typically employ more covert and complex methods, making them difficult to detect with simple traffic analysis.

[0004] In this context, simulation execution techniques combined with static and dynamic analysis have become an effective means. By running and observing the behavior of unknown programs in a controlled environment, they can comprehensively reveal the potential maliciousness of programs. Furthermore, threat intelligence-based detection methods utilize threat data and intelligence from various channels, providing security analysts with rich contextual information and real-time security alerts, greatly enhancing the ability to identify well-designed, covert attacks. However, this approach also faces the challenge of processing large amounts of unstructured and multilingual text data. Extracting useful information from complex data and reducing noise interference are key to improving the efficiency and accuracy of threat intelligence analysis.

[0005] Furthermore, while advanced technologies such as knowledge graphs and graph neural networks can further enhance the depth and breadth of threat detection, their implementation and maintenance require substantial expertise and human resources, and also place high demands on the quality and quantity of data tags. Therefore, effectively utilizing these technologies with limited resources, and addressing issues such as tag scarcity and uneven data distribution, have become significant challenges facing the security field. Thus, effectively detecting and issuing early warnings of APTs under conditions of tag scarcity and uneven data distribution is a pressing issue that needs to be addressed in the field of threat intelligence analysis. Summary of the Invention

[0006] This invention addresses the shortcomings of traditional techniques in detecting APT attacks, including slow speed, insufficient detection and recognition rates, scarce datasets, inconsistent label quality, and insufficient quantity. It provides a method and system for early identification and warning of APT attacks based on heterogeneous graphs. The method comprises four steps: threat intelligence gathering, IOCS extraction and attribute heterogeneous graph modeling, upstream contrastive pre-training, and downstream threat locs identification. By establishing a graph contrastive learning model and incorporating information theory principles to maximize the mutual information between positive and negative samples, the method reduces the need for labels and mitigates the impact of sample noise. This method offers faster convergence and superior threat identification capabilities, effectively reducing losses caused by delayed identification during APT attacks.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is: a method for early identification and warning of APT attacks based on heterogeneous graphs, comprising the following steps:

[0008] S1, Threat Intelligence Collection: Collect threat intelligence reports through web crawlers or open-source intelligence websites. The threat intelligence reports shall include at least organization analysis reports, malware and vulnerability analysis reports.

[0009] S2, IOCS Extraction - Attribute Heterogeneous Graph Modeling: Based on the assistance of open source threat intelligence, extract IOC entities, entity label information and the relationship between IOC entities, expand the relationship between IOC nodes, supplement entities and relationships beyond the threat intelligence report, and define meta-paths to extract high-level semantic information from the heterogeneous graph.

[0010] S3, Upstream Contrastive Pre-training: The contrastive model utilizes HSAGE and HGAT encoders to construct two branches, which generate embeddings for nodes in the heterogeneous graph. and Same node and Embeddings between nodes are considered positive samples, while embeddings between other nodes are considered negative samples; fusion is achieved through an attention mechanism. and The final node embedding H is obtained; pooling is performed on all node embeddings to obtain the full graph embeddings v1 and v2, and the contrastive loss function is used to maximize the embedding. and The mutual information between H and v1, and between H and v2, is minimized.

[0011] S4, Downstream Threat LOCS Identification: Utilizes upstream heterogeneous graph neural networks to encode features from multi-source APT attack data, extracts key information about APT attacks, and inputs the encoding results into different classification models to predict APT attacks.

[0012] As an improvement to the present invention, step S2 specifically includes the following steps:

[0013] S21: Based on the assistance of open-source threat intelligence, APT reports are tagged, IOC entities, tag information of entities, and relationships between IOC entities are extracted, and text statistical features are supplemented; the IOC entities include at least domain name, IP address, hash file, CVE vulnerability, and attack method; the attributes of IOC nodes include at least domain name WHOIS information, IP address ASN number, and geographical location;

[0014] S22: Expand the relationships between IOCs nodes to include entities and relationships beyond those reported in threat intelligence reports, including at least IPs that have been resolved and domains accessed by malware;

[0015] S23: Construct a heterogeneous information network and define meta-paths to extract high-level semantic information from the heterogeneous graph; the type nodes of the heterogeneous information network include domain names, IP addresses, hash files, and reports.

[0016] As another improvement of the present invention, the indicators in the APT report in step S21 are hash values, IP addresses, domain names and URLs, thereby constructing an initial set of indicator nodes.

[0017] As another improvement of the present invention, in the expansion of step S22, for each IOC node, its related domain name, IP address, URL and file are queried and new IOC nodes are added. The various relationships between each IOC node and other nodes are queried in detail, and these new edges are included in the dataset. The relationships between each IOC node and other nodes include, but are not limited to, parsing relationships, communication relationships, download relationships and calling relationships.

[0018] As another improvement of the present invention, the loss function for training the contrastive learning model in step S3 is:

[0019] L GIB =-L NCE (h1,h2)+δ·(D KL(p(H,v1)||p(H)p(v1))+D KL (p(H,v2)||p(H)p(v2)))

[0020] Where L NCE (h1,h2) describes the mutual information between h1 and h2, D KL (p(H,v1)||p(H)p(v1)) describes the mutual information between H and v1, where p(H,v1) represents the joint probability distribution, p(H)p(v1) represents the marginal probability distribution, and δ is a learnable parameter.

[0021] To achieve the above objectives, the present invention also adopts the following technical solution: a heterogeneous graph-based early identification and warning system for APT attacks, comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the methods described above.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) This invention proposes innovations from two perspectives: APT detection model method and model loss function design. In terms of detection model method, this method uses two different heterogeneous graph encoders, namely HSAGE and HGAT, to establish two different branch encoders. In terms of loss function design, this method maximizes the mutual information between h1 and h2 based on the InfoNCE estimator, and minimizes the mutual information between H and v1 and H and v2 based on KL divergence, thereby suppressing noise information and achieving better results for downstream locs detection tasks.

[0024] (2) Compared with existing methods, this method can more accurately identify APT attacks in the early stage of the attack; it can effectively solve the problems of slow convergence speed, large data noise, small number of labels and uneven quality of traditional methods, resulting in low model performance. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating a method for early identification and warning of APT attacks based on heterogeneous graphs according to the present invention.

[0026] Figure 2 This is a diagram showing the results of the adversarial experiment in Test Example 2 of this invention. Detailed Implementation

[0027] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0028] Example 1

[0029] A method for early identification and warning of APT attacks based on heterogeneous graphs, such as Figure 1 As shown, it consists of four parts: the first part is threat intelligence gathering; the second part is IOCs extraction and attribute heterogeneous graph modeling; the third part is comparative model training; and the fourth part is downstream IOCs classification testing.

[0030] This invention selects a heterogeneous graph dataset as the test object to test and evaluate the method of this invention, and further describes it in detail. The method for early identification and warning of APT attacks based on heterogeneous graphs specifically includes the following steps:

[0031] Step S1, Threat Intelligence Collection: Collect threat intelligence reports through web crawlers or open-source intelligence websites; in this embodiment, a total of 574 threat intelligence reports were collected from the Internet, including organization analysis reports, malicious code, vulnerability analysis reports, etc.

[0032] Step S2, IOCS extraction - attribute heterogeneous graph modeling, includes the following sub-steps;

[0033] Step S21 involves tagging APT reports with the assistance of open-source threat intelligence platforms and extracting IOC entities such as domain names, IP addresses, hash files, CVE vulnerabilities, and attack methods, as well as their tag information and the relationships between these IOC entities. These publicly available threat intelligence platforms include QiAnXin, NSFOCUS, alienvault, and Threatminer, supplementing the data with textual statistical features such as domain name text similarity, domain name length and the proportion of repeated characters, domain name WHOIS information, IP address ASN, and geographical location as attributes of the IOC nodes.

[0034] We obtained open-source threat intelligence from 20 APT (Advanced Persistent Threat) groups over the past five years from domestic and international security platforms. Using regular expressions, we extracted four types of indicators (hash values, IP addresses, domain names, and URLs) from these APT reports, constructing an initial set of indicator nodes. Specifically, IP addresses, domain names, and URLs in the threat intelligence were truncated—that is, markers were inserted into the strings—to prevent the corrupted string structure from triggering hyperlinks in the reader, thus avoiding the risk of linking to them. Therefore, before applying regular expressions, this embodiment restored the modified IOCs to their original format to ensure successful IOC extraction.

[0035] Step S22: Expand the relationships between IOCs nodes using an open-source threat intelligence platform, supplementing entities and relationships beyond the threat intelligence report, such as IPs resolved by domain names, domains accessed by malware, etc., including a large number of benign domain names and IPs.

[0036] This embodiment utilizes the API of the renowned network security platform VirusTotal to significantly expand the number and relationships of IOCs (Integrated Control Centers) in the network. Through meticulous observation and analysis of real attack data, it was found that infrastructure nodes within the same organization can form a connected graph through 2-3 hops, demonstrating strong correlations between nodes within the same organization; conversely, the correlations between nodes in different organizations are relatively weak. Based on this finding, the IOCs extracted from the report were expanded using VirusTotal's API. Regarding the number of nodes, for each IOC node, new IOC nodes were added, including its related domain names, IP addresses, URLs, and files, significantly increasing the network's size and density. Regarding the expansion of relationships, various relationships between each IOC node and other nodes were queried in detail, such as resolution relationships, communication relationships, download relationships, and call relationships, and these new edges were incorporated into the dataset. This methodology not only enhances network connectivity and coverage but also makes the network mapping more complete and realistic, providing more accurate support for network security protection.

[0037] Step S23: A heterogeneous information network consisting of four types of nodes—domain name, IP address, hash file, and report—is defined, and meta-paths are defined to extract high-level semantic information from the heterogeneous graph.

[0038] This method is based on the extraction of four types of threat intelligence nodes, including v ip v domain v hash v report Construct a heterogeneous graph. Based on information such as the subnet and autonomous system to which the IP address belongs, construct the feature matrix x of the IP address. ip Based on the domain name's registration time, registrant, and statistical information such as information entropy, it can be used as the domain name's feature matrix x. domain Based on the file IDs of the malware, malware families can be used as the feature matrix x of malware. hash Furthermore, each node in the heterogeneous graph has a label attribute y that reflects its own category. For IP addresses, this embodiment only labels their type. This article categorizes domain names into two types: malicious domain names and normal domain names. Similarly, for malware, this embodiment further categorizes it into malicious and normal types. Regarding threat intelligence, it is categorized into 20 APT groups, namely... in Indicates non-organization, in the formula All are constants.

[0039] Five different meta-paths are defined to describe semantic relationships in threat intelligence.

[0040] (1)

[0041] (2)

[0042] (3)

[0043] (4)

[0044] (5)

[0045] Where m1 represents the resolution and being resolved relationship between a domain name and an IP address; m2 represents the communication and download relationship between malware and a domain name; m3 represents the relationship between a domain name and its subdomains; and m4 and m5 represent the connection relationships between hash files. These five types of metapaths constitute the metapath set.

[0046] Step S3, the contrastive learning model training step, includes the following sub-steps;

[0047] Step S31: This step proposes an asymmetric heterogeneous graph encoder for upstream tasks in graph contrastive learning. Two different heterogeneous graph encoders, HSAGE and HGAT, are used to establish two distinct branch encoders. The two branches encode and aggregate nodes in the heterogeneous graph respectively to obtain two different embeddings. Embeddings of the same node from different branches are considered positive samples of each other, while embeddings of other nodes are considered positive samples of each other. and Or other branch embedding and It is considered a negative sample. Then, for and Perform an attention-based fusion process to derive the node embeddings H. Next, for all nodes h... u Perform average pooling to obtain full graph embeddings v1 and v2 on the two branches.

[0048] Step S32: To effectively handle noise and redundant information in threat intelligence map data, a novel contrastive loss function based on information bottleneck theory is adopted. When designing the information bottleneck-based loss function, the mutual information between v1 and H, and between v2 and H, is minimized to reduce redundant information in the fused embeddings; while the mutual information between h1 and h2 is maximized to ensure commonality between branch embeddings, thereby achieving the goal of learning sufficient classification features. Therefore, the loss function of this method maximizes the mutual information between k1 and k2 based on the InfoNCE estimator, and minimizes the mutual information between H and v1, and between H and v2 based on KL divergence.

[0049] This method can optimize the vectorized representation of nodes by reducing unnecessary information transmission while retaining key information. The loss function is designed as follows:

[0050] L GIB =-L NCE (h1,h2)+δ·(D KL (p(H,v1)||p(H)p(v1))+D KL (p(H,v2)||p(H)p(v2)))

[0051] Step S4, downstream locs classification test: Use the upstream heterogeneous graph neural network to encode features of multi-source data of APT attacks, extract key information of APT attacks, and input the encoding results into different classification models to predict APT attacks.

[0052] Test case

[0053] 1. Test Example 1:

[0054] Test objective: To test the performance of this method against other base models in binary classification detection of APT threat attacks, in order to verify the superiority of this method in detection performance compared to other models.

[0055] Test environment: Model training and evaluation experiments were conducted on an environment with an i7 10700F CPU, 32GB RAM, and an RTX 4090 GPU.

[0056] Testing method: The accuracy, precision, and recall of the experimental results were evaluated based on the confusion matrix. The calculation formulas are as follows:

[0057]

[0058] Parameter settings: As shown in the table below, the ratio of training set, test set, and validation set is 7:2:1;

[0059] Hyperparameter setting table

[0060] Learning rate GAT head count N Number of hidden layers in GNN Dropout Epoch 0.002 8 3 0.2 1000

[0061] Dataset description: The dataset used in this test is the graph dataset collected in steps S1 and S2 of Example 1, which contains 27,915 domain names.

[0062] Test Description: The performance of the model in binary classification of malicious domain names was evaluated in detail, and compared with heterogeneous graph neural network algorithms, homogeneous graph neural network algorithms, and graph contrastive learning methods. The F1 score was selected as the primary evaluation metric in the experiments.

[0063] Test Results: The feasibility analysis results of the model in threat intelligence report classification are shown in the table below:

[0064] Performance of each model in binary classification of malicious domain names

[0065]

[0066]

[0067] Results Analysis: The experimental results are shown in the table above, where 0 represents a normal domain name and 1 represents a malicious domain name. As can be seen from the table, the model algorithm proposed in this invention achieved optimal performance on this dataset, with overall F1 scores of 81.2% and 77.6% respectively, significantly outperforming other algorithms.

[0068] 2. Test Example 2:

[0069] Test objective: To verify that this method still has good performance when the training data has very few labels, by comparing its performance with other base models with those of other models.

[0070] Test environment: Model training and evaluation experiments were conducted on an environment with an i7 10700F CPU, 32GB RAM, and an RTX 4090 GPU.

[0071] Testing method: The accuracy, precision, and recall of the experimental results were evaluated based on the confusion matrix. The calculation formulas are as follows:

[0072]

[0073] Parameter settings are shown in the table below:

[0074] Hyperparameter setting table

[0075] Learning rate GAT head count N Number of hidden layers in GNN Dropout Epoch 0.002 8 3 0.2 1000

[0076] Dataset description: The dataset used in this test is the graph dataset collected in steps S1 and S2 of Example 1, which contains 27,915 domain names.

[0077] Experimental Description: To evaluate the performance of different models with different numbers of labels, we fixed the validation set ratio at 0.1 and the sum of the training and test set ratios at 0.9 on the dataset. We also set the training set ratios to 0.02, 0.05, 0.1, 0.2, and 0.5 respectively. We compared the performance by changing the different training set ratios. In the experiment, the F1 score was selected as the main evaluation metric.

[0078] Test Results: Analysis of Model Adversarial Experiment Results Figure 2 As shown: Result analysis: Figure 2 The results show the F1 scores of various models under different numbers of labels. When the number of labels is only 0.02%, the model achieves an F1 score of nearly 70% for binary classification detection of malicious domains. The second best performing model, HGT, only achieves a detection performance of 54%, while other baseline models are even lower than 50%. This further illustrates that this model has a lower dependency on other models and performs better under conditions with fewer labels.

[0079] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A method for early identification and warning of APT attacks based on heterogeneous graphs, characterized in that... It includes the following steps: S1, Threat Intelligence Collection: Collect threat intelligence reports through web crawlers or open-source intelligence websites. The threat intelligence reports shall include at least organization analysis reports, malware reports, and vulnerability analysis reports. S2, IOCs Extraction - Attribute Heterogeneous Graph Modeling: Based on the assistance of open source threat intelligence, extract IOC entities, entity label information and the relationship between IOC entities, expand the relationship between IOC nodes, supplement entities and relationships beyond the threat intelligence report, and define meta-paths to extract high-level semantic information from the heterogeneous graph. S3, Upstream Contrastive Pre-training: The contrastive learning model utilizes the HSAGE and HGAT encoders to construct two branches, which generate embeddings for nodes in the heterogeneous graph. and Same node and Embeddings between nodes are considered positive samples, while embeddings between other nodes are considered negative samples; fusion is achieved through an attention mechanism. and The final node embedding H is obtained; pooling is performed on all node embeddings to obtain the full graph embeddings v1 and v2, and the contrastive loss function is used to maximize the embedding. and The mutual information between H and v1, and between H and v2, is minimized; the loss function for training the contrastive learning model is: L GIB =-L NCE (h1,h2)+δ·(D KL (p(H,v1)‖p(H)p(v1))+D KL (p(H,v2)‖p(H)p(v2))) Where L NCE (h1,h2) describes the mutual information between h1 and h2, D KL (p(h,v1)||P(H)p(v1)) describes the mutual information between H and v1, p(H,v1) represents the joint probability distribution, p(H)p(v1) represents the marginal probability distribution, and δ is a learnable parameter; S4, Downstream Threat LOCS Identification: Utilizes upstream heterogeneous graph neural networks to encode features from multi-source APT attack data, extracts key information about APT attacks, and inputs the encoding results into different classification models to predict APT attacks.

2. The method for early identification and warning of APT attacks based on heterogeneous graphs as described in claim 1, characterized in that: Step S2 specifically includes the following steps: S21: Based on the assistance of open-source threat intelligence, APT reports are tagged, IOC entities, tag information of entities, and relationships between IOC entities are extracted, and text statistical features are supplemented; the IOC entities include at least domain name, IP address, hash file, CVE vulnerability, and attack method; the attributes of IOC nodes include at least domain name WHOIS information, IP address ASN number, and geographical location; S22: Expand the relationships between IOCs nodes to include entities and relationships beyond those reported in threat intelligence reports, including at least IPs that have been resolved and domains accessed by malware; S23: Construct a heterogeneous information network and define meta-paths to extract high-level semantic information from the heterogeneous graph; the type nodes of the heterogeneous information network include domain names, IP addresses, hash files, and reports.

3. The method for early identification and warning of APT attacks based on heterogeneous graphs as described in claim 2, characterized in that: The metrics in the APT report in step S21 are hash values, IP addresses, domain names, and URLs, which are used to construct the initial set of metric nodes.

4. The method for early identification and warning of APT attacks based on heterogeneous graphs as described in claim 3, characterized in that: In the expansion of step S22, for each IOC node, its related domain name, IP address, hash file and new IOC node are queried and added. The various relationships between each IOC node and other nodes are queried in detail, and the new edges are included in the dataset. The relationships between each IOC node and other nodes include resolution relationships, communication relationships, download relationships and calling relationships.

5. The method for early identification and warning of APT attacks based on heterogeneous graphs as described in claim 1, characterized in that: In step S4, after the pre-training phase of the upstream task, the embedded representations of the network nodes of the threat heterogeneous information are obtained. The downstream task aims to identify potential threat types based on these embedded representations. The classification models include logistic regression, random forest, SVM and decision tree.

6. A system for early identification and warning of APT attacks based on heterogeneous graphs, comprising a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-5 above.

Images