A threat detection method and system based on hop count decoupling heterogeneous source tracing graphs
By constructing an event-level directed heterogeneous source tracing multigraph and performing direction unification and inbound edge pruning, combined with hop count decoupling of input representation and edge features, the problem of semantic confusion and high false alarm rate in heterogeneous source tracing graph scenarios of traditional detection methods is solved, and efficient and accurate unsupervised threat detection is achieved.
Patent Information
- Application Number
- CN202610962021.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-30
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to effectively detect unknown attacks in advanced persistent threats (APTs), especially when attackers use legitimate system tools or normal processes to disguise themselves, making them prone to missed detections. Furthermore, traditional methods face issues such as semantic confusion, unstable training, and high false positive rates in heterogeneous source graph scenarios.
By constructing an event-level directed heterogeneous source tracing multigraph, performing direction unification and ingress edge pruning, constructing node and edge features, using hop count to decouple input representation, and using a graph encoder for joint training to generate node embeddings, anomaly scores are calculated for threat detection.
It achieves efficient and accurate unsupervised threat detection, reduces false alarm rate, improves detection accuracy and adaptability, and can stably output anomaly scores in complex heterogeneous source tracing graph scenarios.
Smart Images

Figure CN123122858A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of cyberspace security, and in particular to a threat detection method and system based on hop count decoupling heterogeneous source tracing graphs. Background Technology
[0002] In Advanced Persistent Threat (APT) detection scenarios, attacks are characterized by long latency, multiple stages, cross-entity involvement, and strong concealment. Traditional detection methods based on rules, signatures, or statistical thresholds, while simple to implement and offering good real-time performance, are insufficient for detecting unknown APT activities or low-frequency, slow-moving attacks, especially when attackers use legitimate system tools or disguise themselves as normal processes. To address this, existing research has gradually introduced source graph-based detection frameworks, treating entities such as processes, files, and network sockets as nodes and events such as read, write, and execute as directed edges to describe causal dependencies within a host. However, existing solutions still have the following limitations: First, log sequence modeling methods struggle to express long-range causality across entities; second, sketch compression or streaming summarization methods easily lose edge semantics and temporal order; third, graph neural network-based methods face semantic confusion, training instability, and high false positive rates in heterogeneous source graph scenarios. The applicant's research revealed that the host origin graph contains diverse node and edge types, with one-hop neighbors often exhibiting cross-type connections. Direct, indiscriminate aggregation leads to representation pollution. Furthermore, the graph size expands dramatically over long periods, with high-frequency entities generating significant background noise that obscures crucial causal evidence related to attacks. In addition, APT detection struggles to obtain sufficient labeled samples, necessitating a detection scheme that can be trained using only benign data, preserves directional and temporal semantics, mitigates type contamination, reduces graph size, and outputs stable anomaly scores. Summary of the Invention
[0003] The present invention aims to at least partially solve one of the technical problems in the related art.
[0004] To address this, this invention proposes a threat detection method based on a hop count-decoupled heterogeneous source tracing graph. By acquiring host audit log streams, system entities are parsed into nodes, and interaction events are parsed into directed edges with timestamps and semantics, constructing an event-level directed heterogeneous source tracing multigraph. Direction unification and time-order-based ingress edge pruning are then performed. Node and edge features are constructed on the pruned graph; node features include entity type and identifier semantics, while edge features include event type and time encoding. Masking is performed on the node features, and zero-hop, one-hop ingress neighbor, and two-hop ingress neighbor aggregation representations are calculated based on the pruned graph and concatenated into a hop count-decoupled input. This input and edge features are fed into a graph encoder, which is jointly trained using data containing only benign samples, reconstructed according to mask attributes, and reconstructed with a directed structure target to obtain node embeddings. Based on the trained encoder, the embeddings of the nodes to be tested are generated, and their distances to the nearest neighbor embeddings in the benign reference embedding library are calculated to obtain anomaly scores and threat detection results. This invention solves the problems of type mixing and causal noise interference in traditional source graph detection by decoupling hop count and preserving heterogeneous semantics, and achieves efficient and accurate unsupervised threat detection.
[0005] Another objective of this invention is to propose a threat detection system based on hop count decoupling heterogeneous source graphs.
[0006] To achieve the above objectives, this invention proposes a threat detection method based on hop count decoupling heterogeneous source graphs, comprising:
[0007] The system obtains the audit log stream of the host system, parses the system entities into nodes and the system interaction events into directed edges with timestamps and event semantics, constructs an event-level directed heterogeneous tracing multigraph, and performs direction unification processing and time-order-based incoming edge pruning processing on the directed heterogeneous tracing multigraph to obtain the pruned tracing graph. Construct node features and edge features for the cropped source graph; node features include entity type features and entity identifier semantic features, and edge features include event type features and time encoding features. The node features are masked, and the zero-hop self-feature representation, one-hop in-neighbor aggregated feature representation, and two-hop in-neighbor aggregated feature representation are calculated based on the cropped source graph. The three are then concatenated to obtain the hop count decoupled input representation. The hop count decoupled input representation and edge features are input into the graph encoder. The graph encoder is jointly trained using training data containing only benign samples, based on the mask attribute reconstruction target and the directed structure reconstruction target, to obtain the node embedding representation. The node embedding to be tested is generated based on the trained graph encoder. The anomaly score is obtained by calculating the distance between the node embedding to be tested and the nearest neighbor embedding in the benign reference embedding library, and the threat detection result is obtained.
[0008] In one embodiment of the present invention, constructing an event-level directed heterogeneous source tracing multigraph and performing direction unification and inbound edge pruning on the corresponding directed heterogeneous source tracing multigraph includes: Extract kernel audit logs or system-wide source tracing logs from the host system in real time and parse them into a standard event stream containing source entity identifier, target entity identifier, event type, and timestamp fields. Processes, files, and network sockets are extracted as nodes, and read, write, execute, connect, receive, load, derive, attribute modification, and delete events are extracted as directed edges. Interaction events that occur repeatedly at different times between the same ordered endpoint pairs are retained as multiple independent event edges, forming an event-level directed heterogeneous tracing multigraph. The event edges of the directed heterogeneous source-tracing multigraph are normalized to ensure consistent information flow direction, thereby adjusting the direction of event edges that reflect information consumption to point from information provider to information consumer. For each target node, sort all incoming edges by timestamp from oldest to newest, retain only the most recent K incoming edges and retain multiple edges formed by repeated events, and obtain the pruned source graph.
[0009] In one embodiment of the present invention, the direction unification process performs information flow consistency direction normalization, including: Identify the semantic type of the event edge and maintain events that generate information output, such as write, send, and connect, in the direction from the source node to the target node; The edges representing information consumption events such as reading, receiving, and loading are flipped so that they point from the information provider to the information consumer, making the direction of all edges in the graph consistent with the actual direction of information flow.
[0010] In one embodiment of the present invention, the time-sequence-based in-edge clipping process includes: Traverse each target node in the source graph before pruning and obtain the set of all incoming edges associated with the corresponding node; Sort each edge in the incoming edge set from oldest to newest according to the timestamp field, and then select the K edges with the newest timestamps from the sorted incoming edge set as retained edges; where K is a fixed threshold or a threshold determined adaptively based on the in-degree of the node. At the same time, retain the multiple edges formed by repeated events in the retained edges, and remove the historical incoming edges that are ranked first, so as to obtain the pruned source graph.
[0011] In one embodiment of the present invention, constructing node features and edge features includes: For each node v in the cropped source graph, construct the node feature vector:
[0012] Where τ(v) represents the entity type of node v, onehot(·) represents the one-hot encoding function, and s(v) represents the semantic embedding obtained by hash projection based on file path, command line arguments, IP address, domain name, or process identifier fields; For each edge e, construct the edge feature vector:
[0013] Where φ(e) represents the event type, b(e) represents the time bucket index obtained by discretization based on the relative timestamp, and PE(·) represents the sinusoidal position encoding function.
[0014] In one embodiment of the present invention, constructing a hop count decoupled input representation includes: A subset M is randomly selected from the node set of the cropped source graph. The original features of the nodes in subset M are replaced with learnable mask labels to obtain the masked node feature matrix. As a characteristic representation of zero-hop; Based on the source map after cropping ,right Perform one-hop neighbor mean aggregation to obtain the one-hop neighbor aggregated feature representation:
[0015] right Perform the in-neighbor mean aggregation again to obtain the two-hop in-neighbor aggregation feature representation:
[0016] Will and Concatenate along the feature dimension to obtain the hop count-decoupled input representation: .
[0017] In one embodiment of the present invention, joint training of the graph encoder includes: An edge-attribute-aware graph encoder is used to process directed event edges. Calculate attention score:
[0018] in , Embedded for source and target nodes. As edge features, The node feature projection matrix, The edge feature projection matrix; Normalize all incoming neighbors of the target node v to obtain normalized attention weights:
[0019] Update node embedding accordingly ;in The message transformation matrix; Training is performed using only benign samples. The mask attribute reconstruction loss and the directed structure reconstruction loss are jointly optimized. The mask attribute reconstruction loss is calculated as follows: ; The directed structure reconstruction loss samples a set E of positive samples from the real edges. + Sample a set of negative samples E from a non-existent ordered pair of nodes. - Calculate the probability of edge existence:
[0020] Therefore, we get:
[0021] Total loss: Where λ is the balancing coefficient, the encoder parameters are updated using the Adam optimizer.
[0022] In one embodiment of the present invention, calculating anomaly scores and outputting threat detection results includes: After training, freeze the graph encoder parameters, generate node embeddings for all benign training samples, and build a reference embedding library. ; Execute the embedding of the node to be detected (z) and the embedding in the reference library. 2. Normalization; The k-nearest neighbor retrieval algorithm is used to find the k nearest neighbor embeddings of z in the reference library, and the anomaly score is calculated:
[0023] in, The i-th nearest neighbor embedding in the reference library; The preset threshold is determined based on the false alarm rate budget on the benign validation subset. If the value exceeds the threshold, the corresponding node is determined to be an abnormal node, and the threat detection result is output.
[0024] In one embodiment of the present invention, after outputting the threat detection result, the method further includes: After determining that a node is an abnormal node, extract the abnormal node and its associated event edges in the pruned source graph to construct an abnormal subgraph; The abnormal subgraph is used as data to display advanced persistent threat alerts, and attack chain backtracking analysis is performed based on the node connection relationships and timestamp information in the abnormal subgraph to locate the source of the attack and the spread path.
[0025] This invention also proposes a threat detection system based on hop count decoupling heterogeneous source tracing graphs, comprising: The graph construction and pruning module is used to obtain the audit log stream of the host system, parse system entities into nodes, parse system interaction events into directed edges with timestamps and event semantics, construct an event-level directed heterogeneous tracing multigraph, and perform direction unification processing and time-order-based incoming edge pruning processing on the directed heterogeneous tracing multigraph to obtain the pruned tracing graph. The feature construction module is used to construct node features and edge features for the cropped source graph; node features include entity type features and entity identifier semantic features, and edge features include event type features and time encoding features. The hop count decoupling module is used to perform masking processing on node features and calculate the zero-hop self-feature representation, one-hop-in-neighbor aggregated feature representation, and two-hop-in-neighbor aggregated feature representation based on the cropped source graph. The three are then concatenated to obtain the hop count decoupling input representation. The training and detection module is used to input the hop count decoupled input representation and edge features into the graph encoder. It uses training data containing only benign samples to jointly train the graph encoder according to the mask attribute reconstruction target and the directed structure reconstruction target to obtain the node embedding representation. Based on the trained graph encoder, the module generates the node embedding to be tested. The module calculates the distance between the node embedding to be tested and the nearest neighbor embedding in the benign reference embedding library to obtain the anomaly score and the threat detection result.
[0026] This invention discloses a threat detection method and system based on hop count decoupling heterogeneous source graphs. By constructing an event-level directed heterogeneous source graph and performing direction unification and time-sequence-based edge pruning, combined with hop count decoupling input representation and edge features, the graph encoder is jointly trained using data containing only benign samples for mask attribute reconstruction and directed structure reconstruction. This generates node embeddings and calculates anomaly scores based on nearest neighbor distance. This effectively solves the detection bias and insufficient generalization problems caused by mixed types, background noise interference, and reliance on attack annotations in existing technologies. It achieves integrated detection across the entire process, from audit log stream parsing, graph structure pruning, hop count decoupling representation to unsupervised anomaly judgment, significantly improving the accuracy and reliability of threat detection and enhancing the method's adaptability and engineering practical value in complex heterogeneous source graph scenarios.
[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1This is a flowchart of a threat detection method based on a hop count-decoupled heterogeneous source graph according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall detection process according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the construction of an event-level directed heterogeneous tracing multiple graph according to an embodiment of the present invention; Figure 4 This is a schematic diagram of in-line clipping based on time sequence according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the zero-hop, one-hop, and two-hop decoupled input structure according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the edge attribute-aware graph encoder structure according to an embodiment of the present invention; Figure 7 This is a schematic diagram of k-nearest neighbor anomaly scoring based on a benign reference embedding library according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of a threat detection system based on a hop count decoupling heterogeneous source graph according to an embodiment of the present invention. Detailed Implementation
[0029] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0031] The following description, with reference to the accompanying drawings, describes a threat detection method and system based on a hop count-decoupled heterogeneous source graph according to an embodiment of the present invention.
[0032] Figure 1 This is a flowchart of a threat detection method based on hop count decoupling heterogeneous source graphs according to an embodiment of the present invention.
[0033] like Figure 1 As shown, a threat detection method based on hop count decoupling heterogeneous source graphs includes the following steps: S1. Obtain the audit log stream of the host system, parse the system entities into nodes, parse the system interaction events into directed edges with timestamps and event semantics, construct an event-level directed heterogeneous tracing multigraph, and perform direction unification processing and time-order-based incoming edge pruning processing on the directed heterogeneous tracing multigraph to obtain the pruned tracing graph. S2, construct node features and edge features for the cropped source graph; node features include entity type features and entity identifier semantic features, and edge features include event type features and time encoding features; S3 performs masking on node features and calculates the zero-hop self-feature representation, one-hop-in-neighbor aggregated feature representation, and two-hop-in-neighbor aggregated feature representation based on the cropped source graph. The three are then concatenated to obtain the hop count decoupled input representation. S4. The hop count decoupled input representation and edge features are input into the graph encoder. The graph encoder is jointly trained using training data containing only benign samples according to the mask attribute reconstruction target and the directed structure reconstruction target to obtain the node embedding representation. Based on the trained graph encoder, the node embedding to be tested is generated. The anomaly score is obtained by calculating the distance between the node embedding to be tested and the nearest neighbor embedding in the benign reference embedding library, and the threat detection result is obtained.
[0034] Specifically, the embodiments of the present invention include: 1. Log parsing and source graph construction: Obtaining the audit log stream or source log stream of the host system, parsing system entities into nodes, parsing system interaction events into directed edges with timestamps and event semantics, and constructing an event-level directed heterogeneous source graph. 2. Direction unification and local pruning: Performing direction unification and local pruning on the directed heterogeneous source graph to obtain a pruned source graph; wherein, direction unification includes determining the edge direction according to the information flow direction, and local pruning includes retaining the nearest preset number of incoming edges for each target node in chronological order. 3. Feature construction: Constructing node features and edge features for the pruned source graph; node features include at least entity type features and entity identifier semantic features, and edge features include at least event type features and time encoding features. 4. Hop number decoupled input construction: Performing masking processing on the node features, and calculating the incoming neighbor aggregation representations of zero-hop, one-hop, and two-hop based on the pruned source graph, concatenating the zero-hop, one-hop, and two-hop representations to obtain the hop number decoupled input representation. 5. Edge Attribute Aware Self-Supervised Training: The hop count decoupled input representation and edge features are input into the edge attribute aware graph encoder. Using only benign samples, the graph encoder is jointly trained based on the mask attribute reconstruction target and the directed structure reconstruction target to obtain node embedding representations. 6. Anomaly Scoring and Detection: Based on the trained graph encoder, test node embeddings are generated from the data to be detected. Nearest neighbor retrieval is performed using a reference embedding library built from benign training samples. Anomaly scores are calculated based on the distance between the test node embedding and its nearest neighbor. APT detection results are output based on the anomaly scores and a preset threshold. 7. Iterative Training: Repeat steps 1-6 until all test data has been processed.
[0035] Furthermore, such as Figure 2 As shown, the implementation of the method in this embodiment of the invention includes the following steps: Step 1: Obtain the audit log stream or source tracing log stream of the host system, parse the system entities into nodes, parse the system interaction events into directed edges with timestamps and event semantics, and construct an event-level directed heterogeneous source tracing multigraph. Step 2: Perform direction unification and local pruning on the directed heterogeneous source tracing multigraph to obtain the pruned source tracing graph; Step 3: Construct node features and edge features for the cropped source graph; Step 4: Perform masking on the node features, and calculate the incoming neighbor aggregation representations for zero-hop, one-hop, and two-hop based on the cropped source graph. Concatenate the zero-hop, one-hop, and two-hop representations to obtain the hop count decoupled input representation. Step 5: Decouple the hop count input representation and the edge feature input to the edge attribute-aware graph encoder. Under the condition of using only benign samples, jointly train the graph encoder according to the mask attribute reconstruction target and the directed structure reconstruction target to obtain the node embedding representation. Step 6: Generate the node embedding to be tested based on the trained graph encoder, call the reference embedding library established by the benign training samples to perform nearest neighbor search, calculate the anomaly score based on the distance between the node embedding to be tested and its nearest neighbor, and output the APT detection result based on the anomaly score and the preset threshold. Step 7: Repeat steps 1 to 6 until all data to be detected has been processed.
[0036] In general, the method described in this embodiment of the invention includes: parsing host audit logs to construct an event-level directed heterogeneous tracing multigraph; unifying edge directions and pruning inbound edges to control graph size; constructing features for nodes and edges respectively; performing zero-hop, one-hop, and two-hop decoupling aggregation on node features to form parallel input; employing an edge attribute-aware graph attention encoder to jointly train node representations using only benign data; establishing a reference embedding library; generating anomaly scores through nearest neighbor retrieval to identify APT activity. This invention can mitigate semantic pollution caused by mixed one-hop neighborhood types, take into account multi-hop context and temporal information, and maintain a high detection recall rate while reducing false positive rate.
[0037] The following provides further explanation of each of the above steps.
[0038] Step 1, log parsing and source graph construction, specifically includes the following steps: Step 1.1: Extract kernel audit logs or full system source logs from the host that needs to perform threat detection in real time.
[0039] Step 1.2: Parse the raw log into a standardized four-tuple or five-tuple event stream, which includes at least the source entity identifier, target entity identifier, event type, and timestamp fields.
[0040] Step 1.3: Extract system entities such as processes, files, and network sockets as graph nodes, and extract system interaction events such as reading, writing, executing, connecting, receiving, loading, deriving, attribute modification, and deleting as directed edges.
[0041] Step 1.4: For repeated interaction events between the same ordered endpoint pairs at different times, retain them as multiple independent event edges, forming an event-level directed heterogeneous sourcing multigraph (e.g., ...). Figure 3 (As shown).
[0042] Step 2, unifying direction and local cropping, specifically includes the following steps: Step 2.1: Perform information flow consistency direction normalization processing on event edges. For events that generate information output, such as writing, sending, and connecting, build edges in the direction from the event source to the event target; for events that represent information consumption, such as reading, receiving, and loading, adjust the edge direction to point from the information provider to the information consumer.
[0043] Step 2.2: Sort all incoming edges of each target node from oldest to newest according to their timestamps.
[0044] Step 2.3: Only retain the most recent preset number K incoming edges, where K is a fixed threshold or a threshold adaptively determined based on the node's in-degree, thereby suppressing the interference of high-frequency background behavior on the local neighborhood. The clipped graph is notated as G'=(V, E').
[0045] Step 2.4: Preserve the multiple edges formed by repeating events in the graph (e.g., Figure 4 (as shown), to maintain the intensity of real interaction.
[0046] Step 3, feature construction, specifically includes the following steps: Step 3.1: Construct a feature vector for each node v
[0047] Where τ(v) represents the entity type of node v, onehot(·) represents the one-hot encoding function, and s(v) represents the semantic embedding obtained by hash projection based on file path, command line arguments, IP address, domain name, or process identifier fields.
[0048] Step 3.2: Construct a feature vector for each edge e.
[0049] Where φ(e) represents the event type, b(e) represents the time bucket index obtained by discretization based on the relative timestamp, and PE(·) represents the sinusoidal position encoding function.
[0050] Step 3.3: Optionally, set the hash projection dimension to 32 or 64 dimensions, and set the time bucket size to seconds or minutes.
[0051] Step 4, hop count decoupling input representation construction, specifically includes the following steps: Step 4.1: Randomly select a subset M from the node set, replace its original features with learnable mask labels, and obtain the masked node feature matrix. .
[0052] Step 4.2: Based on the pruned directed heterogeneous origin graph G', perform... Perform one-hop neighbor mean aggregation to obtain .
[0053] Step 4.3: For Perform the neighbor mean aggregation again to obtain ,in This means that the mean of each node is calculated over its set of incoming neighbors.
[0054] Step 4.4: Concatenate the zero-hop (masked features), one-hop, and two-hop representations along the feature dimension to form the graph encoder input (e.g., ...). Figure 5 (As shown):
[0055] Step 5, edge attribute-aware self-supervised training, specifically includes the following steps: Step 5.1: Employ an edge attribute-aware graph encoder (e.g., ...) Figure 6 As shown), for directed event edges Calculate attention score:
[0056] in The node feature projection matrix is used to map the representations of the source and target nodes to a unified attention space, thereby extracting the semantic relationships between nodes. The edge feature projection matrix is used to map the edge attributes composed of event type and time encoding, enabling the model to perceive the semantic differences of different interaction events when calculating attention.
[0057] Step 5.2: Normalize all incoming neighbors of the target node v to obtain normalized attention weights. .
[0058] Step 5.3: Update node embedding .in This is the message transformation matrix, used to convert the features of neighboring nodes into the actual information being transmitted, and then weighted and aggregated with attention weights to update the target node representation.
[0059] Step 5.4: Perform joint training using only benign samples. The training objectives include mask attribute reconstruction loss and directed structure reconstruction loss.
[0060] Step 5.5: Calculation of loss for mask attribute reconstruction: .
[0061] Step 5.6: Calculation of Directed Structure Reconstruction Loss: Sample the set of positive samples E from the real edges. + Sample a set of negative samples E from non-existent ordered node pairs. - Calculate the probability of edge existence: ,Then .
[0062] Step 5.7: Total Loss , where λ is the balance coefficient, and the encoder parameters are updated using the Adam optimizer.
[0063] Step 6, Anomaly Scoring and Detection Output, specifically includes the following steps: Step 6.1: After training, freeze the graph encoder, generate node embeddings for all benign training samples, and build a reference embedding library. .
[0064] Step 6.2: First execute the embedding z of the node to be detected and the embedding in the reference library. 2. Normalization.
[0065] Step 6.3: Calculate the anomaly score using k-nearest neighbor retrieval. (like Figure 7 (As shown).
[0066] Step 6.4: Determine the preset threshold based on the false positive rate budget on the reserved benign validation subset. When s(z) is greater than the threshold, the corresponding node is determined to be an abnormal node.
[0067] Step 6.5: Optionally, the abnormal nodes and their associated event edges are extracted into an abnormal subgraph for APT alarm display and attack chain backtracking.
[0068] Step 7, Iterative Application: For streaming audit logs, repeat steps 1 to 6 to continuously score newly added nodes or subgraphs for anomaly detection, thereby achieving real-time APT detection.
[0069] The method in this invention avoids semantic pollution caused by directly mixing heterogeneous messages in the traditional one-hop neighborhood through the decoupled construction of zero-hop, one-hop, and two-hop representations. It stably preserves the multi-hop causal context related to advanced persistent threat attacks and reduces false anomalies caused by background interactions. Simultaneously, by normalizing the consistent direction of information flow, preserving multiple edges, and pruning the most recent incoming edges in chronological order, it retains event direction, frequency of occurrence, and recent causal evidence while suppressing background noise, effectively improving deployability in large-scale host scenarios. This invention achieves end-to-end optimization from heterogeneous graph construction to hop count decoupling detection, significantly improving the accuracy and engineering applicability of threat detection.
[0070] Furthermore, the methods of this invention can be widely and deeply applied to enterprise-level endpoint defense, critical infrastructure protection, and intelligent unsupervised detection scenarios for advanced persistent threats (APTs). Addressing the technical pain point in real-world massive host audit logs where the highly "type-mixed" nature of local 1-hop neighbors causes traditional graph neural networks to blur abnormal boundaries and trigger massive false alarms during feature aggregation, this embodiment proposes a heterogeneous source graph detection system based on "hop count decoupling" and edge-aware feature fusion.
[0071] Firstly, during the data construction phase, the system dynamically parses the raw streaming logs into an "event-level directed heterogeneous multigraph" containing entities such as processes, files, and network objects, as well as interactive events such as read, write, and execute. To ensure controllable temporal logic and computing power, the system reverses the edge direction of consumer events such as READ / RECV to maintain the consistency of information flow direction. Furthermore, it employs a "time-ordered inbound edge pruning" strategy, retaining only the K nearest input edges to a node, effectively constraining the excessive growth of local subgraphs. Simultaneously, to enrich feature dimensions, the system injects feature vectors containing entity category and semantic hash information into nodes, and attribute vectors containing interaction type and sinusoidal position time encoding into event edges.
[0072] Secondly, in the core model processing and training stage, to address the problem of high noise in cross-type interactions, this solution abandons the traditional full-neighbor hybrid aggregation and innovatively adopts a hop count decoupling mechanism: through sparse mean aggregation technology, it provides the model encoder with independent statistical views of 0 hops (node self-features), 1 hop (direct cross-type neighbor information), and 2 hops (stable type consistency function return structure) in parallel; subsequently, using the "Edge-aware Graph Attention Network (Edge-aware GAT)," edge attributes (event semantics and temporal context) are dynamically injected into the neighbor attention scoring mechanism for message passing; the model is fully trained using normal business logs from the enterprise intranet, and the training objective is composed of "mask attribute reconstruction loss (forcing the model to infer node identity from the mixed context)" and "directed structure reconstruction loss (maintaining the directed interaction pattern of normal behavior)."
[0073] Finally, regarding the application and implementation results, during real-time detection, the system calculates the kNN Euclidean distance between the node under test and the normal behavior features in the benchmark training library, using this distance as the anomaly score. This embodiment was tested on five mainstream APT assessment datasets with tens of millions of nodes, including the DARPA Transparent Computing E3 tracking logs, StreamSpot, and Unicorn Wget. The solution achieved near-saturation recall and detection capabilities. For example, it achieved a 99.99% anomaly recall rate on the TRACE dataset (containing 5.89 million nodes) while suppressing the false positive rate (FPR) to 0.09%; a 100.00% recall rate and a 0.12% false positive rate on the THEIA dataset; and a 99.80% recall rate and a 0.03% false positive rate on the CADETS dataset. Compared with several existing state-of-the-art baseline methods, this approach maintains an excellent detection rate while improving precision by up to 5.88%, F1 score by up to 2.58%, and significantly reducing the overall false alarm rate by up to 85.00%.
[0074] The method described in this invention effectively solves the long-standing problem of long-cycle latent threat alert fatigue that has plagued security operations teams with extremely low computational overhead and zero supervised learning cost, significantly enhancing the accuracy of enterprise security situation awareness. Furthermore, this invention combines edge event semantics, time encoding, and a nearest neighbor distance scoring mechanism based on a benign reference embedding library, balancing detection accuracy, robustness, and engineering implementation complexity. This scheme effectively reduces the false positive rate while maintaining a high detection recall rate. Experiments show that on the DARPATC dataset, this method reduces the false positive rate to below 0.1%, an 85% reduction compared to existing methods. The technical solution of this invention is applicable to entity-level anomaly detection as well as log batch-level, time window-level, and subgraph-level anomaly detection, exhibiting good task compatibility and transfer application value.
[0075] To implement the method of the above embodiments, such as Figure 8 As shown, this invention proposes a threat detection system 10 based on a hop count-decoupled heterogeneous source tracing graph, comprising: The graph construction and pruning module 100 is used to obtain the audit log stream of the host system, parse system entities into nodes, parse system interaction events into directed edges with timestamps and event semantics, construct an event-level directed heterogeneous tracing multigraph, and perform direction unification processing and time-order-based incoming edge pruning processing on the directed heterogeneous tracing multigraph to obtain the pruned tracing graph.
[0076] The feature construction module 200 is used to construct node features and edge features for the cropped source graph; the node features include entity type features and entity identifier semantic features, and the edge features include event type features and time encoding features.
[0077] The hop count decoupling module 300 is used to perform masking processing on node features and calculate the zero-hop self-feature representation, the one-hop-in-neighbor aggregated feature representation, and the two-hop-in-neighbor aggregated feature representation based on the cropped source graph. The three are then concatenated to obtain the hop count decoupling input representation.
[0078] The training and detection module 400 is used to input the hop count decoupled input representation and edge features into the graph encoder. It uses training data containing only benign samples to jointly train the graph encoder according to the mask attribute reconstruction target and the directed structure reconstruction target to obtain the node embedding representation. Based on the trained graph encoder, the module generates the node embedding to be tested. The module calculates the distance between the node embedding to be tested and the nearest neighbor embedding in the benign reference embedding library to obtain the anomaly score and the threat detection result.
[0079] The system of this invention realizes integrated detection throughout the entire process, from log parsing, graph pruning, hop count decoupling representation to unsupervised anomaly judgment. It effectively solves the problems of mixed types and background noise interference in the prior art, and significantly improves the accuracy of threat detection and its engineering practicality under large-scale deployment.
[0080] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0081] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
Claims
1. A threat detection method based on hop count decoupling heterogeneous source graphs, characterized in that, include: The system obtains the audit log stream of the host system, parses the system entities into nodes and the system interaction events into directed edges with timestamps and event semantics, constructs an event-level directed heterogeneous tracing multigraph, and performs direction unification processing and time-order-based incoming edge pruning processing on the directed heterogeneous tracing multigraph to obtain the pruned tracing graph. Construct node and edge features for the cropped source graph; Node features include entity type features and entity identifier semantic features, while edge features include event type features and time encoding features; The node features are masked, and the zero-hop self-feature representation, one-hop in-neighbor aggregated feature representation, and two-hop in-neighbor aggregated feature representation are calculated based on the cropped source graph. The three are then concatenated to obtain the hop count decoupled input representation. The hop count decoupled input representation and edge features are input into the graph encoder. The graph encoder is jointly trained using training data containing only benign samples, based on the mask attribute reconstruction target and the directed structure reconstruction target, to obtain the node embedding representation. The node embedding to be tested is generated based on the trained graph encoder. The anomaly score is obtained by calculating the distance between the node embedding to be tested and the nearest neighbor embedding in the benign reference embedding library, and the threat detection result is obtained.
2. The method of claim 1, wherein, Construct event-level directed heterogeneous source tracing multigraphs and perform direction unification and inbound edge pruning on the corresponding directed heterogeneous source tracing multigraphs, including: Extract kernel audit logs or system-wide source tracing logs from the host system in real time and parse them into a standard event stream containing source entity identifier, target entity identifier, event type, and timestamp fields. Processes, files, and network sockets are extracted as nodes, and read, write, execute, connect, receive, load, derive, attribute modification, and delete events are extracted as directed edges. Interaction events that occur repeatedly at different times between the same ordered endpoint pairs are retained as multiple independent event edges, forming an event-level directed heterogeneous tracing multigraph. The event edges of the directed heterogeneous source-tracing multigraph are normalized to ensure consistent information flow direction, thereby adjusting the direction of event edges that reflect information consumption to point from information provider to information consumer. For each target node, sort all incoming edges by timestamp from oldest to newest, retain only the most recent K incoming edges and retain multiple edges formed by repeated events, and obtain the pruned source graph.
3. The method of claim 2, wherein, In the process of unifying the direction of information flow, information flow consistency direction normalization is performed, including: Identify the semantic type of the event edge and maintain events that generate information output, such as write, send, and connect, in the direction from the source node to the target node; The edges representing information consumption events such as reading, receiving, and loading are flipped so that they point from the information provider to the information consumer, making the direction of all edges in the graph consistent with the actual direction of information flow.
4. The method of claim 2, wherein, Time-sequence-based in-edge clipping processing includes: Traverse each target node in the source graph before pruning and obtain the set of all incoming edges associated with the corresponding node; Sort each edge in the incoming edge set from oldest to newest according to the timestamp field, and then select the K edges with the newest timestamps from the sorted incoming edge set as retained edges; where K is a fixed threshold or a threshold determined adaptively based on the in-degree of the node. At the same time, retain the multiple edges formed by repeated events in the retained edges, and remove the historical incoming edges that are ranked first, so as to obtain the pruned source graph.
5. The method of claim 1, wherein, Constructing node features and edge features includes: For each node v in the cropped source graph, construct the node feature vector: Where τ(v) represents the entity type of node v, onehot(·) represents the one-hot encoding function, and s(v) represents the semantic embedding obtained by hash projection based on file path, command line arguments, IP address, domain name, or process identifier fields; For each edge e, construct the edge feature vector: Where φ(e) represents the event type, b(e) represents the time bucket index obtained by discretization based on the relative timestamp, and PE(·) represents the sinusoidal position encoding function.
6. The method of claim 1, wherein, Constructing a hop-count-decoupled input representation includes: A subset M is randomly selected from the node set of the cropped source graph. The original features of the nodes in subset M are replaced with learnable mask labels to obtain the masked node feature matrix. As a characteristic representation of zero-hop; Based on the source map after cropping ,right Perform one-hop neighbor mean aggregation to obtain the one-hop neighbor aggregated feature representation: For The two-hop in-neighbor aggregation feature representation is obtained by performing in-neighbor mean aggregation again. will be described in detail below. and concatenated along the feature dimension to obtain a hop count decoupled input representation: 。 7. The method as described in claim 1, characterized in that, Joint training of the graph encoder includes: The edge attribute-aware graph encoder is adopted to encode the directed event edges Computing attention scores: wherein , are source and target node embeddings, is an edge feature, is a node feature projection matrix, is an edge feature projection matrix; Normalize all incoming neighbors of the target node v to obtain normalized attention weights: updating the node embeddings accordingly ; wherein is a message transformation matrix; Training is performed using only benign samples. The mask attribute reconstruction loss and the directed structure reconstruction loss are jointly optimized. The mask attribute reconstruction loss is calculated as follows: ; The directed structure reconstruction loss samples a set E of positive samples from the real edges. + Sample a set of negative samples E from a non-existent ordered pair of nodes. - Calculate the probability that the edge exists: Therefore, we get: Total loss: Where λ is the balancing coefficient, the encoder parameters are updated using the Adam optimizer.
8. The method of claim 1, wherein, Calculate the anomaly score and output the threat detection results, including: After training, freeze the graph encoder parameters, generate node embeddings for all benign training samples, and build a reference embedding library. ; Execute the embedding of the node to be detected (z) and the embedding in the reference library.
2. Normalization; The k-nearest neighbor retrieval algorithm is used to find the k nearest neighbor embeddings of z in the reference library, and the anomaly score is calculated: wherein, is the i-th nearest neighbor embedding in the reference library; The preset threshold is determined based on the false alarm rate budget on the benign validation subset. If the value exceeds the threshold, the corresponding node is determined to be an abnormal node, and the threat detection result is output.
9. The method as described in claim 8, characterized in that, The output of threat detection results also includes: After determining that a node is an abnormal node, extract the abnormal node and its associated event edges in the pruned source graph to construct an abnormal subgraph; The abnormal subgraph is used as data to display advanced persistent threat alerts, and attack chain backtracking analysis is performed based on the node connection relationships and timestamp information in the abnormal subgraph to locate the source of the attack and the spread path.
10. A threat detection system based on hop count decoupling heterogeneous source tracing graphs, characterized in that, include: The graph construction and pruning module is used to obtain the audit log stream of the host system, parse system entities into nodes, parse system interaction events into directed edges with timestamps and event semantics, construct an event-level directed heterogeneous tracing multigraph, and perform direction unification processing and time-order-based incoming edge pruning processing on the directed heterogeneous tracing multigraph to obtain the pruned tracing graph. The feature construction module is used to construct node features and edge features for the cropped source graph; Node features include entity type features and entity identifier semantic features, while edge features include event type features and time encoding features; The hop count decoupling module is used to perform masking processing on node features and calculate the zero-hop self-feature representation, one-hop-in-neighbor aggregated feature representation, and two-hop-in-neighbor aggregated feature representation based on the cropped source graph. The three are then concatenated to obtain the hop count decoupling input representation. The training and detection module is used to input the hop count decoupled input representation and edge features into the graph encoder. It uses training data containing only benign samples to jointly train the graph encoder according to the mask attribute reconstruction target and the directed structure reconstruction target to obtain the node embedding representation. Based on the trained graph encoder, the module generates the node embedding to be tested. The module calculates the distance between the node embedding to be tested and the nearest neighbor embedding in the benign reference embedding library to obtain the anomaly score and the threat detection result.
