A method and system for provenance graph compression based on causal semantic consistency

Abstract

The application discloses a kind of based on causal semantic consistency's traceability graph compression method and system.The method is: according to system audit log, constructs initial traceability graph;For the node of the initial traceability graph, respectively extract the structural embedding feature and semantic embedding feature of the node;Based on the structural embedding feature, semantic embedding feature of node, the causal semantic consistency measure between nodes is calculated;Based on the causal semantic consistency measure, the node of the initial traceability graph is aggregated, and the compressed traceability graph is obtained.The application solves the problem that attack key information is lost caused by single-dimensional compression of existing traceability graph compression method, compression effect and detection accuracy cannot be considered, achieves high compression ratio, while retaining the causal dependence and semantic information of attack key, improves the efficiency of downstream security analysis, and does not lose the effect of detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of network security and source tracing analysis technology, specifically relating to a source graph compression method and system with consistent causal semantics. Background Technology

[0002] Source graphs have become an important foundation for attack investigation and anomaly detection in enterprise systems. However, system audit logs generate massive amounts of events, resulting in extremely large source graphs. This not only hinders efficient security analysis but also brings significant storage and computing overhead.

[0003] In existing technologies, source tracing-based security analysis techniques have been extensively studied. For example, BackTracker implements causal tracing of intrusions, PriorTracker accelerates source tracing through priority ranking, and graph learning-based methods such as PROGRAPHER and MAGIC achieve anomaly detection. However, these methods all face the scalability problem of excessively large source tracing graphs.

[0004] To address this issue, existing research has proposed various source graph compression techniques, categorized into structure-based compression, semantic-based compression, and hybrid compression. Structure-based compression, such as LogApprox, ProvGRP, and Dehydrator, compresses graphs by identifying recurring topological structures; however, these methods ignore semantic differences and are prone to losing key attack information. Semantic-based compression, such as NodeMerge, aggregates nodes based on semantic attributes; however, attackers can spoof semantics, leading to the incorrect merging of malicious nodes. Hybrid methods are mostly task-specific and lack a universal joint mechanism.

[0005] The disadvantages of existing technologies are as follows: (1) Pure structural compression will mask the special causal structure of the attack, burying the special causal chain of the attack into the repeated normal structure, resulting in the loss of the attack path.

[0006] (2) Pure semantic compression will incorrectly merge malicious and normal nodes. Attackers can merge malicious and normal nodes by spoofing semantics, thus losing attack clues.

[0007] (3) Existing hybrid methods lack a universal joint metric and cannot simultaneously achieve both high compression ratio and complete preservation of attack information. Summary of the Invention

[0008] The purpose of this invention is to propose a source graph compression method and system with consistent causal semantics, in order to solve the problems of loss of key attack information and inability to balance compression effect and detection accuracy caused by single-dimensional compression in existing source graph compression methods. The goal is to achieve a high compression ratio while fully preserving the causal dependencies and semantic information of key attacks, thereby improving the efficiency of downstream security analysis without sacrificing detection accuracy.

[0009] To achieve the above objectives, the present invention adopts the following technical solution.

[0010] A source graph compression method with causal semantic consistency includes the following steps: Preprocess the system audit logs, construct an initial source graph, and remove duplicate redundant edges; Structural embedding features and semantic embedding features are extracted from the nodes of the initial source graph; Calculate the causal semantic consistency measure between nodes based on the structural embedding features and semantic embedding features; Based on the causal semantic consistency measure, the initial source graph is aggregated to obtain a compressed source graph.

[0011] Furthermore, the system audit logs are preprocessed to construct an initial source graph and remove duplicate redundant edges, including: Convert the system audit logs into an initial source graph, where nodes are system entities and edges are causal dependency events between entities; Duplicate edges with the same source node, destination node, and operation type within the same time slice are merged, while retaining the interaction frequency information.

[0012] Furthermore, structural embedding features and semantic embedding features are extracted from the nodes of the initial source graph, including: Based on the adjacency structure and node type features of the initial source graph, the structural embedding features of the nodes are learned through a graph convolutional network. The structural embedding features are used to characterize the k-hop neighborhood structure, causal interaction pattern and structural role of the nodes. The semantic attributes of edges in the initial source graph are encoded by a language model pre-trained in the security domain to obtain the semantic embedding of the edges; the semantic embeddings of all edges associated with a node are aggregated to obtain the semantic embedding features of the node, which are used to characterize the behavioral semantic features of the node.

[0013] Furthermore, the causal semantic consistency measure between nodes is calculated based on the structural embedding features and semantic embedding features, including: For each node, its structural embedding features and semantic embedding features are concatenated to obtain the node's joint features; Joint embedding of nodes is learned based on graph representation learning model; Calculate the cosine similarity of the joint embeddings of two nodes as a measure of causal semantic consistency between nodes.

[0014] Further, node aggregation is performed on the initial source graph based on the causal semantic consistency measure, including: Traverse the node pairs in the initial source graph and calculate the causal semantic consistency measure of the node pairs; When the causal semantic consistency measure of a node pair exceeds a preset threshold, the two nodes are merged into one aggregate node, inheriting the dependency edges of the original nodes and updating the node embeddings. The aggregate node fully inherits all incoming and outgoing edges from both original nodes. If duplicate edges with the same origin and destination appear, the edges are merged and the interaction frequency count is accumulated to preserve the original causal dependency relationship. The aggregate node updates both structural embeddings and semantic embeddings simultaneously. If two nodes A and B are merged into one aggregate node, the aggregation operation process is as follows: (1) Calculate the mean of the structural embeddings of node A and node B to generate a new structural embedding of the aggregate node; (2) Calculate the mean of the semantic embeddings of node A and node B to generate a new semantic embedding of the aggregate node; After aggregation, the new structural embedding is concatenated with the semantic embedding to regenerate the joint embedding of the aggregated node, which is used for subsequent consistency and similarity calculation with other nodes.

[0015] The above node aggregation steps are executed iteratively until no node pair that satisfies the aggregation conditions is found, resulting in a compressed source graph.

[0016] Furthermore, it also includes inputting the compressed initial source map into the downstream security analysis module for attack detection and security investigation.

[0017] A source graph compression system with causal semantic consistency includes: The preprocessing module is used to preprocess the system audit logs, construct an initial source graph, and remove duplicate redundant edges. The feature extraction module is used to extract structural embedding features and semantic embedding features from the nodes of the initial source graph, respectively. The consistency measurement module is used to calculate the causal semantic consistency measurement between nodes based on the structural embedding features and semantic embedding features. The graph compression module is used to aggregate nodes in the initial source graph based on the causal semantic consistency measure to obtain a compressed source graph.

[0018] Furthermore, the feature extraction module includes: The structural embedding unit is used to learn the structural embedding of nodes through a graph convolutional network based on the adjacency structure and node type features of the initial source graph. The semantic embedding unit is used to encode the semantic attributes of edges using a language model pre-trained in the security domain, and aggregate them to obtain the semantic embedding of nodes.

[0019] Furthermore, it also includes a downstream analysis module, which is used to apply the compressed source map to attack detection and security investigation, thereby improving the efficiency of security analysis.

[0020] The present invention has achieved the following beneficial effects.

[0021] 1. This invention employs a dual-dimensional joint compression mechanism, simultaneously considering the structural causal features and behavioral semantic features of nodes. This overcomes the shortcomings of single-dimensional compression, avoids the loss of key attack information, and ensures that the compressed graph still retains the complete attack propagation path.

[0022] 2. The causal semantic consistency measure proposed in this invention can safely guide node aggregation. Aggregation will only occur when nodes meet consistency in both dimensions. This can effectively merge redundant normal behaviors, achieving a compression ratio of up to 30 times, while also preventing malicious nodes from being merged incorrectly and fully preserving attack-related information.

[0023] 3. The compressed graph of the present invention can be directly used for downstream security analysis. Without sacrificing detection accuracy, it reduces the downstream detection time by 50%, significantly improves the efficiency of security analysis, and solves the problem of analysis efficiency of large-scale traceability graphs.

[0024] 4. The dual-dimensional module of the present invention is complementary. Structural compression eliminates structural redundancy, and semantic compression eliminates semantic redundancy. The combination of the two can achieve better results than the individual modules, taking into account both compression ratio and detection performance.

[0025] 5. The framework of this invention is universal and can be adapted to various downstream security analysis tasks. Whether it is attack detection or security investigation, the compressed graph can be used directly without modifying the downstream tasks, and it has good versatility and scalability. Attached Figure Description

[0026] Figure 1 This is a diagram of the overall architecture of the CSCProv framework.

[0027] Figure 2 This is a system diagram of the present invention.

[0028] Figure 3 This is a comparison chart of the compression ratio effects of different methods.

[0029] Figure 4 This is a comparison chart showing the time compression effects of different methods.

[0030] Figure 5Comparison of attack information retention effects after compression of the source map.

[0031] Figure 6 This is a comparison chart of the causal information loss rate after the source map is compressed. Detailed Implementation

[0032] To make the various technical features, advantages, or effects of the present invention more apparent and understandable, detailed descriptions are provided below through embodiments.

[0033] One embodiment of the present invention specifically proposes a source graph compression method with consistent causal semantics, such as... Figure 1 As shown, it includes the following steps: Step S1: Preprocess the system audit logs, construct an initial source graph, and remove duplicate redundant edges; Step S2: Extract structural embedding features and semantic embedding features from the nodes of the initial source graph, respectively; Step S3: Calculate the causal semantic consistency measure between nodes based on the structural embedding features and semantic embedding features; Step S4: Based on the causal semantic consistency measure, perform node aggregation on the initial source graph to obtain a compressed source graph; Step S5: Input the compressed source map into the downstream security analysis module for attack detection and security investigation.

[0034] In an optional embodiment of the present invention, step S1 involves preprocessing the system audit logs, constructing an initial source graph, and deduplicating redundant edges, including: Step S11: Convert the system audit log into an initial source graph, where nodes are system entities, including processes, files, network streams, memory objects, etc., and edges are causal dependency events between entities, such as process creation, file reading and writing, network connection, etc. Each edge contains attributes such as operation type and timestamp. Step S12: Normalize and deduplicate the edges, representing each edge as a quadruple (u, op, v, t). For repeated edges with the same source node, destination node, operation type, and within the same time slice, only one representative edge is retained. At the same time, the interaction count is recorded, eliminating redundancy while retaining the interaction frequency information.

[0035] In an optional embodiment of the present invention, step S2 involves extracting structural embedding features and semantic embedding features from the nodes of the initial source graph, including: Step S21, extract structural embedding features: Based on the adjacency structure and node type features of the initial source graph, the structural embedding of the node is learned through a graph convolutional network. This embedding can capture the k-hop neighborhood structure, causal interaction pattern and structural role of the node, reflecting the causal dependency features of the node. This invention employs a Topology-Specific Graph Convolutional Network (GCN), requiring only two types of pure structural features as input: the source graph adjacency matrix and node type labels. No textual semantic information is needed. During the network's forward propagation, the model automatically aggregates multi-order neighborhood information, eliminating the need for manual extraction of k-hop structures, interaction patterns, and structural roles. The acquisition method is as follows: (1) k-hop neighborhood: Each layer of graph convolution completes one-hop neighbor feature aggregation. After stacking L layers of GCN, the model can automatically capture all the adjacent topology within the range of L hops (k-hop) of a node. (2) Causal interaction mode: The edges of the source graph represent the causal dependencies between processes / files / networks. GCN transmits neighborhood interaction features layer by layer, implicitly encoding the interaction rules of nodes in long-term causal links; (3) Structural roles: The neighborhood topology distribution of different nodes (processes / files / sockets) has inherent differences. The network can automatically distinguish the topological roles of nodes in the graph (central hub, leaf node, etc.) through training.

[0036] Step S22, extract semantic embedding features: Encode the semantic attributes (command line parameters, file paths, network metadata, etc.) of the edges using the SecBERT model pre-trained in the security domain to obtain the semantic embedding of the edges; then aggregate the semantic embeddings of all edges associated with a node to obtain the semantic embedding of the node, which reflects the behavioral semantic features of the node.

[0037] In an optional embodiment of the present invention, step S3, which calculates a causal semantic consistency measure between nodes based on the structural embedding features and semantic embedding features, includes: Step S31: For each node, concatenate its structural embedding features and semantic embedding features to obtain the joint features of the node; Step S32: Based on the graph representation learning model, learn the joint embedding of nodes, which encodes both structural and semantic information. Step S33: Calculate the cosine similarity of the joint embeddings of the two nodes as a measure of causal semantic consistency between the nodes. The higher the similarity, the more suitable the node is for aggregation.

[0038] In an optional embodiment of the present invention, step S4, which involves aggregating nodes in the initial source graph based on the causal semantic consistency measure, includes: Step S41: Traverse the node pairs in the initial source graph and calculate the causal semantic consistency measure of the node pairs. Step S42: When the metric of a node pair exceeds a preset threshold, merge the two nodes into an aggregate node, inherit all the dependent edges of the original node, remove duplicate edges, and aggregate the joint embedding of the two nodes as the embedding of the new node. Step S43: Iteratively execute the above node aggregation steps until there are no more node pairs that meet the aggregation conditions, and obtain the compressed source graph.

[0039] In an optional embodiment of the present invention, in step S5, the compressed source tracing graph is input into a downstream security analysis module, such as an attack detection model. Since the size of the graph is greatly reduced, the detection time can be reduced by 50%, while the detection accuracy is almost the same as the original graph, without any loss.

[0040] like Figure 2 As shown, an embodiment of the present invention provides a source graph compression system based on causal semantic consistency, characterized in that it includes a preprocessing module, a feature extraction module, a consistency measurement module, and a graph compression module; The preprocessing module is used to construct an initial source map based on the system audit logs; The feature extraction module is used to extract the structural embedding features and semantic embedding features of the nodes in the initial source graph; The consistency measurement module is used to calculate the causal semantic consistency measurement between nodes based on the structural embedding features and semantic embedding features of the nodes. The graph compression module is used to aggregate the nodes in the initial source graph based on the causal semantic consistency measure to obtain a compressed source graph.

[0041] Preferably, it also includes a downstream analysis module for using the compressed source map for attack detection and security investigation, thereby improving the efficiency of security analysis.

[0042] An embodiment of the present invention provides a computing device, characterized in that it includes: a processor and a memory storing a computer program, wherein the computer program is executed by the processor to perform the above-described method.

[0043] An embodiment of the present invention provides a computer-readable storage medium, characterized in that it stores instructions that, when executed on a computer, cause the computer to perform the above-described method.

[0044] A higher compression ratio indicates greater graph compression and better lightweighting; a lower compression time indicates lower computational overhead and stronger real-time performance; a lower attack information loss (AIL) percentage indicates less loss of attack-related events during compression; a lower causal information loss (CIL) percentage indicates more complete preservation of causal dependencies in the system; the comparison baselines ProTracer, TAPAS, PrioTracker, and ProvGRP were selected, and this invention is CSCProv.

[0045] The image shows the effect of compression and lightweighting. Figure 3 As shown, on the entire dataset (CADETS / THEIA / CLEARSCOPE / FIVEDIRECTIONS / TRACE), CSCProv's compression ratio is significantly higher than all the comparison algorithms; ProvGRP's compression capability is second, followed by TAPAS, PrioTracker, and ProTracer, whose compression ratios decrease in that order. Advantages: This invention alone can achieve 30x compression of the source graph, which can significantly reduce the number of nodes and edges, greatly reduce storage consumption, and solve the industry pain points of storage explosion and excessive downstream detection computing power caused by the generation of massive source graphs from audit logs.

[0046] Compressed computing efficiency is as follows Figure 4 As shown, ProTracer has the lowest compression time, but the compression margin is extremely small; ProvGRP has a decent compression ratio, but the compression time is significantly higher for the entire dataset, resulting in high computational costs; TAPAS and PrioTracker have moderate compression times; CSCProv's compression time is only higher than the minimalist ProTracer, but much lower than ProvGRP, TAPAS, and PrioTracker. The advantages of this invention are: achieving the optimal compression ratio with only a small computational overhead, balancing high compression ratio and low time consumption, while also considering lightweight design and real-time engineering deployment, making it suitable for large-scale online log preprocessing scenarios on enterprise hosts.

[0047] The ability to retain attack event information, such as Figure 5As shown, AIL represents the percentage of attack events lost during compression, with lower values ​​being better. The AIL value of CSCProv on all datasets remains stable in the range of 4% to 5%. The AIL of TAPAS is concentrated in the range of 7.5% to 9.5%, while that of ProvGRP is as high as 12% to 14.5%. Advantages: The attack-related events lost in this invention are far fewer than those in existing one-dimensional compression schemes. It does not filter out key attack behaviors such as APT penetration, lateral movement, and data theft due to large-scale compression, thus ensuring the integrity of the data source for subsequent attack detection, forensics, and tracing.

[0048] System causal dependency preservation capability, such as Figure 6 As mentioned above, CIL represents the proportion of causal edges lost during compression; the lower the value, the more complete the attack propagation chain. The CIL of the entire CSCProv dataset is only 6%~8.5%; the CIL of TAPAS is 10.5%~13.8%, and the CIL of ProvGRP reaches 17.5%~20.5%. Advantages: Existing compression methods based solely on structure / semantics will significantly destroy the causal relationships between processes, files, and networks; while this invention adopts causal semantic double consistency constraint aggregation, which preserves the complete attack propagation chain to the greatest extent. Analysts can rely on the compressed graph to completely reproduce the entire APT attack path.

[0049] Summary of the overall beneficial effects of this invention: 1. Compression performance advantage: Compared with the existing four mainstream source map compression schemes such as ProTracer, TAPAS, PrioTracker and ProvGRP, the present invention CSCProv can achieve up to 30 times compression of source maps, which greatly reduces the storage and transmission costs of massive audit maps; 2. Balanced computational efficiency: At ultra-high compression ratios, the compression operation time is much lower than that of ProvGRP, TAPAS, and PrioTracker, balancing lightweight and real-time processing requirements, and lowering the computational threshold for engineering deployment. 3. Low loss of attack information: The two key indicators of attack information loss (AIL) and causal dependency loss (CIL) are significantly better than existing methods. The compression process discards very few attack events and does not disrupt the causal propagation link of the attack. 4. Addressing existing technological shortcomings: Overcoming the two core shortcomings of pure structural compression being easily masked by covert attacks and pure semantic compression being easily bypassed by LotL legitimate process spoofing attacks, in large-scale enterprise source tracing and analysis scenarios, it simultaneously meets the two core requirements of graph lightweighting and complete preservation of attack information, thereby improving the accuracy and analysis efficiency of downstream APT detection and security forensics.

[0050] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. A source graph compression method based on causal semantic consistency, comprising the following steps: Construct an initial source map based on the system audit logs; For each node in the initial source graph, extract the structural embedding features and semantic embedding features of the node respectively; Calculate the causal semantic consistency measure between nodes based on the structural embedding features and semantic embedding features of nodes; The nodes of the initial source graph are aggregated based on the causal semantic consistency measure to obtain a compressed source graph.

2. The method according to claim 1, characterized in that, The steps for constructing the initial source graph include: using entities in the system audit log as nodes, establishing edges between entities with causal dependencies; merging edges with the same source node, destination node, and operation type within the same time slice, and retaining interaction frequency information to obtain the initial source graph.

3. The method according to claim 1 or 2, characterized in that, The steps for extracting the structural embedding features and semantic embedding features of the nodes include: learning the structural embedding features of the nodes through a graph convolutional network based on the adjacency structure and node type of the initial source graph; the structural embedding features are used to characterize the k-hop neighborhood structure, causal interaction pattern and structural role of the nodes; encoding the semantic attributes of the edges in the initial source graph through a language model pre-trained in the security domain to obtain the semantic embedding of the edges; aggregating the semantic embeddings of all edges associated with the nodes to obtain the semantic embedding features of the nodes; the semantic embedding features are used to characterize the behavioral semantic features of the nodes.

4. The method according to claim 1, characterized in that, The steps for calculating the causal semantic consistency measure between nodes include: concatenating the structural embedding features and semantic embedding features of each node to obtain the joint features of the node, and inputting them into the graph representation learning model to obtain the joint embedding of the node; calculating the cosine similarity of the joint embeddings of two nodes as the causal semantic consistency measure between nodes.

5. The method according to claim 1, characterized in that, The steps for aggregating nodes in the initial source graph based on the causal semantic consistency measure include: traversing the node pairs in the initial source graph and calculating the causal semantic consistency measure for each node pair; when the causal semantic consistency measure of a node pair exceeds a preset threshold, merging the node pair into an aggregate node, which inherits the dependency edges of the node pair, and generating the structural embedding features and semantic embedding features of the aggregate node based on the structural embedding features and semantic embedding features of the node pair, thus completing one node aggregation; iteratively executing node aggregation until there are no node pairs that satisfy the aggregation conditions, thereby obtaining the compressed source graph.

6. The method according to claim 1, characterized in that, The compressed source map is input into the downstream security analysis module for attack detection and security investigation.

7. A source graph compression system based on causal semantic consistency, characterized in that, It includes a preprocessing module, a feature extraction module, a consistency measurement module, and a graph compression module; The preprocessing module is used to construct an initial source map based on the system audit logs; The feature extraction module is used to extract the structural embedding features and semantic embedding features of the nodes in the initial source graph; The consistency measurement module is used to calculate the causal semantic consistency measurement between nodes based on the structural embedding features and semantic embedding features of the nodes. The graph compression module is used to aggregate the nodes in the initial source graph based on the causal semantic consistency measure to obtain a compressed source graph.

8. The system according to claim 7, characterized in that, It also includes a downstream analysis module, which uses the compressed source map for attack detection and security investigation, improving the efficiency of security analysis.

9. A computing device, characterized in that, include: A processor, a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, A storage instruction that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1 to 6.

Images