Traceability graph compression method and device, electronic equipment and storage medium

Abstract

This application relates to a source graph compression method, apparatus, electronic device, and storage medium, applied in the field of network security technology. The method includes: collecting audit log data from an application during its operation; constructing a source graph to be compressed based on the audit log data; wherein nodes in the source graph represent entities in the audit log data, and directed edges in the source graph represent the relationships between entities; matching isomorphic portions of the source graph to be compressed as regions to be compressed based on multiple template subgraphs in a pre-built template library; wherein the template subgraphs describe the normal behavior patterns of the application; and performing lossy compression on the regions to be compressed to obtain the compressed source graph. This application can effectively compress data while retaining key information.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to a method, apparatus, electronic device and storage medium for compressing traceability graphs. Background Technology

[0002] In recent years, major cybersecurity incidents have occurred frequently both domestically and internationally. APT (Advanced Persistent Threat) attacks, due to their highly organized, targeted, and meticulously planned nature, are not only highly threatening but also difficult to detect, making APT detection a challenging task. Network situational awareness technologies, which analyze the overall security situation of a system from a global perspective, have emerged to address this need.

[0003] Key technologies for situational awareness include data acquisition, anomaly detection, and attack attribution. Through data acquisition, security systems can comprehensively monitor and analyze the network environment, detecting network attacks and threats. Anomaly detection models based on machine learning and deep learning algorithms analyze and mine various data within the network environment to identify abnormal events, ensuring that security systems can take timely and appropriate measures. Using the anomaly detection results, network attack attribution algorithms analyze and mine network attack data to identify attack sources and paths, providing a foundation for subsequent countermeasures.

[0004] Because APT attacks typically last for extended periods, preserving a comprehensive historical record of system events is crucial. However, with such massive amounts of data, enterprises may have to discard all data periodically. This discarded data may contain critical information, rendering attack attribution ineffective. Furthermore, attack detection and investigation algorithms become slow, and delayed anomaly detection and time-consuming, manpower-intensive attack attribution become unacceptable. Therefore, optimizing and reducing data while retaining key and valid information to meet situational awareness requirements has become a pressing technical challenge. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a traceability graph compression method, apparatus, electronic device, and storage medium.

[0006] According to a first aspect of this application, a source graph compression method is provided, comprising:

[0007] Collect audit log data during application runtime;

[0008] A source graph to be compressed is constructed based on the audit log data; wherein, the nodes in the source graph to be compressed represent entities in the audit log data, and the directed edges in the source graph to be compressed represent the relationships between the entities;

[0009] Based on multiple template subgraphs in a pre-built template library, a portion isomorphic to the template subgraph is matched from the source graph to be compressed as the region to be compressed; wherein, the template subgraph is used to describe the normal behavior pattern of the application;

[0010] The region to be compressed is subjected to lossy compression to obtain the compressed traceability map.

[0011] Optionally, the method for constructing the template library includes:

[0012] Run test cases on the application and collect audit log data during the execution of each test case;

[0013] A source graph is constructed based on the audit log data corresponding to each test case, resulting in a source graph set for each test case. In the source graph set, each node in the source graph represents an entity in the audit log data, and the directed edges in the source graph represent the relationships between the entities.

[0014] For each test case, a subgraph mining algorithm is used to mine frequent items from multiple source graphs in the source graph set to obtain the template subgraph corresponding to the source graph set.

[0015] The set of template subgraphs corresponding to multiple source tracing graph sets is used as a template library.

[0016] Optionally, the step of performing lossy compression on the region to be compressed to obtain the compressed traceability map includes:

[0017] Assume that the source graph to be compressed is isomorphic to N template subgraphs, and the N template subgraphs correspond to M applications, where N and M are both positive integers, and N is greater than or equal to M;

[0018] For a single template subgraph, the application node corresponding to the template subgraph is taken as the first node, and a new second node is generated according to the template information of the template subgraph. The second node represents all nodes of the region to be compressed corresponding to the template subgraph.

[0019] The first node and the second node are connected by an edge, wherein the edge between the first node and the second node is generated according to the template information;

[0020] Delete the original nodes and edges within the region to be compressed corresponding to the template subgraph, and reconnect the nodes outside the region to be compressed that are connected to the original nodes within the region to be compressed to the first node and the second node.

[0021] Specifically, if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is a predecessor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is a predecessor node of the first node; if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is a successor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is a successor node of the second node.

[0022] Optionally, the step of using a subgraph mining algorithm to perform frequent item mining on multiple source graphs in the source graph set to obtain a template subgraph corresponding to the test case includes:

[0023] Using the gSpan algorithm, frequent item mining is performed on multiple source graphs in the source graph set to obtain the template subgraph corresponding to the test case.

[0024] Optionally, matching the portion isomorphic to the template subgraph from the source graph to be compressed as the region to be compressed includes:

[0025] Using the VF2 algorithm, the regions that are isomorphic to the template subgraph are matched from the source graph to be compressed as regions to be compressed.

[0026] Optionally, constructing the source map to be compressed based on the audit log data includes:

[0027] Entities are extracted from the audit log data, and each extracted entity is used as a node;

[0028] The relationships between entities are parsed from the audit log data, and the relationships between entities are represented by directed edges;

[0029] Add attribute information to each node and each directed edge based on the audit log data.

[0030] Optionally, the audit log data collected during the operation of the application includes:

[0031] Start the data collector and define the filtering rules, then start the container containing the test cases;

[0032] Execute the test cases, and after waiting for a preset time period, stop and destroy the container;

[0033] Shut down the data collector and save the output audit log data.

[0034] According to a second aspect of this application, a traceability map compression apparatus is provided, comprising:

[0035] The audit log data collection module is used to collect audit log data during the application's operation.

[0036] The source graph construction module is used to construct a source graph to be compressed based on the audit log data; wherein, the nodes in the source graph to be compressed represent entities in the audit log data, and the directed edges in the source graph to be compressed represent the relationships between the entities;

[0037] The region to be compressed module is used to match the portion isomorphic to the template subgraph from the source graph to be compressed as the region to be compressed, based on multiple template subgraphs in a pre-built template library; wherein the template subgraph is used to describe the normal behavior pattern of the application.

[0038] The lossy compression module is used to perform lossy compression on the area to be compressed to obtain the compressed traceability map.

[0039] Optionally, the audit log data collection module is also used to run test cases on the application and collect audit log data during the execution of each test case;

[0040] The source graph construction module is also used to construct a source graph based on the audit log data corresponding to each test case, thereby obtaining a source graph set corresponding to each test case; wherein, in the source graph set, each node in the source graph represents an entity in the audit log data, and the directed edges in the source graph represent the association relationship between the entities;

[0041] The source map compression device further includes:

[0042] The template subgraph mining module is used to perform frequent item mining on multiple traceability graphs in the traceability graph set corresponding to each test case using a subgraph mining algorithm to obtain the template subgraph corresponding to the traceability graph set.

[0043] The template library building module is used to create a collection of template subgraphs corresponding to multiple source map sets as a template library.

[0044] Optionally, the lossy compression module is specifically configured to assume that the source graph to be compressed is isomorphic to N template subgraphs, the N template subgraphs corresponding to M applications, where N and M are both positive integers, and N is greater than or equal to M; and,

[0045] For a single template subgraph, the application node corresponding to the template subgraph is taken as the first node, and a new second node is generated based on the template information of the template subgraph. The second node represents all nodes in the region to be compressed corresponding to the template subgraph; and,

[0046] The first node and the second node are connected by an edge, wherein the edge between the first node and the second node is generated based on the template information; and...

[0047] Delete the original nodes and edges within the region to be compressed corresponding to the template subgraph, and reconnect the nodes outside the region to be compressed that are connected to the original nodes within the region to be compressed to the first node and the second node.

[0048] Specifically, if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is a predecessor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is a predecessor node of the first node; if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is a successor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is a successor node of the second node.

[0049] Optionally, the template subgraph mining module is specifically used to perform frequent item mining on multiple traceability graphs in the traceability graph set corresponding to each test case using the gSpan algorithm, so as to obtain the template subgraph corresponding to the test case.

[0050] Optionally, the region to be compressed determination module is specifically used to match the portion isomorphic to the template subgraph from the source graph to be compressed as the region to be compressed, based on multiple template subgraphs in a pre-built template library and using the VF2 algorithm.

[0051] Optionally, the source graph construction module is specifically used to extract entities from the audit log data and use each extracted entity as a node; parse the relationship between each entity from the audit log data and represent the relationship between each entity through directed edges; and add attribute information to each node and each directed edge according to the audit log data.

[0052] Optionally, the audit log data acquisition module is specifically used to start the data collector and formulate filtering rules, start the container containing test cases, execute the test cases, and after waiting for a preset time period, stop and destroy the container; close the data collector and save the output audit log data.

[0053] According to a third aspect of this application, an electronic device is provided, comprising: a processor configured to execute a computer program stored in a memory, wherein the computer program, when executed by the processor, implements the method described in the first aspect.

[0054] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0055] According to a fifth aspect of this application, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to perform the method described in the first aspect.

[0056] The technical solution provided in this application has the following advantages compared with the prior art:

[0057] By collecting audit log data during application runtime, a source map to be compressed is constructed based on the audit log data. Using multiple template subgraphs from a pre-built template library, isomorphic portions of the source map to be compressed are matched as regions to be compressed. Lossy compression is then applied to these regions. The compressed source map can then be used as input to downstream anomaly detection tasks. Since the template subgraphs are obtained by executing test cases in the application, collecting corresponding audit log data, constructing a source map corresponding to the audit log data, and mining the source map using a template subgraph mining algorithm, the template subgraphs learn the application's behavioral patterns. Using these behavioral patterns for lossy compression can effectively reduce the data size while preserving key information, thereby effectively reducing the burden on downstream tasks without causing negative impacts. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of a template library construction method in an embodiment of this application;

[0061] Figure 2 This is a schematic diagram illustrating the construction of a template library in an embodiment of this application;

[0062] Figure 3 This is a schematic diagram of template subgraph mining in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram of the compression process in an embodiment of this application;

[0064] Figure 5 This is a flowchart of a source graph compression method in an embodiment of this application;

[0065] Figure 6 This is a compression diagram illustrating a scenario where a template subgraph is successfully matched in an embodiment of this application.

[0066] Figure 7 This is a compression diagram illustrating a scenario in which multiple template subgraphs corresponding to the same application are successfully matched in an embodiment of this application.

[0067] Figure 8 This is a compression diagram illustrating a scenario in which multiple template sub-graphs corresponding to different applications are successfully matched in an embodiment of this application.

[0068] Figure 9 This is a schematic diagram of a source graph compression method in an embodiment of this application;

[0069] Figure 10 This is a schematic diagram of the structure of an electronic device in an embodiment of this application. Detailed Implementation

[0070] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0071] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.

[0072] Data compression can essentially be broken down into two parts: identifying compressible content within the data and how to reasonably reduce that content while retaining key information. Understandably, data unrelated to attacks (i.e., benign data) is compressible. The definition of benign data can be derived using high-level semantics; therefore, the compressed content can be content that is semantically unrelated to attacks. To ensure that the compressed result does not negatively impact subsequent attack investigations, high-level semantic attributes are often added to the compressed data to allow security personnel to understand the general state before compression.

[0073] Source graphs, as a powerful threat modeling tool, can be used to identify compressible content in data. Subgraphs within a source graph often represent different behaviors containing high-level semantics, allowing for the differentiation between normal and anomalous behaviors.

[0074] Based on this, software testing methods can be used to generate template subgraphs, which can accurately describe the behaviors corresponding to test cases. First, normal test cases in software testing do not include attack behaviors, so the generated source diagram will only include normal behaviors. Second, test cases in software testing target a specific function in the software, and their execution is highly similar to that of users; therefore, the generated source diagram will also be highly similar to the actual source diagram. Finally, software development often provides rich and comprehensive test cases, which can improve the coverage of normal behaviors.

[0075] However, due to the numerous random factors involved in the execution process, directly using the source graph generated from a single test case execution as the template subgraph for this behavior is inappropriate. To reduce randomness and ensure the representativeness of the template subgraph, frequent item mining (FOM) methods from data mining can be used. First, multiple source graphs are obtained by executing multiple test cases. Then, a frequent item mining algorithm is used to mine the common substructures on these source graphs. Finally, the mined parts are filtered to obtain the template subgraph.

[0076] See Figure 1 , Figure 1 This is a flowchart of a method for constructing a template library in an embodiment of this application, which may include the following steps:

[0077] Step S102: Run test cases on the application and collect audit log data during the execution of each test case.

[0078] In this embodiment, since repeated testing of different applications and test cases is required, manual execution is too time-consuming. Therefore, a script can be used to control the data collection process. That is, test cases are run automatically, and a data collector (such as the Sysdig collector) is used to automate data collection. The data collection process can specifically be as follows:

[0079] The data collection environment is built using containers, which contain the application and test cases for testing. During data collection, the system first starts the data collector, then starts the container and executes the corresponding test cases within the container, and finally shuts down the container and the data collector, and reclaims the collected log files containing audit log data.

[0080] Step S104: Construct a source graph based on the audit log data corresponding to each test case to obtain a source graph set corresponding to each test case; wherein, each node in the source graph set represents an entity in the audit log data, and the directed edges in the source graph represent the association between the entities.

[0081] By executing test cases containing semantics, audit log data representing specific behaviors can be obtained. Based on this audit log data, a source graph can be constructed. The two key components of a source graph are nodes representing entities and edges representing the relationships between entities; therefore, the method for constructing a source graph is as follows:

[0082] Entities are extracted from audit log data, and each extracted entity is treated as a node. Examples include processes, files, IPv4 sockets, IPv6 sockets, pipes, and inter-process communication sockets. Other entities, being less common and irrelevant to mainstream security analysis models, can be filtered during algorithm execution.

[0083] Next, the relationships between entities are parsed from the audit log data, and these relationships are represented by directed edges. Attribute information is then added to each node and directed edge based on the audit log data. Node attributes include: a unique identifier, node type, host location, and, if the node is a process, process ID and process name; if the node is a file, file path. Edge attributes mainly include: timestamp, system call, source and target nodes, and their information, effectively describing the order of events, the flow of information, and event details, thus supporting subsequent tasks.

[0084] In the constructed source graph, a node represents an entity, including processes, files, sockets, and hosts, and has a unique identifier and other attributes related to its specific entity type. For example, a file has a path attribute, a process has a process name attribute, and a socket has an IP address attribute. Directed edges represent events between two nodes, commonly including information flow events such as read and write, and control flow events such as clone and create. An edge contains at least four elements: a unique identifier, unique identifiers for the source and target nodes, and a timestamp, and may also have other attributes. The timestamp is important because it stores information about the order of events, which can provide significant assistance for causal analysis.

[0085] Step S106: For each test case, the source graph set is used to perform frequent item mining on multiple source graphs in the source graph set using a subgraph mining algorithm to obtain the template subgraph corresponding to the source graph set.

[0086] After obtaining multiple source graphs corresponding to each test case, the template subgraph mining algorithm can be used to extract the template subgraph corresponding to the test case from the multiple source graphs. Template subgraph mining is essentially a frequent subgraph mining problem. The main challenges of this problem lie in the generation of the candidate set and the removal of false positives. The former challenge is that a single graph can generate a large number of subgraphs, while the latter challenge is that verifying graph isomorphism is an NP-hard problem.

[0087] The gSpan algorithm is a pattern growth-based algorithm that, instead of generating candidates and pruning false positives during operation, integrates subgraph expansion and verification into a single process, thus accelerating the mining process. Therefore, as a fundamental subgraph mining algorithm, gSpan can effectively address the two challenges mentioned above. Optionally, the gSpan algorithm can be used to perform frequent item mining on multiple source graphs in the source graph set to obtain template subgraphs corresponding to test cases.

[0088] Compared to other methods, the gSpan algorithm does not require generating a candidate set. All frequent subgraphs are derived from graphs with fewer edges, thus eliminating the need to test these candidates. This method uses depth-first search, and although its worst-case efficiency is the same as breadth-first search, it effectively saves space. The algorithm continuously shrinks the dataset during runtime, a process that improves efficiency. The gSpan algorithm's processing steps are as follows:

[0089] Remove infrequent nodes and edges from the dataset, and then re-encode the remaining edges using the method described above and put them into the output set of the saved results in order (a frequent graph containing only one edge is also a frequent subgraph).

[0090] For each graph with a frequent edge, the SubMining growth algorithm is iteratively called to continuously find more frequent subgraphs with more edges.

[0091] Remove edges from the dataset that are identical to the edges that were mined, thereby further reducing the size.

[0092] Finally, the gSpan algorithm checks the support; it terminates when the support of a graph in the dataset falls below that level.

[0093] Step S108: Use the set of template subgraphs corresponding to multiple source tracing graph sets as a template library.

[0094] Figure 2This is a schematic diagram illustrating the construction of a template library in an embodiment of this application. First, a data acquisition environment can be built using Docker technology. Then, after starting the Sysdig collector, test cases are executed within the container to obtain audit log data during test case execution. The entire process is controlled by a script and executed multiple times. Next, a source graph construction algorithm is used to construct a source graph from the audit logs. During this process, a large amount of irrelevant information is removed, retaining only key information about the relationships between entities. Finally, a template subgraph mining algorithm based on gSpan is used to perform frequent item mining on the source graph set corresponding to each test case, ultimately obtaining the template subgraph corresponding to the source graph set. Figure 3 This demonstrates the process of extracting template subgraphs from a source graph set containing 10 source graphs in actual operation.

[0095] After building a template library containing multiple template subgraphs, data compression can be performed based on these subgraphs. See also... Figure 4 , Figure 4 This is a schematic diagram of the compression process in an embodiment of this application. First, audit log data during the application's operation is acquired through a data collector; a source map to be compressed is constructed based on the audit log data. Then, template subgraphs from the template library and compression algorithms are used to compress the source map. Finally, the compressed log data (i.e., the source map) is sent to downstream tasks for anomaly detection and attack investigation.

[0096] See Figure 5 , Figure 5 This is a flowchart of a source graph compression method in an embodiment of this application, which may include the following steps:

[0097] Step S502: Collect audit log data of the application during its operation.

[0098] This step can be the same as step S102, namely, start the data collector and formulate filtering rules, start the container containing test cases; execute the test cases, and after waiting for a preset time period, stop and destroy the container; close the data collector and save the output audit log data.

[0099] Step S504: Construct a source graph to be compressed based on the audit log data; wherein, the nodes in the source graph to be compressed represent entities in the audit log data, and the directed edges in the source graph to be compressed represent the relationships between the entities.

[0100] Entities can be extracted from the audit log data, and each extracted entity can be used as a node. The relationships between the entities can be parsed from the audit log data and represented by directed edges. Attribute information can be added to each node and each directed edge based on the audit log data. This step is the same as the process in step S104, and the details can be found in the description of step S104, which will not be repeated here.

[0101] Step S506: Based on multiple template subgraphs in the pre-built template library, match the portion isomorphic to the template subgraph from the source graph to be compressed as the region to be compressed; wherein, the template subgraph is used to describe the normal behavior pattern of the application.

[0102] After obtaining the template subgraphs, a method is needed to identify isomorphic parts of the source graph to be compressed that are identical to these template subgraphs in order to complete the compression. This problem is essentially a subgraph isomorphism problem, which determines whether there is a mapping relationship between two graphs such that all nodes and edges of one graph can be mapped to nodes and edges of the other graph while maintaining the association relationship.

[0103] The VF2 algorithm is an exact matching algorithm for solving the subgraph isomorphism problem, and it outperforms other similar algorithms on large graphs. Based on a recursive approach, the algorithm divides the matching process into forward and backtracking. During the forward process, the algorithm attempts to match nodes on two graphs and verifies whether the matching conditions are met. If successful, it moves forward further; if unsuccessful, it backtracks to try other nodes until no more nodes can be used for forward matching. Therefore, this embodiment of the application can utilize the VF2 algorithm to match isomorphic portions of the source graph to be compressed as the region to be compressed.

[0104] This application adds the application name attribute to specific nodes (application nodes executing test cases) in the template subgraph and sets them as the initial matching node. It also incorporates a check on the initial node name during semantic verification. When there are many template subgraphs in the template library, this operation effectively reduces invalid matches and significantly improves algorithm efficiency. During matching, the source graph to be compressed is matched against each template subgraph in the template library. The matching positions of successfully matched template subgraphs and the source graph to be compressed are recorded, and this information is used in the next compression process. For example, the nodes and edges corresponding to the template subgraphs in the source graph to be compressed can be recorded, and these recorded nodes and edges can be used as the region to be compressed.

[0105] Step S508: Perform lossy compression on the region to be compressed to obtain the compressed traceability map.

[0106] After obtaining the region to be compressed in the source graph, lossy compression can be performed on the region. Lossy compression aims to reduce the number of nodes and edges as much as possible while ensuring no semantic loss. The region to be compressed is actually a description of a single action performed by the application; therefore, from the perspective of information flow, the information flow within the region to be compressed can be regarded as a whole.

[0107] Optionally, the region to be compressed can be reduced to two nodes and one edge. One node is the application node corresponding to the template subgraph, and the other node is a new node generated based on the template information, representing all nodes in the region to be compressed. The two nodes are connected by an edge, which is also generated based on the template information, and its attributes include the behavioral semantics represented by the template. After generating the new node and edge, the system deletes the original nodes and edges and reconnects the nodes outside the region to be compressed that were previously connected to these nodes to the application node and the new node.

[0108] Assume the source graph to be compressed is isomorphic to N template subgraphs, and the N template subgraphs correspond to M applications, where N and M are both positive integers, and N is greater than or equal to M. When N is greater than M, it indicates a scenario where multiple template subgraphs correspond to the same application; when N is equal to M, it indicates that each template subgraph corresponds to a different application.

[0109] For a single template subgraph, the application node corresponding to the template subgraph is taken as the first node, and a new second node is generated based on the template information of the template subgraph. The second node represents all nodes in the region to be compressed corresponding to the template subgraph. The first node and the second node are connected by an edge, which is generated based on the template information.

[0110] Delete the existing nodes and edges within the region to be compressed corresponding to the template subgraph, and reconnect the nodes outside the region that are connected to the existing nodes within the region to the first and second nodes. The rules for connecting the nodes outside the region that are connected to the existing nodes within the region to the first or second node are as follows:

[0111] If a node outside the region to be compressed that is connected to an existing node within the region to be compressed is the predecessor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is the predecessor node of the first node. If a node outside the region to be compressed that is connected to an existing node within the region to be compressed is the successor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is the successor node of the second node.

[0112] These two rules preserve the original outgoing and incoming edges of the compressed region, effectively ensuring that the external information flow remains unchanged due to compression. See also Figure 6 Assuming a template subgraph is matched, corresponding to a region to be compressed (i.e. Figure 6 The area enclosed by the dashed line is compressed into two nodes and one edge. The original relationships outside the area are preserved. In the figure, node M is the application node corresponding to the template, and node N is the node newly added during compression.

[0113] See Figure 7 Suppose two template subgraphs are matched, corresponding to two regions to be compressed (i.e. Figure 7 (The area enclosed by the dashed line). If two areas to be compressed overlap and the two template subgraphs correspond to the same application, the application node can be retained while two new nodes are added. Subsequent processing follows the same procedure. Figure 6 The processing method for matching a template subgraph is similar: connect the original predecessor node to the application node, and connect the original successor node to the two new nodes respectively.

[0114] See Figure 8 Suppose two template subgraphs are matched, corresponding to two regions to be compressed (i.e. Figure 7 (The area enclosed by the dashed line) If two areas to be compressed overlap and the applications corresponding to the two template subgraphs are different, the application nodes can be retained in each of the two areas to be compressed, and new nodes can be added. Connect the application nodes and new nodes in each area to be compressed, connect the original predecessor nodes to the application nodes, and connect the original successor nodes to the two new nodes respectively. It can be seen that its processing is similar to... Figure 6 The processing method for matching a template subgraph is similar.

[0115] The entire compression process essentially replaces the portion of the source graph to be compressed that is isomorphic to the template subgraph with two nodes and one edge. These two nodes represent the entry and exit points of the subgraph region, respectively, while the edge represents the activity within that region. Experiments revealed that the number of edges with application nodes as source nodes is significantly greater than the number of edges with application nodes as target nodes, indicating that application nodes are often upstream in the information flow. Therefore, this embodiment uses application nodes as entry nodes and newly constructed nodes as exit nodes. This method preserves critical information within the information flow, thus providing a reliable data foundation for subsequent anomaly detection and attack investigations while simultaneously completing the compression process.

[0116] The source graph compression method in this application collects audit log data during application runtime and constructs a source graph to be compressed based on the audit log data. Using multiple template subgraphs from a pre-built template library, isomorphic portions of the source graph to be compressed are matched as regions to be compressed. These regions are then subjected to lossy compression. The compressed source graph can then be used as input to downstream anomaly detection tasks. Since the template subgraphs are obtained by executing test cases, collecting corresponding audit log data, constructing a source graph corresponding to the audit log data, and mining the source graph using a template subgraph mining algorithm, the template subgraphs learn the application's behavioral patterns. Using these behavioral patterns for lossy compression can effectively reduce the data size while preserving key information, thereby effectively reducing the burden on downstream tasks without causing negative impacts.

[0117] Corresponding to the above method embodiments, this application also provides a traceability map compression device, see [link to relevant documentation]. Figure 9 The traceability image compression device 900 includes:

[0118] The audit log data acquisition module 902 is used to collect audit log data during the operation of the application.

[0119] The source graph construction module 904 is used to construct a source graph to be compressed based on audit log data; wherein, the nodes in the source graph to be compressed represent entities in the audit log data, and the directed edges in the source graph to be compressed represent the relationships between the entities;

[0120] The region to be compressed determination module 906 is used to match the part that is isomorphic to the template subgraph from the source graph to be compressed as the region to be compressed, based on multiple template subgraphs in a pre-built template library; wherein, the template subgraph is used to describe the normal behavior pattern of the application.

[0121] The lossy compression module 908 is used to perform lossy compression on the area to be compressed, and obtain the compressed trace map.

[0122] Optionally, the audit log data acquisition module 902 is also used to run test cases on the application and collect audit log data during the execution of each test case;

[0123] The source graph construction module 904 is also used to construct a source graph based on the audit log data corresponding to each test case, and obtain a source graph set corresponding to each test case; wherein, the nodes in each source graph in the source graph set represent entities in the audit log data, and the directed edges in the source graph represent the association relationships between the entities;

[0124] The traceability map compression device 900 also includes:

[0125] The template subgraph mining module is used to mine frequent items from multiple traceability graphs in the traceability graph set corresponding to each test case using a subgraph mining algorithm to obtain the template subgraph corresponding to the traceability graph set.

[0126] The template library building module is used to create a collection of template subgraphs corresponding to multiple source map sets as a template library.

[0127] Optionally, the lossy compression module 908 is specifically configured to assume that the source graph to be compressed is isomorphic to N template subgraphs, the N template subgraphs correspond to M applications, N and M are both positive integers, and N is greater than or equal to M; and,

[0128] For a single template subgraph, the application node corresponding to the template subgraph is used as the first node, and a new second node is generated based on the template information of the template subgraph. The second node represents all nodes in the region to be compressed corresponding to the template subgraph; and,

[0129] Connect the first node and the second node with an edge, wherein the edge between the first node and the second node is generated based on template information; and,

[0130] Delete the original nodes and edges within the region to be compressed corresponding to the template subgraph, and reconnect the nodes outside the region to be compressed that are connected to the original nodes within the region to be compressed to the first node and the second node.

[0131] Specifically, if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is the predecessor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is the predecessor node of the first node; if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is the successor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is the successor node of the second node.

[0132] Optionally, the template subgraph mining module is specifically used to perform frequent item mining on multiple traceability graphs in the traceability graph set corresponding to each test case using the gSpan algorithm, so as to obtain the template subgraph corresponding to the test case.

[0133] Optionally, the region to be compressed determination module 906 is specifically used to match the part that is isomorphic to the template subgraph from the source graph to be compressed as the region to be compressed, based on multiple template subgraphs in a pre-built template library and using the VF2 algorithm.

[0134] Optionally, the traceability graph construction module 904 is specifically used to extract entities from audit log data and use each extracted entity as a node; parse the relationship between each entity from the audit log data and represent the relationship between each entity through directed edges; and add attribute information to each node and each directed edge according to the audit log data.

[0135] Optionally, the audit log data acquisition module 902 is specifically used to start the data collector and formulate filtering rules, start the container containing test cases, execute the test cases, and after waiting for a preset time period, stop and destroy the container; close the data collector and save the output audit log data.

[0136] The specific details of each module or unit in the above-mentioned device have been described in detail in the corresponding methods, so they will not be repeated here.

[0137] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0138] This application also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the traceability graph compression method described in this example embodiment.

[0139] Reference Figure 10 , Figure 10 This is a schematic diagram of the structure of an electronic device in an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0140] like Figure 10 As shown, the electronic device may include: a processor 1002, a communication interface 1004, a memory 1006, and a communication bus 1008.

[0141] The processor 1002, communication interface 1004, and memory 1006 communicate with each other via communication bus 1008.

[0142] Communication interface 1004 is used to communicate with other electronic devices or servers.

[0143] The processor 1002 is used to execute program 1010, specifically the relevant steps in the above method embodiments.

[0144] Specifically, program 1010 may include program code that includes computer operation instructions.

[0145] The processor 1002 may be a central processing unit, a specific integrated circuit, or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0146] Memory 1006 is used to store program 1010. Memory 1006 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0147] Specifically, program 1010 can be used to cause processor 1002 to execute the steps in the above-described source graph compression method embodiment.

[0148] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0149] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described source graph compression method.

[0150] It should be noted that the computer-readable storage medium shown in this application can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, radio frequency, etc., or any suitable combination thereof.

[0151] In this embodiment of the application, a computer program product is also provided, which, when run on a computer, causes the computer to execute the above-described traceability graph compression method.

[0152] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for compressing a source map, characterized in that, include: Collect audit log data during application runtime; A source graph to be compressed is constructed based on the audit log data; wherein, the nodes in the source graph to be compressed represent entities in the audit log data, and the directed edges in the source graph to be compressed represent the relationships between the entities; Based on multiple template subgraphs in a pre-built template library, a portion isomorphic to the template subgraph is matched from the source graph to be compressed as the region to be compressed; wherein, the template subgraph is used to describe the normal behavior pattern of the application; The region to be compressed is subjected to lossy compression to obtain the compressed source graph; lossy compression refers to minimizing the number of nodes and edges as much as possible while ensuring that the semantics are not lost. The method for constructing the template library includes: Run test cases on the application and collect audit log data during the execution of each test case; A source graph is constructed based on the audit log data corresponding to each test case, resulting in a source graph set for each test case. The multiple source graphs in the source graph set are obtained by executing the test case multiple times. Each node in the source graph set represents an entity in the audit log data, and the directed edges in the source graph represent the relationships between the entities. For each test case, a subgraph mining algorithm is used to mine frequent items from multiple source graphs in the source graph set to obtain the template subgraph corresponding to the source graph set. The set of template subgraphs corresponding to multiple source tracing graph sets is used as a template library; The step of performing lossy compression on the region to be compressed to obtain the compressed traceability map includes: Assume that the source graph to be compressed is isomorphic to N template subgraphs, and the N template subgraphs correspond to M applications, where N and M are both positive integers, and N is greater than or equal to M; For a single template subgraph, the application node corresponding to the template subgraph is taken as the first node, and a new second node is generated according to the template information of the template subgraph. The second node represents all nodes of the region to be compressed corresponding to the template subgraph. The first node and the second node are connected by an edge, wherein the edge between the first node and the second node is generated according to the template information; Delete the original nodes and edges within the region to be compressed corresponding to the template subgraph, and reconnect the nodes outside the region to be compressed that are connected to the original nodes within the region to be compressed to the first node and the second node. Specifically, if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is a predecessor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is a predecessor node of the first node; if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is a successor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is a successor node of the second node.

2. The method according to claim 1, characterized in that, The subgraph mining algorithm is used to perform frequent item mining on multiple source graphs in the source graph set to obtain the template subgraph corresponding to the test case, including: Using the gSpan algorithm, frequent item mining is performed on multiple source graphs in the source graph set to obtain the template subgraph corresponding to the test case.

3. The method according to claim 1, characterized in that, The step of matching the portion isomorphic to the template subgraph from the source graph to be compressed as the region to be compressed includes: Using the VF2 algorithm, the regions that are isomorphic to the template subgraph are matched from the source graph to be compressed as regions to be compressed.

4. The method according to claim 1, characterized in that, The step of constructing the source map to be compressed based on the audit log data includes: Entities are extracted from the audit log data, and each extracted entity is used as a node; The relationships between entities are parsed from the audit log data, and the relationships between entities are represented by directed edges; Add attribute information to each node and each directed edge based on the audit log data.

5. The method according to claim 1, characterized in that, The audit log data collected during the operation of the application includes: Start the data collector and define the filtering rules, then start the container containing the test cases; Execute the test cases, and after waiting for a preset time period, stop and destroy the container; Shut down the data collector and save the output audit log data.

6. A traceability map compression device, characterized in that, The device includes: The audit log data collection module is used to collect audit log data during the application's operation. The source graph construction module is used to construct a source graph to be compressed based on the audit log data; wherein, the nodes in the source graph to be compressed represent entities in the audit log data, and the directed edges in the source graph to be compressed represent the relationships between the entities; The region to be compressed module is used to match the portion isomorphic to the template subgraph from the source graph to be compressed as the region to be compressed, based on multiple template subgraphs in a pre-built template library; wherein the template subgraph is used to describe the normal behavior pattern of the application. The lossy compression module is used to perform lossy compression on the region to be compressed to obtain the compressed source graph; lossy compression refers to minimizing the number of nodes and edges as much as possible while ensuring that the semantics are not lost. The audit log data collection module is also used to run test cases on the application and collect audit log data during the execution of each test case. The source graph construction module is further configured to construct a source graph based on the audit log data corresponding to each test case, thereby obtaining a source graph set corresponding to each test case; wherein, the multiple source graphs in the source graph set are obtained by executing the test case multiple times; the nodes in each source graph in the source graph set represent entities in the audit log data, and the directed edges in the source graph represent the association relationships between the entities; The source map compression device further includes: The template subgraph mining module is used to perform frequent item mining on multiple traceability graphs in the traceability graph set corresponding to each test case using a subgraph mining algorithm to obtain the template subgraph corresponding to the traceability graph set. The template library building module is used to collect the set of template subgraphs corresponding to multiple source map sets as a template library; Specifically, the lossy compression module is configured to assume that the source graph to be compressed is isomorphic to N template subgraphs, where the N template subgraphs correspond to M applications, and N and M are both positive integers, with N greater than or equal to M; for a single template subgraph, the application node corresponding to the template subgraph is used as a first node, and a new second node is generated based on the template information of the template subgraph, where the second node represents all nodes in the region to be compressed corresponding to the template subgraph; and, The first node and the second node are connected by an edge, wherein the edge between the first node and the second node is generated based on the template information; and... Delete the original nodes and edges within the region to be compressed corresponding to the template subgraph, and reconnect the nodes outside the region to be compressed that are connected to the original nodes within the region to be compressed to the first node and the second node. Specifically, if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is a predecessor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is a predecessor node of the first node; if a node outside the region to be compressed that is connected to an existing node within the region to be compressed is a successor node of an existing node within the region to be compressed, then the node outside the region to be compressed that is connected to an existing node within the region to be compressed is a successor node of the second node.

7. An electronic device, characterized in that, include: A processor for executing a computer program stored in a memory, wherein the computer program, when executed by the processor, implements the method of any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-5.

Images