A network attack investigation method and system based on enhanced traceability graph retrieval

Abstract

The present invention provides a network attack investigation method and system based on enhanced traceability graph retrieval, comprising: collecting logs from multiple platforms and processing them into a unified format; constructing the processed logs into a traceability graph using a traceability graph construction algorithm; identifying and detecting activity communities in the traceability graph to reveal the associations between sensitive activities; generating a multi-level summary of system activities based on the traceability graph, covering community summaries, entity summaries, and relationship summary information; improving the accuracy of query responses and the efficiency of information extraction through GraphRAG technology, and providing attack investigation reports through an interactive query interface. The present invention can process logs collected by multiple platforms and different log collectors, more efficiently capture the full picture of complex attacks, provide security analysts with comprehensive and rapid attack investigation results, and provide support for network security forensic analysis.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a network attack investigation method and system based on provenance graph retrieval enhancement. Background Art

[0002] In today's highly interconnected cyberworld, cyberattacks are increasing in frequency and sophistication, particularly Advanced Persistent Threat (APT) attacks, which pose significant challenges and security threats to the entire cyberspace. To combat these cyberthreats, enterprises and organizations widely utilize audit logs to record system activity, attempting to conduct effective attack investigations and trace the source after an attack occurs. This allows for timely response measures, prevents recurrence of similar attacks, and mitigates losses. Large enterprise networks may generate billions of log records daily, of which only a tiny fraction are related to actual attack activity. Sifting through this vast amount of data for valuable information is like searching for a needle in a haystack. Efficiently extracting useful information from this massive volume of log data presents a significant challenge. While rule-based approaches can quickly identify individual anomalous log entries, they lack comprehensiveness and are unable to contextualize the full attack landscape. Furthermore, they suffer from high false positive rates. They often fall short in addressing complex attack patterns and fail to provide comprehensive and accurate investigation results. In recent years, provenance graph-based methods have been proven to effectively mitigate these issues, but they still cannot describe attacks in natural language. Large Language Model (LLM) technology can address this issue. Therefore, combining provenance graphs with LLMs is a promising solution.

[0003] Since a certain country proposed the Transparent Computing Project in 2016, detection methods based on provenance graphs have shown great potential in attack investigation. This approach, based on system audit logs, uses causal analysis to model the relationships between different entities, converting system logs into a graph representation. Data provenance technology can capture the temporal and causal relationships between different entities (such as processes and files). This approach effectively leverages the rich contextual information in logs, enabling the correlation of combined attacks.

[0004] LLM can understand and generate natural language text, offering powerful information extraction and analysis capabilities. By combining LLM technology, security analysts can more efficiently extract useful information from massive log volumes, improving the speed and accuracy of attack investigations. However, naive LLM-based attack investigation methods typically directly input log text into the LLM and generate results. This approach struggles to effectively identify and correlate different behaviors within complex attacks, and therefore fails to provide a complete picture of the attack. Retrieval-Augmented Generation (RAG) combines retrieval and generation capabilities, retrieving relevant information from a large number of documents and generating answers using a generative model. GraphRAG builds on RAG by incorporating the structure of a knowledge graph, enabling it to retrieve and answer questions within the knowledge graph. RAG uses techniques such as community detection and hierarchical summarization to segment attack activities and further aggregate them into attack chains. Finally, combined with LLM, it processes query statements and generates natural language answers, helping security analysts fully understand the attack process and improving analysis efficiency. Summary of the Invention

[0005] The present invention provides a network attack investigation method and system based on enhanced traceability graph retrieval, which is used to address the defects in network security in the existing technology and has the advantages of high result accuracy, high investigation efficiency, and user friendliness.

[0006] In a first aspect, the present invention provides a network attack investigation method based on enhanced traceability graph retrieval, comprising:

[0007] Collect system activity data and generate audit logs that fully cover host and network behavior;

[0008] Extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and entity interaction relationships;

[0009] A community detection algorithm is used to divide the activities in the traceability graph into different communities to obtain independent community subgraphs;

[0010] Generate a community hierarchical summary for each community, and aggregate all community hierarchical summaries to form an attack chain analysis;

[0011] Determine the attack investigation question, and based on the attack investigation question, use a graph retrieval enhancement method to extract corresponding relevant information from the traceability graph and community hierarchical summary, process the relevant information based on a large-scale language model, and generate network attack investigation results.

[0012] According to the present invention, a network attack investigation method based on provenance graph retrieval enhancement is provided, which collects system activity data and generates an audit log that comprehensively covers host and network behavior, including:

[0013] Multiple log collectors are used to collect audit logs in different operating systems and scenarios. Sysmon and ETW tools are used on Windows systems to monitor system kernel events, application events, hardware performance, and network activity, comprehensively tracking and recording system activity. On Linux systems, Sysmon and Auditd are used to configure audit rules to record and save logs of key system events, including file access, process startup, user authentication, and system calls.

[0014] The Cuckoo Auxiliary component is used to implement the Sysmon log collection function. After the client completes sample analysis, the Sysmon log records are automatically returned. After each analysis, the sandbox and Sysmon configuration snapshots are restored to automatically complete the sample log analysis.

[0015] According to the present invention, a network attack investigation method based on provenance graph retrieval enhancement is provided, which extracts key entities and entity interaction relationships in the system from the audit log, and constructs a provenance graph based on the key entities and the entity interaction relationships, including:

[0016] Preprocessing the audit log to obtain a preprocessed log record;

[0017] Extracting the key entities and the entity interaction relationships from the pre-processed log records, and saving the key entities and the entity interaction relationships to a database;

[0018] The extracted key entities and the entity interaction relationships are constructed into the traceability graph.

[0019] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, the audit log is preprocessed to obtain a preprocessed log record, including:

[0020] Perform denoising and deduplication processing on the original audit logs to remove irrelevant or low-quality noise log records, as well as completely duplicate log records;

[0021] Unify and align the formats of timestamps from different sources of logs in multiple systems or devices, and arrange log events in chronological order.

[0022] Correspondingly, extracting the key entities and the entity interaction relationships from the pre-processed log records, and saving the key entities and the entity interaction relationships to a database, includes:

[0023] Traversing the pre-processed log records, extracting three types of key entities, namely, process, file, and network connection socket, one by one, and extracting corresponding key entity attributes for different types of key entities, wherein the process number, process path, and command line parameters are extracted for the key entity of the process type, the file name and absolute path of the file are extracted for the key entity of the file type, and the network IP, port, and protocol are extracted for the key entity of the network connection type;

[0024] Extract process creation, file reading and writing, and network connections between different types of key entities;

[0025] Assign unique identifiers as graph nodes to different types of key entities. For process-type key entities, use process name + PID as unique identifier. For file-type key entities, use the absolute path of the file in the system as unique identifier. For network connection-type key entities, use<IP, Port, Protocol> The combination serves as a unique identifier;

[0026] Storing the extracted key entities, key entity attribute information, and the interactions of the entity interactions in the form of a database;

[0027] Correspondingly, the extracted key entities and the entity interaction relationships are constructed into the traceability graph, including:

[0028] Initialize the traceability graph, construct the traceability graph into a directed graph, and determine the directed graph structure including the graph node set and the graph directed edge set;

[0029] Obtain all system event information from the database and traverse all events;

[0030] Extract the source node, target node and related event details from each event, and construct each event as a directed edge from the source node to the target node;

[0031] The directed graph structure is updated, and the source node, the target node, and the corresponding directed edges are added to the directed graph structure.

[0032] According to the present invention, a network attack investigation method based on enhanced traceability graph retrieval is provided, which uses a community detection algorithm to divide activities in the traceability graph into different communities to obtain independent community subgraphs, including:

[0033] Perform node initialization and movement, determine that all nodes are separate communities, and move each node to the community where the adjacent node is located to maximize modularity;

[0034] After the initial community division is completed, the Leiden algorithm is used to move nodes within each community to achieve a higher modularity of nodes within the community;

[0035] After the community division is completed, each community is treated as a super node, the directed graph structure is simplified, and the above steps are repeated until the modularity no longer increases.

[0036] According to the present invention, a network attack investigation method based on provenance graph retrieval enhancement is provided. Node initialization and movement are performed, all nodes are determined to be separate communities, and each node is moved to the community where the adjacent node is located to maximize modularity, including:

[0037] At the beginning of the Leiden algorithm, an independent community ID is assigned to each node. The node determines whether to change its community based on its connection relationship with other nodes. The node moves to a new community to improve the modularity of the community structure.

[0038] Correspondingly, after completing the initial community division, the Leiden algorithm is used to move nodes within each community to achieve a higher modularity of nodes within the community, including:

[0039] After completing node initialization and movement, the community is divided and the connection density between each node and other community members is iteratively calculated to determine whether the node belongs to another community.

[0040] Nodes are moved based on the strength of their connections with other nodes, and a greedy movement process is used to determine the community with the maximum gain.

[0041] Correspondingly, after the community division is completed, each community is treated as a super node, the directed graph structure is simplified, and the above steps are repeated until the modularity no longer increases, including:

[0042] After completing the community division and node movement, each community is determined to be a new super node, the connections between the new super nodes are recalculated, and the communities are re-divided;

[0043] After the redivision of the community is completed, the nodes are moved again to optimize the local area to maximize the modularity of the merged community.

[0044] According to the present invention, a network attack investigation method based on provenance graph retrieval enhancement is provided, which generates a community hierarchical summary for each community and summarizes all community hierarchical summaries to form an attack chain analysis, including:

[0045] Generate entity summaries that describe each entity in the traceability graph, including processes, files, and network connections, and input entity attributes into a large-scale language model to obtain entity descriptions;

[0046] Generate a relationship summary that summarizes the main interaction behaviors between entities in the community, including process creation, file access, and network connection, to describe the behavior between two entities;

[0047] A community summary is generated to provide an overview of the entire community, and the entity summary and the relationship summary are combined to generate attack intentions, attack steps, and attack impact in the community.

[0048] According to the present invention, a network attack investigation method based on provenance graph retrieval enhancement generates an entity summary that describes each entity in the provenance graph, including processes, files, and network connections. Entity attributes are input into a large-scale language model to obtain entity descriptions, including:

[0049] Determine prompts for the process, the file, and the network connection respectively, input node information in a structured form into a large-scale language model to obtain entity descriptions, and enable the large-scale language model to return entity summaries of a specific structure as a specified output;

[0050] Correspondingly, a relationship summary is generated to summarize the main interaction behaviors between entities in the community, including process creation, file access, and network connection, to describe the behavior between two entities, including:

[0051] Based on the quality of the cue words input into the large-scale language model, the relationship information description reflects the specific behavior between the two entities;

[0052] Determine prompt to input relation information in a structured form into a large-scale language model to obtain relation description, and make the large-scale language model return a relation summary of a specific structure with a prescribed output;

[0053] Correspondingly, a community summary is generated to provide an overview of the entire community. By combining the entity summary and the relationship summary, the attack intent, attack steps, and attack impact in the community are generated, including:

[0054] Generate a community title and a community summary based on the entity summary and the relationship summary, and output a threat score and a score explanation;

[0055] Determine the prompt output community discovery and enable large-scale language models to return community summaries with specific structures as prescribed output.

[0056] According to the present invention, a network attack investigation method based on provenance graph retrieval enhancement is provided. The method determines an attack investigation question, extracts corresponding relevant information from a provenance graph and a community hierarchical summary based on the attack investigation question using a graph retrieval enhancement method, processes the relevant information based on a large-scale language model, and generates network attack investigation results, including:

[0057] Receiving prompt words for the attack investigation question, parsing the prompt words, and using natural language processing to determine key objectives and contextual requirements of the query;

[0058] Use graph retrieval enhancement generation technology to obtain query responses;

[0059] After the retrieval task is completed, the retrieved context is input into the large-scale language model, which combines the retrieval information to generate the final natural language answer.

[0060] According to the present invention, a network attack investigation method based on provenance graph retrieval enhancement is provided, which receives prompt words of the attack investigation question, parses the prompt words, and uses natural language processing to determine the key objectives and context requirements of the query, including:

[0061] According to the prompt word, using a word embedding model, the grammatical structure of the prompt word query sentence is parsed to identify key targets, wherein the key targets include attack nodes, attack paths, and attacker behaviors;

[0062] Correspondingly, we use graph retrieval enhancement generation technology to obtain query responses, including:

[0063] A global search implementation is used, using Map-Reduce technology to perform global question retrieval. In the mapping phase, a large-scale language model is used to divide community reports into multiple batches, generating intermediate responses with scores that represent the importance of opinions. In the reduction phase, the mapping results are merged to output the overall query response, which includes a global view of the attack event and potential attack paths.

[0064] It adopts a local retrieval implementation method, uses vector search or keyword search technology, searches and parses keywords, identifies a set of initial nodes or document sets that are similar to the user's query concept, traverses and sorts the initial search results, and displays the most relevant documents to generate a response.

[0065] In a second aspect, the present invention further provides a network attack investigation system based on enhanced traceability graph retrieval, comprising:

[0066] Log collection module, used to collect system activity data and generate audit logs that fully cover host and network behavior;

[0067] A traceability graph construction module is used to extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and the entity interaction relationships;

[0068] A community detection module is used to divide the activities in the traceability graph into different communities using a community detection algorithm to obtain independent community subgraphs;

[0069] Hierarchical summary module, used to generate community hierarchical summaries for each community and aggregate all community hierarchical summaries to form attack chain analysis;

[0070] The investigation interaction module is used to determine the attack investigation questions. Based on the attack investigation questions, the corresponding relevant information is extracted from the traceability graph and community hierarchical summary using a graph retrieval enhancement method, and the relevant information is processed based on a large-scale language model to generate network attack investigation results.

[0071] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the network attack investigation method enhanced by source graph retrieval as described above is implemented.

[0072] The network attack investigation method and system based on provenance graph retrieval enhancement provided by the present invention start from the network attack investigation, model the attack event through the provenance graph, discover the correlation between system events through the community discovery algorithm, and then generate a hierarchical system event summary to help security analysts quickly locate the attack source and understand the attack path. Combining retrieval enhancement generation technology with large-scale language models, while fully utilizing the generation capabilities of large-scale language models, it provides more accurate and effective contextual information for large-scale language models, thereby helping security personnel quickly obtain detailed and accurate attack investigation reports from large amounts of logs, and then quickly respond to attacks. Compared with existing methods, the present invention can significantly improve the accuracy of results and investigation efficiency in attack investigation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0074] Figure 1 This is a flow chart of a network attack investigation method based on enhanced traceability graph retrieval provided by the present invention;

[0075] Figure 2 This is a schematic diagram of the structure of the network attack investigation system based on provenance graph retrieval enhancement provided by the present invention;

[0076] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0077] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0078] To address the shortcomings of existing network security technologies, this paper provides a network attack investigation method based on enhanced provenance graph retrieval. This method can efficiently identify and analyze network attack behaviors, including but not limited to phishing attacks, malware propagation, unauthorized access, and other security threats. Compared with existing technologies, this method has the advantages of high result accuracy, high investigation efficiency, and user-friendliness.

[0079] Figure 1 FIG. 1 is a flow chart of a network attack investigation method based on enhanced traceability graph retrieval provided by an embodiment of the present invention. Figure 1 Shown, including:

[0080] Step 1: Collect system activity data and generate audit logs that fully cover host and network behavior;

[0081] Step 2: Extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and entity interaction relationships;

[0082] Step 3: Use a community detection algorithm to divide the activities in the traceability graph into different communities to obtain independent community subgraphs;

[0083] Step 4: Generate a community hierarchical summary for each community, and summarize all community hierarchical summaries to form an attack chain analysis;

[0084] Step 5: Determine the attack investigation question. Based on the attack investigation question, use the graph retrieval enhancement method to extract the corresponding relevant information from the traceability graph and community hierarchical summary. Process the relevant information based on the large-scale language model to generate the network attack investigation results.

[0085] In one embodiment, step 1 specifically includes the following steps:

[0086] Step 1.1: Use the corresponding log collector to collect audit logs based on different operating systems and usage scenarios.

[0087] In step 1.2, use the Cuckoo Auxiliary component to automate sample log analysis.

[0088] In one embodiment, the specific implementation of step 1.1 is as follows:

[0089] In Windows systems, use Sysmon and ETW tools to monitor system kernel events, application events, hardware performance, and network activity, comprehensively tracking and recording system activities.

[0090] In Linux, use Sysmon and Auditd to configure audit rules to record key system events, such as file access, process startup, user authentication, and system calls, and save the logs.

[0091] In one embodiment, the specific implementation of step 1.2 is as follows:

[0092] Use the Cuckoo Auxiliary component to customize the additional Sysmon log collection function module. The client automatically returns the complete Sysmon log record after completing sample analysis, and restores the sandbox and Sysmon configuration snapshots after each analysis, realizing the automation of sample log analysis.

[0093] In one embodiment, step 2 specifically includes the following steps:

[0094] Step 2.1, preprocessing the original audit log data obtained in step 1.1;

[0095] Step 2.2: Extract key entities and the relationships between these entities from the preprocessed logs and store them in a database.

[0096] In step 2.3, the extracted key entities and entity interaction relationships are constructed into a traceability graph.

[0097] In one embodiment, step 2.1 is specifically implemented as follows:

[0098] De-noise and deduplicate the raw log data, removing irrelevant or low-quality noise log records, such as operating system startup records. Compare key fields in the logs and remove completely duplicate log records to prevent noise and duplicate records from affecting the analysis results.

[0099] When processing logs from multiple systems or devices, format and align timestamps from different sources uniformly, and arrange log events in chronological order.

[0100] In one embodiment, the specific implementation of step 2.2 is as follows:

[0101] Traverse the preprocessed logs and extract three types of entities, namely processes, files, and network connection sockets, as key entities one by one. For each type of key entity, extract the corresponding key entity attributes. For process entities, extract the process number, process path, and command line parameters; for file entities, extract the file name and absolute path of the file; and for network connection entities, extract the network IP, port, and protocol.

[0102] Extract relationships between key entities such as process creation, file reading and writing, and network connections;

[0103] Assign unique identifiers to different entities as nodes in the graph. For process entities, use "process name + PID" as the unique identifier. For file entities, use the absolute path of the file in the system as the unique identifier. For network connection entities, use<IP, Port, Protocol> The combination serves as a unique identifier;

[0104] The extracted entities, entity attribute information and interactions between entities are stored in the form of a database.

[0105] In one embodiment, the specific implementation of step 2.3 is as follows:

[0106] Step 2.3.1: Initialize the provenance graph and construct it as a directed graph. Define the graph structure G = (V, E), where V is the set of nodes and E is the set of directed edges.

[0107] Step 2.3.2, obtain all system event information from the database saved in step 2.2, and traverse all events;

[0108] Step 2.3.3: For each event, extract the source node (src), target node (dst), and related event details, and construct the event as a directed edge from the source node (src) to the target node (dst);

[0109] Step 2.3.4, update the graph structure and add the source node, target node and corresponding directed edges to graph G.

[0110] In one embodiment, step 3 specifically includes:

[0111] Step 3.1, node initialization and movement, treat all nodes as separate communities, and then try to move each node to the community where the adjacent node is located to maximize modularity;

[0112] Step 3.2: After completing the preliminary community division, the Leiden algorithm moves nodes within each community to ensure that the nodes within the community have a higher modularity;

[0113] Step 3.3: After the community division phase, the algorithm treats each community as a supernode, simplifies the graph structure, and repeats the previous steps until the modularity no longer increases significantly.

[0114] In one embodiment, step 3.1 is specifically implemented as follows:

[0115] At the beginning of the Leiden algorithm, each node is assigned a unique community ID. Based on its connections with other nodes, the node determines whether to change its community. The node attempts to move itself to a new community, thereby increasing the modularity of the community structure.

[0116] In one embodiment, step 3.2 is specifically implemented as follows:

[0117] After node initialization and movement, the community is further divided. By calculating the connection density between each node and other community members, it is determined whether the node should belong to another community. This process can be repeated, and each node decides whether to change its community based on its modularity gain.

[0118] The node movement process is greedy, that is, each time the community with the greatest gain is selected. The main basis for node movement is the strength of its connection with other nodes. Nodes tend to join communities that can improve modularity.

[0119] In one embodiment, step 3.3 is specifically implemented as follows:

[0120] After completing the community division and node movement, each community is regarded as a new "super node", and the connections between these super nodes are recalculated to re-divide the community;

[0121] After the division is completed, the nodes are moved again to optimize the local area to ensure that the merged community can still maximize the modularity.

[0122] In one embodiment, step 4 specifically includes:

[0123] Step 4.1: Generate entity summaries, describing each entity in the traceability graph, including processes, files, and network connections. When generating entity summaries, the attributes of the entity are input into the LLM to obtain the corresponding description;

[0124] Step 4.2: Generate a relationship summary, which summarizes the main interaction behaviors between entities in the community, such as process creation, file access, network connection, etc. In this way, the behavior between two entities can be described in detail.

[0125] Step 4.3, community summary generation, provides an overview of the entire community, integrates the entity summary and relationship summary in the community, and comprehensively generates attack intentions, attack steps and their impact in the community.

[0126] In one embodiment, step 4.1 is specifically implemented as follows:

[0127] The construction of the traceability graph takes into account three entities, including processes, files, and network connections. Prompts are designed for each of the three entities to input node information in a structured form into the LLM to obtain a description of the entity, and the LLM is required to return an entity summary of a specific structure as a specified output.

[0128] In one embodiment, step 4.2 is specifically implemented as follows:

[0129] Design prompts for input to the LLM. The description of the relationship should be able to describe the specific behavior between the two entities. For example, if process A creates process B at time T, the command line is C, and the specific parameters indicate that process A is performing network detection by creating process B.

[0130] The design prompt inputs the relationship information into the LLM in a structured form to obtain a description of the relationship, and requires the LLM to return a relationship summary of a specific structure in a specified output.

[0131] In one embodiment, step 4.3 is specifically implemented as follows:

[0132] Generate a summary of the community obtained by community detection, integrate the entity summary and relationship summary in the community, generate a community title and community summary based on the entities and relationships in the community, and give a threat score and an explanation of the score. Finally, give the discovery of the community, such as what role a certain entity plays in the community. Design a prompt to require LLM to return a community summary of a specific structure as a specified output.

[0133] In one embodiment, step 5 specifically includes:

[0134] Step 5.1: Receive the query (i.e., prompt word) input by the security analyst and use natural language processing technology to determine the key objectives and context requirements of the query;

[0135] Step 5.2: Combine Graph Retrieval Enhancement Generation (GraphRAG) technology to improve the accuracy and completeness of query responses and perform global and local retrieval.

[0136] In step 5.3, after completing the retrieval task, the retrieved context is input into the collocation LLM, which combines the retrieved relevant information to generate the final natural language answer.

[0137] In one embodiment, step 5.1 is specifically implemented as follows:

[0138] By receiving the query (prompt word) input by the user, the word embedding model is used to analyze the grammatical structure of the query and identify the key targets. The targets may include attack nodes, attack paths, or attacker behaviors.

[0139] In one embodiment, step 5.2 is specifically implemented as follows:

[0140] Step 5.2.1, global search

[0141] We use Map-Reduce technology to perform global question retrieval. In the mapping phase, we use LLM to divide community reports into multiple batches to generate intermediate responses with scores. The scores are used to indicate the importance of this opinion.

[0142] In the reduction phase, the mapping results are merged to form a response to the overall query, and the output includes a global view of the attack event and potential attack paths;

[0143] Step 5.2.2, local search

[0144] Using vector search or keyword search technology, the parsed keywords are searched to identify a set of initial nodes or document sets that are conceptually similar to the user query. These initial search results are then traversed and sorted, and the most relevant documents are displayed to generate a response.

[0145] In one embodiment, step 5.3 is specifically implemented as follows:

[0146] The retrieved context information is input into the LLM, and the prompt is designed to generate the final natural language answer and feed it back to the security analyst.

[0147] The network attack investigation system based on enhanced source graph retrieval provided by the present invention is described below. The network attack investigation system based on enhanced source graph retrieval described below and the network attack investigation method based on enhanced source graph retrieval described above can be referenced to each other.

[0148] Figure 2 Schematic diagram of the structure of the network attack investigation system based on enhanced traceability graph retrieval provided by an embodiment of the present invention. Figure 2 Shown, including:

[0149] Log collection module, used to collect system activity data and generate audit logs that fully cover host and network behavior;

[0150] A traceability graph construction module is used to extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and the entity interaction relationships;

[0151] A community detection module is used to divide the activities in the traceability graph into different communities using a community detection algorithm to obtain independent community subgraphs;

[0152] Hierarchical summary module, used to generate community hierarchical summaries for each community and aggregate all community hierarchical summaries to form attack chain analysis;

[0153] The investigation interaction module is used to determine the attack investigation questions. Based on the attack investigation questions, the corresponding relevant information is extracted from the traceability graph and community hierarchical summary using a graph retrieval enhancement method, and the relevant information is processed based on a large-scale language model to generate network attack investigation results.

[0154] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a network attack investigation method based on traceability graph retrieval enhancement, the method comprising: collecting system activity data to generate an audit log that comprehensively covers host and network behavior; extracting key entities and entity interaction relationships in the system from the audit log, and constructing a traceability graph based on the key entities and the entity interaction relationships; using a community detection algorithm to divide the activities in the traceability graph into different communities to obtain independent community subgraphs; generating a community hierarchical summary for each community, and aggregating all community hierarchical summaries to form an attack chain analysis; determining an attack investigation question, and based on the attack investigation question, extracting corresponding relevant information from the traceability graph and the community hierarchical summary using a graph retrieval enhancement method, processing the relevant information based on a large-scale language model, and generating network attack investigation results.

[0155] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0157] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A network attack investigation method based on provenance graph retrieval enhancement, characterized in that: include: Collect system activity data and generate audit logs that fully cover host and network behavior; Extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and entity interaction relationships; A community detection algorithm is used to divide the activities in the traceability graph into different communities to obtain independent community subgraphs; Generate a community hierarchical summary for each community, and aggregate all community hierarchical summaries to form an attack chain analysis; Determine the attack investigation question, extract relevant information from the provenance graph and community hierarchical summary using a graph retrieval enhancement method based on the attack investigation question, process the relevant information based on a large-scale language model, and generate network attack investigation results, including: Receiving prompt words for the attack investigation question, parsing the prompt words, and using natural language processing to determine key objectives and contextual requirements of the query; Use graph retrieval enhancement generation technology to obtain query responses; After the retrieval task is completed, the retrieved context is input into the large-scale language model, which combines the retrieval information to generate the final natural language answer; Receiving prompt words for the attack investigation question, parsing the prompt words, and using natural language processing to determine key objectives and contextual requirements of the query, including: According to the prompt word, using a word embedding model, the grammatical structure of the prompt word query sentence is parsed to identify key targets, wherein the key targets include attack nodes, attack paths, and attacker behaviors; Correspondingly, we use graph retrieval enhancement generation technology to obtain query responses, including: A global search implementation is used, using Map-Reduce technology to perform global question retrieval. In the mapping phase, a large-scale language model is used to divide community reports into multiple batches, generating intermediate responses with scores that represent the importance of opinions. In the reduction phase, the mapping results are merged to output the overall query response, which includes a global view of the attack event and potential attack paths. It adopts a local retrieval implementation method, uses vector search or keyword search technology, searches and parses keywords, identifies a set of initial nodes or document sets that are similar to the user's query concept, traverses and sorts the initial search results, and displays the most relevant documents to generate a response.

2. The network attack investigation method based on enhanced traceability graph retrieval according to claim 1 is characterized in that: Collects system activity data and generates comprehensive audit logs covering host and network behavior, including: Multiple log collectors are used to collect audit logs in different operating systems and scenarios. Sysmon and ETW tools are used on Windows systems to monitor system kernel events, application events, hardware performance, and network activity, comprehensively tracking and recording system activity. On Linux systems, Sysmon and Auditd are used to configure audit rules to record and save logs of key system events, including file access, process startup, user authentication, and system calls. The Cuckoo Auxiliary component is used to implement the Sysmon log collection function. After the client completes sample analysis, the Sysmon log records are automatically returned. After each analysis, the sandbox and Sysmon configuration snapshots are restored to automatically complete the sample log analysis.

3. The network attack investigation method based on enhanced traceability graph retrieval according to claim 1 is characterized in that: Extracting key entities and entity interaction relationships in the system from the audit log, and constructing a traceability graph based on the key entities and entity interaction relationships, including: Preprocessing the audit log to obtain a preprocessed log record; Extracting the key entities and the entity interaction relationships from the pre-processed log records, and saving the key entities and the entity interaction relationships to a database; The extracted key entities and the entity interaction relationships are constructed into the traceability graph.

4. The network attack investigation method based on enhanced traceability graph retrieval according to claim 3 is characterized in that: The audit log is preprocessed to obtain a preprocessed log record, including: Perform denoising and deduplication processing on the original audit logs to remove irrelevant or low-quality noise log records, as well as completely duplicate log records; Unify and align the formats of timestamps from different sources of logs in multiple systems or devices, and arrange log events in chronological order. Correspondingly, extracting the key entities and the entity interaction relationships from the pre-processed log records, and saving the key entities and the entity interaction relationships to a database, includes: Traversing the pre-processed log records, extracting three types of key entities, namely, process, file, and network connection socket, one by one, and extracting corresponding key entity attributes for different types of key entities, wherein the process number, process path, and command line parameters are extracted for the key entity of the process type, the file name and absolute path of the file are extracted for the key entity of the file type, and the network IP, port, and protocol are extracted for the key entity of the network connection type; Extract process creation, file reading and writing, and network connections between different types of key entities; Assign unique identifiers as graph nodes to different types of key entities. For process-type key entities, use process name + PID as unique identifier. For file-type key entities, use the absolute path of the file in the system as unique identifier. For network connection-type key entities, use<IP, Port, Protocol> The combination serves as a unique identifier; Storing the extracted key entities, key entity attribute information, and the interactions of the entity interactions in the form of a database; Correspondingly, the extracted key entities and the entity interaction relationships are constructed into the traceability graph, including: Initialize the traceability graph, construct the traceability graph into a directed graph, and determine the directed graph structure including the graph node set and the graph directed edge set; Obtain all system event information from the database and traverse all events; Extract the source node, target node and related event details from each event, and construct each event as a directed edge from the source node to the target node; The directed graph structure is updated, and the source node, the target node, and the corresponding directed edges are added to the directed graph structure.

5. The network attack investigation method based on enhanced traceability graph retrieval according to claim 1 is characterized in that: A community detection algorithm is used to divide the activities in the traceability graph into different communities to obtain independent community subgraphs, including: Perform node initialization and movement, determine that all nodes are separate communities, and move each node to the community where the adjacent node is located to maximize modularity; After the initial community division is completed, the Leiden algorithm is used to move nodes within each community to achieve a higher modularity of nodes within the community; After the community division is completed, each community is treated as a super node, the directed graph structure is simplified, and the above steps are repeated until the modularity no longer increases.

6. The network attack investigation method based on enhanced traceability graph retrieval according to claim 5 is characterized in that: Initialize and move nodes, determine that all nodes are separate communities, and move each node to the community where the adjacent node is located to maximize modularity, including: At the beginning of the Leiden algorithm, an independent community ID is assigned to each node. The node determines whether to change its community based on its connection relationship with other nodes. The node moves to a new community to improve the modularity of the community structure. Correspondingly, after completing the initial community division, the Leiden algorithm is used to move nodes within each community to achieve a higher modularity of nodes within the community, including: After completing node initialization and movement, the community is divided and the connection density between each node and other community members is iteratively calculated to determine whether the node belongs to another community. Nodes are moved based on the strength of their connections with other nodes, and a greedy movement process is used to determine the community with the maximum gain. Correspondingly, after the community division is completed, each community is treated as a super node, the directed graph structure is simplified, and the above steps are repeated until the modularity no longer increases, including: After completing the community division and node movement, each community is determined to be a new super node, the connections between the new super nodes are recalculated, and the communities are re-divided; After the redivision of the community is completed, the nodes are moved again to optimize the local area to maximize the modularity of the merged community.

7. The network attack investigation method based on enhanced traceability graph retrieval according to claim 1 is characterized in that: Generate a community hierarchical summary for each community, and aggregate all community hierarchical summaries to form an attack chain analysis, including: Generate entity summaries that describe each entity in the traceability graph, including processes, files, and network connections, and input entity attributes into a large-scale language model to obtain entity descriptions; Generate a relationship summary that summarizes the main interaction behaviors between entities in the community, including process creation, file access, and network connection, to describe the behavior between two entities; A community summary is generated to provide an overview of the entire community, and the entity summary and the relationship summary are combined to generate attack intentions, attack steps, and attack impact in the community.

8. The network attack investigation method based on enhanced traceability graph retrieval according to claim 7 is characterized in that: Generate entity summaries that describe each entity in the traceability graph, including processes, files, and network connections. Input entity attributes into a large-scale language model to obtain entity descriptions, including: Determine prompts for the process, the file, and the network connection respectively, input node information in a structured form into a large-scale language model to obtain entity descriptions, and enable the large-scale language model to return entity summaries of a specific structure as a specified output; Correspondingly, a relationship summary is generated to summarize the main interaction behaviors between entities in the community, including process creation, file access, and network connection, to describe the behavior between two entities, including: Based on the quality of the cue words input into the large-scale language model, the relationship information description reflects the specific behavior between the two entities; Determine prompt to input relation information in a structured form into a large-scale language model to obtain relation description, and make the large-scale language model return a relation summary of a specific structure with a prescribed output; Correspondingly, a community summary is generated to provide an overview of the entire community. By combining the entity summary and the relationship summary, the attack intent, attack steps, and attack impact in the community are generated, including: Generate a community title and a community summary based on the entity summary and the relationship summary, and output a threat score and a score explanation; Determine the prompt output community discovery and enable large-scale language models to return community summaries with specific structures as prescribed output.

Images