Network attack investigation method and system based on traceability graph retrieval enhancement

Abstract

The invention provides a network attack investigation method and system based on traceability graph retrieval enhancement, and the method comprises the steps: collecting multi-platform logs, and processing the multi-platform logs into a unified format; constructing the processed logs into a traceability graph through a traceability graph construction algorithm; identifying and detecting an activity community in the traceability graph, and revealing an association relationship between sensitive activities; in combination with the traceability graph, a multi-level abstract of system activities is generated, and community abstract information, entity abstract information and relation abstract information are covered; the accuracy of query response and the efficiency of information extraction are improved through a GraphRAG technology, and an attack survey report is given through an interactive query interface. According to the method, logs collected by multiple platforms and different log collectors can be processed, the full view of complex attacks can be captured more efficiently, a comprehensive and rapid attack survey result is provided for security analysts, and support is provided for network security evidence obtaining and analysis.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a network attack investigation method and system based on provenance graph retrieval enhancement. Background Art

[0002] In today's highly interconnected network world, the frequency and complexity of cyber attacks are increasing, especially Advanced Persistent Threat (APT) attacks, which have brought severe challenges and security threats to the entire cyberspace. In order to deal with these cyber threats, enterprises and organizations widely use audit logs to record system activities, trying to conduct effective attack investigations and trace the source after the attack occurs, so as to take corresponding emergency response measures in time to avoid the recurrence of similar attacks and reduce losses. In large enterprise networks, billions of log records may be generated every day, of which only a few are related to actual attack activities. It is like looking for a needle in a haystack to filter out valuable information from such a huge amount of data. Faced with a large amount of log data, how to efficiently extract useful information from it has become a major challenge. Although rule-based methods can quickly locate abnormal individual log entries, they cannot associate the full picture of the attack, lack integrity, and have a high false alarm rate. When dealing with complex attack patterns, they often perform insufficiently and cannot provide comprehensive and accurate investigation results. In recent years, provenance graph-based methods have been proven to effectively alleviate the above problems, but they still cannot provide attack descriptions in natural language. Large Language Model (LLM) technology can make up for this problem. Therefore, combining provenance graphs and LLM is a potential solution.

[0003] Since a country proposed the transparent computing project in 2016, the detection method based on traceability graph has shown great potential in the field of attack investigation. This method is based on system audit logs, and through causal association analysis, it models different entity relationships and converts system logs into graph representations. Data traceability technology can capture the time and causal relationships between different entities (such as processes and files). This method effectively utilizes the rich contextual information in the logs, thereby having the ability to associate combined attacks.

[0004] LLM can understand and generate natural language text and has powerful information extraction and analysis capabilities. By combining LLM technology, security analysts can more efficiently extract useful information from massive logs and improve the speed and accuracy of attack investigations. However, the simple LLM-based attack investigation method usually directly inputs the text of the log into LLM and generates results. This method is difficult to effectively identify and associate different behaviors in complex attacks, and therefore cannot provide a complete picture of the attack. Retrieval-Augmented Generation (RAG) technology is a technology that combines retrieval and generation capabilities, retrieves relevant information from a large number of documents, and generates answers in combination with a generation model. GraphRAG integrates the structure of the knowledge graph on the basis of RAG, enabling it to obtain the ability to retrieve and answer knowledge graphs. RAG technology divides attack activities through community detection, hierarchical summarization and other technologies and further summarizes them into attack chains. Finally, combined with LLM to process query statements and generate natural language answers, it can help security analysts fully understand the attack process and improve analysis efficiency. Summary of the invention

[0005] The present invention provides a network attack investigation method and system based on source traceability graph retrieval enhancement, which are used to solve the defects of network security in the prior art and have the advantages of high result accuracy, high investigation efficiency, and user friendliness.

[0006] In a first aspect, the present invention provides a network attack investigation method based on provenance graph retrieval enhancement, comprising: Collect system activity data and generate audit logs that fully cover host and network behavior; Extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and the entity interaction relationships; A community detection algorithm is used to divide the activities in the traceability graph into different communities to obtain independent community subgraphs; Generate a community hierarchical summary for each community, and aggregate all community hierarchical summaries to form an attack chain analysis; Determine the attack investigation question, and based on the attack investigation question, use a graph retrieval enhancement method to extract corresponding relevant information from the traceability graph and the community hierarchical summary, process the relevant information based on a large-scale language model, and generate a network attack investigation result.

[0007] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, system activity data is collected to generate an audit log that fully covers host and network behaviors, including: A variety of log collectors are used to collect the audit logs in different operating systems and scenarios. In Windows, Sysmon and ETW tools are used to monitor system kernel events, application events, hardware performance, and network activities, and to comprehensively track and record system activities. In Linux, Sysmon and Auditd tools are used to configure audit rules to record key system events and save logs. The key system events include file access, process startup, user authentication, and system calls. Use Cuckoo Auxiliary component to implement Sysmon log collection function. After the client completes sample analysis, it automatically returns Sysmon log records. After each analysis, it restores the sandbox and Sysmon configuration snapshots to automatically complete sample log analysis.

[0008] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, key entities and entity interaction relationships in the system are extracted from the audit log, and a provenance graph is constructed from the key entities and the entity interaction relationships, including: Preprocessing the audit log to obtain a preprocessed log record; Extracting the key entities and the entity interaction relationships from the preprocessed log records, and saving the key entities and the entity interaction relationships to a database; The extracted key entities and the entity interaction relationships are constructed into the traceability graph.

[0009] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, the audit log is preprocessed to obtain a preprocessed log record, including: Perform denoising and deduplication processing on the original audit logs to remove irrelevant or low-quality noise log records and completely duplicate log records; Unify and align the formats of timestamps from different sources of logs in multiple systems or devices, and arrange log events in chronological order; Correspondingly, extracting the key entity and the entity interaction relationship from the preprocessed log record, and saving the key entity and the entity interaction relationship to a database, includes: Traversing the preprocessed log records, extracting three types of key entities, namely, process, file and network connection socket, one by one, and extracting corresponding key entity attributes for different types of key entities, wherein the process number, process path and command line parameters are extracted for the key entity of the process type, the file name and absolute path of the file are extracted for the key entity of the file type, and the network IP, port and protocol are extracted for the key entity of the network connection type; Extract process creation, file reading and writing, and network connections between different types of key entities; Assign unique identifiers as graph nodes to different types of key entities. For process-type key entities, use process name + PID as the unique identifier. For file-type key entities, use the absolute path of the file in the system as the unique identifier. For network connection-type key entities, use<IP, Port, Protocol> The combination serves as a unique identifier; The extracted key entities, key entity attribute information and the interactions of the entity interaction relationships are stored in the form of a database; Correspondingly, the extracted key entities and the entity interaction relationships are constructed into the traceability graph, including: Initialize the traceability graph, construct the traceability graph as a directed graph, and determine the directed graph structure including a graph node set and a graph directed edge set; Obtain all system event information from the database and traverse all events; Extract the source node, target node and related event details from each event, and construct each event as a directed edge from the source node to the target node; The directed graph structure is updated, and the source node, the target node, and the corresponding directed edges are added to the directed graph structure.

[0010] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, a community detection algorithm is used to divide the activities in the provenance graph into different communities to obtain independent community subgraphs, including: Perform node initialization and movement, determine that all nodes are separate communities, and move each node to the community where the adjacent node is located to maximize modularity; After the initial community division is completed, the Leiden algorithm is used to move nodes within each community to make the nodes within the community obtain a higher modularity; After the community division is completed, each community is regarded as a super node, the directed graph structure is simplified, and the above steps are repeated until the modularity no longer increases.

[0011] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, node initialization and movement are performed, all nodes are determined to be separate communities, and each node is moved to the community where the adjacent node is located to maximize the modularity, including: At the beginning of the Leiden algorithm, an independent community ID is assigned to each node. The node determines whether to change its community based on its connection relationship with other nodes. The node moves to a new community to improve the modularity of the community structure. Correspondingly, after the initial community division is completed, the Leiden algorithm is used to move nodes within each community so that the nodes within the community have a higher modularity, including: After completing node initialization and movement, the community is divided, and the connection density between each node and other members of the community is iteratively calculated to determine whether the node belongs to another community; Nodes are moved based on the connection strength between the nodes and other nodes, and a greedy movement process is used to determine the community with the maximum gain. Correspondingly, after the community division is completed, each community is regarded as a super node, the directed graph structure is simplified, and the above steps are repeated until the modularity is no longer improved, including: After completing the community division and node movement, each community is determined to be a new super node, the connections between the new super nodes are recalculated, and the communities are re-divided; After the redivision of the community is completed, the nodes are moved again to optimize the local area to maximize the modularity of the merged community.

[0012] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, a community hierarchical summary is generated for each community, and all community hierarchical summaries are summarized to form an attack chain analysis, including: Generate an entity summary to describe each entity in the traceability graph, including processes, files, and network connections, and input entity attributes into a large-scale language model to obtain an entity description; Generate a relationship summary that summarizes the main interaction behaviors between entities in the community, including process creation, file access, and network connection to describe the behavior between two entities; A community summary is generated to provide an overview of the entire community, and the entity summary and the relationship summary are combined to generate attack intentions, attack steps, and attack impacts in the community.

[0013] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, an entity summary is generated to describe each entity in the provenance graph, including processes, files and network connections, and entity attributes are input into a large-scale language model to obtain an entity description, including: Determine prompts for the process, the file, and the network connection respectively, input node information into a large-scale language model in a structured form to obtain entity descriptions, and enable the large-scale language model to return entity summaries of a specific structure with a specified output; Correspondingly, a relationship summary is generated to summarize the main interaction behaviors between entities in the community, including process creation, file access, and network connection, to describe the behavior between two entities, including: According to the quality of the prompt words input into the large-scale language model, the relationship information description reflects the specific behavior between the two entities; Determine prompt to input relation information into a large-scale language model in a structured form to obtain relation description, and make the large-scale language model return a relation summary of a specific structure with a prescribed output; Correspondingly, a community summary is generated to provide an overview of the entire community, and the entity summary and the relationship summary are combined to generate the attack intention, attack steps and attack impact in the community, including: Generate a community title and a community summary according to the entity summary and the relationship summary, and output a threat score and a score explanation; Determine the prompt output community discovery and enable large-scale language models to return community summaries of specific structures as prescribed outputs.

[0014] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, an attack investigation question is determined, and according to the attack investigation question, a graph retrieval enhancement method is used to extract corresponding relevant information from a provenance graph and a community hierarchical summary, and the relevant information is processed based on a large-scale language model to generate a network attack investigation result, including: Receiving prompt words of the attack investigation question, parsing the prompt words, and using natural language processing to determine key objectives and context requirements of the query; Use graph retrieval enhancement generation technology to obtain query responses; After completing the retrieval task, the retrieved context is input into the large-scale language model, which combines the retrieval information to generate the final natural language answer.

[0015] According to a network attack investigation method based on provenance graph retrieval enhancement provided by the present invention, a prompt word of the attack investigation question is received, the prompt word is parsed, and the key target and context requirements of the query are determined by natural language processing, including: According to the prompt word, using a word embedding model, the grammatical structure of the prompt word query sentence is parsed to identify key targets, wherein the key targets include attack nodes, attack paths, and attacker behaviors; Correspondingly, the graph retrieval enhancement generation technology is used to obtain query responses, including: A global search implementation is adopted, and Map-Reduce technology is used to perform global question search. In the mapping stage, a large-scale language model is used to divide community reports into multiple batches, and intermediate responses with scores are generated. The scores represent the importance of opinions. In the reduction stage, the mapping results are merged and the overall query response is output. The overall query response includes a global view of the attack event and potential attack paths; It adopts the local retrieval implementation method, uses vector search or keyword search technology, searches and parses keywords, identifies a set of initial nodes or document sets that are similar to the user's query concept, traverses and sorts the initial search results, and displays the most relevant documents to generate a response.

[0016] In a second aspect, the present invention further provides a network attack investigation system based on provenance graph retrieval enhancement, comprising: Log collection module, used to collect system activity data and generate audit logs that fully cover host and network behavior; A traceability graph construction module is used to extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and the entity interaction relationships; A community detection module, used to divide the activities in the traceability graph into different communities using a community detection algorithm to obtain independent community subgraphs; Hierarchical summary module, used to generate community hierarchical summaries for each community and aggregate all community hierarchical summaries to form attack chain analysis; The investigation interaction module is used to determine the attack investigation questions, and according to the attack investigation questions, use the graph retrieval enhancement method to extract the corresponding relevant information from the traceability graph and the community hierarchical summary, and process the relevant information based on the large-scale language model to generate the network attack investigation results.

[0017] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the network attack investigation method based on enhanced source graph retrieval as described above is implemented.

[0018] The network attack investigation method and system based on provenance graph retrieval enhancement provided by the present invention start from network attack investigation, model attack events through provenance graph, discover the correlation between system events through community discovery algorithm, and then generate hierarchical system event summaries to help security analysts quickly locate attack sources and understand attack paths. Combined with retrieval enhancement generation technology and large-scale language models, while fully utilizing the generation capabilities of large-scale language models, more accurate and effective context information is provided for large-scale language models, thereby helping security personnel quickly obtain detailed and accurate attack investigation reports from a large number of logs, and then quickly respond to attacks. Compared with existing methods, the present invention can significantly improve the accuracy of results and investigation efficiency in attack investigation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a network attack investigation method based on enhanced traceability graph retrieval provided by the present invention; Figure 2 It is a structural schematic diagram of a network attack investigation system based on source traceability graph retrieval enhancement provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In view of the deficiencies in network security in the prior art, the present invention provides a network attack investigation method based on enhanced traceability graph retrieval, which can efficiently identify and analyze network attack behaviors, including but not limited to phishing attacks, malware propagation, unauthorized access and other security threats. Compared with the prior art, it has the advantages of high result accuracy, high investigation efficiency, and user-friendliness.

[0023] Figure 1 is a flow chart of a network attack investigation method based on enhanced traceability graph retrieval provided by an embodiment of the present invention, such as Figure 1 As shown, including: Step 1: Collect system activity data and generate audit logs that fully cover host and network behavior; Step 2: Extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and entity interaction relationships; Step 3: Use a community detection algorithm to divide the activities in the traceability graph into different communities to obtain independent community subgraphs; Step 4: Generate a community hierarchical summary for each community, and summarize all community hierarchical summaries to form an attack chain analysis; Step 5: Determine the attack investigation question. Based on the attack investigation question, use the graph retrieval enhancement method to extract the corresponding relevant information from the traceability graph and the community hierarchical summary, process the relevant information based on the large-scale language model, and generate the network attack investigation results.

[0024] In one embodiment, step 1 specifically includes the following steps: Step 1.1: Use the corresponding log collector to collect audit logs according to different operating systems and usage scenarios; Step 1.2, use Cuckoo Auxiliary component to automate sample log analysis.

[0025] In one embodiment, the specific implementation of step 1.1 is: In Windows systems, use Sysmon and ETW tools to monitor system kernel events, application events, hardware performance, and network activities, and comprehensively track and record system activities. In the Linux system, use the Sysmon tool and Auditd tool to configure audit rules to record key system events such as file access, process startup, user authentication, and system calls, and save the logs.

[0026] In one embodiment, the specific implementation of step 1.2 is: Use Cuckoo Auxiliary components to customize additional Sysmon log collection function modules. After completing sample analysis, the client automatically returns complete Sysmon log records and restores sandbox and Sysmon configuration snapshots after each analysis, thus automating sample log analysis.

[0027] In one embodiment, step 2 specifically includes the following steps: Step 2.1, preprocessing the original audit log data obtained in step 1.1; Step 2.2, extract key entities and the relationships between these entities from the preprocessed logs and store them in the form of a database; Step 2.3: Construct the extracted key entities and entity interaction relationships into a traceability graph.

[0028] In one embodiment, the specific implementation of step 2.1 is: Perform denoising and deduplication processing on the original log data to remove irrelevant or low-quality noise log records, such as operating system startup records, etc. Compare the key fields of the logs and remove completely duplicate log records to prevent noise and duplicate records from affecting the analysis results. When processing logs from multiple systems or devices, format and align timestamps from different sources in a unified manner, and arrange log events in chronological order.

[0029] In one embodiment, the specific implementation of step 2.2 is: Traverse the preprocessed logs and extract three types of entities, namely, process, file and network connection socket, as key entities one by one. For different types of key entities, extract the corresponding key entity attributes. For process entities, extract the process number, process path and command line parameters; for file entities, extract the file name and absolute path of the file; for network connection entities, extract the network IP, port and protocol; Extract relationships between key entities such as process creation, file reading and writing, and network connections; Assign unique identifiers to different entities as nodes in the graph. For process entities, use "process name + PID" as the unique identifier. For file entities, use the absolute path of the file in the system as the unique identifier. For network connection entities, use<IP, Port, Protocol> The combination serves as a unique identifier; The extracted entities, entity attribute information and interactions between entities are stored in the form of a database.

[0030] In one embodiment, the specific implementation of step 2.3 is: Step 2.3.1, initialize the traceability graph, construct the traceability graph as a directed graph, and define the graph structure G = (V, E), where V is the set of nodes in the graph and E is the set of directed edges in the graph; Step 2.3.2, obtain all system event information from the database saved in step 2.2, and traverse all events; Step 2.3.3, for each event, extract the source node (src), the destination node (dst), and the related event details, and construct the event as a directed edge from the source node (src) to the destination node (dst); Step 2.3.4, update the graph structure and add the source node, target node and corresponding directed edges to graph G.

[0031] In one embodiment, step 3 specifically includes: Step 3.1, node initialization and movement, treat all nodes as separate communities, and then try to move each node to the community where the adjacent node is located to maximize the modularity; Step 3.2, after completing the preliminary community division, the Leiden algorithm moves nodes within each community to ensure that the nodes within the community have a higher modularity; Step 3.3: After the community division phase, the algorithm treats each community as a supernode, simplifies the graph structure, and repeats the previous steps until the modularity is no longer significantly improved.

[0032] In one embodiment, step 3.1 is specifically implemented as follows: At the beginning of the Leiden algorithm, each node is assigned an independent community ID, and the node determines whether to change the community it belongs to based on its connection relationship with other nodes. The node will try to move itself to a new community, thereby improving the modularity of the community structure.

[0033] In one embodiment, step 3.2 is specifically implemented as follows: After completing node initialization and movement, the community is further divided, and the connection density between each node and other members of the community is calculated to determine whether the node should belong to another community. This process can be repeated, and each node decides whether to change its community based on the modularity gain; The process of node movement is greedy, that is, the community with the largest gain is selected each time. The main basis for node movement is the strength of its connection with other nodes. Nodes tend to join communities that can improve modularity.

[0034] In one embodiment, step 3.3 is specifically implemented as follows: After completing the community division and node movement, each community is regarded as a new "super node", and the connections between these super nodes are recalculated to re-divide the community; After the division is completed, the nodes are moved again to optimize the local area to ensure that the merged community can still maximize the modularity.

[0035] In one embodiment, step 4 specifically includes: Step 4.1, generate entity summary, describing each entity in the traceability graph, including processes, files and network connections. When generating entity summary, the attributes of the entity will be input into LLM to obtain the corresponding description; Step 4.2, relationship summary generation, summarizes the main interaction behaviors between entities in the community, such as process creation, file access, network connection, etc. In this way, the behavior between two entities can be specifically described; Step 4.3, community summary generation, provides an overview of the entire community, integrates entity summaries and relationship summaries in the community, and comprehensively generates attack intentions, attack steps, and their impacts in the community.

[0036] In one embodiment, step 4.1 is specifically implemented as follows: The construction of the traceability graph takes into account three entities, including processes, files, and network connections. Prompts are designed for the three entities to input node information into the LLM in a structured form to obtain a description of the entity, and the LLM is required to return an entity summary of a specific structure as a specified output.

[0037] In one embodiment, step 4.2 is specifically implemented as follows: Design the prompt words input to LLM. The description of the relationship should be able to describe the specific behavior between the two entities. For example, process A creates process B at time T, and the command line is C. The specific parameters indicate that process A is performing network detection by creating process B. The design prompt inputs the relationship information into LLM in a structured form to obtain a description of the relationship, and requires LLM to return a relationship summary of a specific structure as a specified output.

[0038] In one embodiment, step 4.3 is specifically implemented as follows: Generate a summary of the community obtained by community detection, integrate the entity summary and relationship summary in the community, generate a community title and community summary based on the entities and relationships in the community, and give a threat score and an explanation of the score. Finally, give the discovery of the community, such as what role a certain entity plays in the community. Design a prompt to require LLM to return a community summary of a specific structure with a specified output.

[0039] In one embodiment, step 5 specifically includes: Step 5.1, receiving the query (i.e., prompt word) input by the security analyst, and using natural language processing technology to determine the key objectives and context requirements of the query; Step 5.2, combine the graph retrieval enhancement generation (GraphRAG) technology to improve the accuracy and completeness of query responses and perform global and local retrieval; Step 5.3, after completing the retrieval task, the retrieved context is input into the collocation LLM, and the LLM combines the retrieved relevant information to generate the final natural language answer.

[0040] In one embodiment, step 5.1 is specifically implemented as follows: By receiving the query (prompt word) input by the user, the word embedding model is used to analyze the grammatical structure of the query sentence and identify the key targets, which may include attack nodes, attack paths or attacker behaviors.

[0041] In one embodiment, step 5.2 is specifically implemented as follows: Step 5.2.1, global search The Map-Reduce technology is used to perform global question retrieval. In the mapping stage, LLM is used to divide the community reports into multiple batches to generate intermediate responses with scores. The scores are used to indicate the importance of this opinion. In the reduction phase, the mapping results are merged to form a response to the overall query, and the output includes a global view of the attack event and potential attack paths; Step 5.2.2, local search Using vector search or keyword search technology, the parsed keywords are searched to identify a set of initial nodes or document sets that are conceptually similar to the user query. These initial search results are then traversed and sorted, and the most relevant documents are displayed to generate a response.

[0042] In one embodiment, step 5.3 is specifically implemented as follows: The retrieved context information is input into the LLM, and the prompt is designed to generate the final natural language answer and feed it back to the security analyst.

[0043] The network attack investigation system based on enhanced source graph retrieval provided by the present invention is described below. The network attack investigation system based on enhanced source graph retrieval described below and the network attack investigation method based on enhanced source graph retrieval described above can be referenced to each other.

[0044] Figure 2 is a schematic diagram of the structure of a network attack investigation system based on enhanced traceability graph retrieval provided by an embodiment of the present invention. Figure 2 As shown, including: Log collection module, used to collect system activity data and generate audit logs that fully cover host and network behavior; A traceability graph construction module is used to extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and the entity interaction relationships; A community detection module, used to divide the activities in the traceability graph into different communities using a community detection algorithm to obtain independent community subgraphs; Hierarchical summary module, used to generate community hierarchical summaries for each community and aggregate all community hierarchical summaries to form attack chain analysis; The investigation interaction module is used to determine the attack investigation questions, and according to the attack investigation questions, use the graph retrieval enhancement method to extract the corresponding relevant information from the traceability graph and the community hierarchical summary, and process the relevant information based on the large-scale language model to generate the network attack investigation results.

[0045] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a network attack investigation method based on the enhanced retrieval of the traceability graph, the method comprising: collecting system activity data, generating an audit log that fully covers the host and network behaviors; extracting key entities and entity interaction relationships in the system from the audit log, and constructing a traceability graph from the key entities and the entity interaction relationships; using a community detection algorithm to divide the activities in the traceability graph into different communities to obtain independent community subgraphs; generating a community hierarchical summary for each community, and summarizing all community hierarchical summaries to form an attack chain analysis; determining an attack investigation question, and according to the attack investigation question, extracting corresponding relevant information from the traceability graph and the community hierarchical summary using a graph retrieval enhancement method, processing the relevant information based on a large-scale language model, and generating a network attack investigation result.

[0046] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0047] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0048] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network attack investigation method based on provenance graph retrieval enhancement, characterized in that: include: Collect system activity data and generate audit logs that fully cover host and network behavior; Extract key entities and entity interaction relationships in the system from the audit log, and construct a traceability graph based on the key entities and the entity interaction relationships; A community detection algorithm is used to divide the activities in the traceability graph into different communities to obtain independent community subgraphs; Generate a community hierarchical summary for each community, and aggregate all community hierarchical summaries to form an attack chain analysis; Determine the attack investigation question, and based on the attack investigation question, use a graph retrieval enhancement method to extract corresponding relevant information from the traceability graph and the community hierarchical summary, process the relevant information based on a large-scale language model, and generate a network attack investigation result.

2. The network attack investigation method based on provenance graph retrieval enhancement according to claim 1 is characterized in that: Collects system activity data and generates comprehensive audit logs covering host and network behavior, including: A variety of log collectors are used to collect the audit logs in different operating systems and scenarios. In Windows, Sysmon and ETW tools are used to monitor system kernel events, application events, hardware performance, and network activities, and to comprehensively track and record system activities. In Linux, Sysmon and Auditd tools are used to configure audit rules to record key system events and save logs. The key system events include file access, process startup, user authentication, and system calls. Use Cuckoo Auxiliary component to implement Sysmon log collection function. After the client completes sample analysis, it automatically returns Sysmon log records. After each analysis, it restores the sandbox and Sysmon configuration snapshots to automatically complete sample log analysis.

3. The network attack investigation method based on source graph retrieval enhancement according to claim 1 is characterized in that: Extracting key entities and entity interaction relationships in the system from the audit log, and constructing a traceability graph based on the key entities and the entity interaction relationships, including: Preprocessing the audit log to obtain a preprocessed log record; Extracting the key entities and the entity interaction relationships from the preprocessed log records, and saving the key entities and the entity interaction relationships to a database; The extracted key entities and the entity interaction relationships are constructed into the traceability graph.

4. The network attack investigation method based on provenance graph retrieval enhancement according to claim 3 is characterized in that: The audit log is preprocessed to obtain a preprocessed log record, including: Perform denoising and deduplication processing on the original audit logs to remove irrelevant or low-quality noise log records and completely duplicate log records; Unify and align the formats of timestamps from different sources of logs in multiple systems or devices, and arrange log events in chronological order; Correspondingly, extracting the key entity and the entity interaction relationship from the preprocessed log record, and saving the key entity and the entity interaction relationship to a database, includes: Traversing the preprocessed log records, extracting three types of key entities, namely, process, file and network connection socket, one by one, and extracting corresponding key entity attributes for different types of key entities, wherein the process number, process path and command line parameters are extracted for the key entity of the process type, the file name and absolute path of the file are extracted for the key entity of the file type, and the network IP, port and protocol are extracted for the key entity of the network connection type; Extract process creation, file reading and writing, and network connections between different types of key entities; Assign unique identifiers as graph nodes to different types of key entities. For process-type key entities, use process name + PID as the unique identifier. For file-type key entities, use the absolute path of the file in the system as the unique identifier. For network connection-type key entities, use<IP, Port, Protocol> The combination serves as a unique identifier; The extracted key entities, key entity attribute information and the interactions of the entity interaction relationships are stored in the form of a database; Correspondingly, the extracted key entities and the entity interaction relationships are constructed into the traceability graph, including: Initialize the traceability graph, construct the traceability graph as a directed graph, and determine the directed graph structure including a graph node set and a graph directed edge set; Obtain all system event information from the database and traverse all events; Extract the source node, target node and related event details from each event, and construct each event as a directed edge from the source node to the target node; The directed graph structure is updated, and the source node, the target node, and the corresponding directed edges are added to the directed graph structure.

5. The network attack investigation method based on provenance graph retrieval enhancement according to claim 1 is characterized in that: The community detection algorithm is used to divide the activities in the traceability graph into different communities to obtain independent community subgraphs, including: Perform node initialization and movement, determine that all nodes are separate communities, and move each node to the community where the adjacent node is located to maximize modularity; After the initial community division is completed, the Leiden algorithm is used to move nodes within each community to make the nodes within the community obtain a higher modularity; After the community division is completed, each community is regarded as a super node, the directed graph structure is simplified, and the above steps are repeated until the modularity no longer increases.

6. The network attack investigation method based on provenance graph retrieval enhancement according to claim 5 is characterized in that: Initialize and move nodes, determine that all nodes are separate communities, and move each node to the community where the adjacent node is located to maximize modularity, including: At the beginning of the Leiden algorithm, an independent community ID is assigned to each node. The node determines whether to change its community based on its connection relationship with other nodes. The node moves to a new community to improve the modularity of the community structure. Correspondingly, after the initial community division is completed, the Leiden algorithm is used to move nodes within each community so that the nodes within the community have a higher modularity, including: After completing node initialization and movement, the community is divided, and the connection density between each node and other members of the community is iteratively calculated to determine whether the node belongs to another community; Nodes are moved based on the connection strength between the nodes and other nodes, and a greedy movement process is used to determine the community with the maximum gain. Correspondingly, after the community division is completed, each community is regarded as a super node, the directed graph structure is simplified, and the above steps are repeated until the modularity is no longer improved, including: After completing the community division and node movement, each community is determined to be a new super node, the connections between the new super nodes are recalculated, and the communities are re-divided; After the redivision of the community is completed, the nodes are moved again to optimize the local area to maximize the modularity of the merged community.

7. The network attack investigation method based on provenance graph retrieval enhancement according to claim 1 is characterized in that: Generate a community hierarchical summary for each community, and aggregate all community hierarchical summaries to form an attack chain analysis, including: Generate an entity summary to describe each entity in the traceability graph, including processes, files, and network connections, and input entity attributes into a large-scale language model to obtain an entity description; Generate a relationship summary that summarizes the main interaction behaviors between entities in the community, including process creation, file access, and network connection to describe the behavior between two entities; A community summary is generated to provide an overview of the entire community, and the entity summary and the relationship summary are combined to generate attack intentions, attack steps, and attack impacts in the community.

8. The network attack investigation method based on enhanced traceability graph retrieval according to claim 7 is characterized in that: Generate an entity summary to describe each entity in the traceability graph, including processes, files, and network connections, and input entity attributes into a large-scale language model to obtain an entity description, including: Determine prompts for the process, the file, and the network connection respectively, input node information into a large-scale language model in a structured form to obtain entity descriptions, and enable the large-scale language model to return entity summaries of a specific structure with a specified output; Correspondingly, a relationship summary is generated to summarize the main interaction behaviors between entities in the community, including process creation, file access, and network connection, to describe the behavior between two entities, including: According to the quality of the prompt words input into the large-scale language model, the relationship information description reflects the specific behavior between the two entities; Determine prompt to input relation information into a large-scale language model in a structured form to obtain relation description, and make the large-scale language model return a relation summary of a specific structure with a prescribed output; Correspondingly, a community summary is generated to provide an overview of the entire community, and the entity summary and the relationship summary are combined to generate the attack intention, attack steps and attack impact in the community, including: Generate a community title and a community summary according to the entity summary and the relationship summary, and output a threat score and a score explanation; Determine the prompt output community discovery and enable large-scale language models to return community summaries of specific structures as prescribed outputs.

9. The network attack investigation method based on provenance graph retrieval enhancement according to claim 1 is characterized in that: Determine the attack investigation question, and based on the attack investigation question, use the graph retrieval enhancement method to extract the corresponding relevant information from the traceability graph and the community hierarchical summary, process the relevant information based on the large-scale language model, and generate the network attack investigation results, including: Receiving prompt words of the attack investigation question, parsing the prompt words, and using natural language processing to determine key objectives and context requirements of the query; Use graph retrieval enhancement generation technology to obtain query responses; After completing the retrieval task, the retrieved context is input into the large-scale language model, which combines the retrieval information to generate the final natural language answer.

10. The network attack investigation method based on enhanced traceability graph retrieval according to claim 9 is characterized in that: Receiving prompt words of the attack investigation question, parsing the prompt words, and using natural language processing to determine key objectives and context requirements of the query, including: According to the prompt word, using a word embedding model, the grammatical structure of the prompt word query sentence is parsed to identify key targets, wherein the key targets include attack nodes, attack paths, and attacker behaviors; Correspondingly, the graph retrieval enhancement generation technology is used to obtain query responses, including: A global search implementation is adopted, and Map-Reduce technology is used to perform global question search. In the mapping stage, a large-scale language model is used to divide community reports into multiple batches, and intermediate responses with scores are generated. The scores represent the importance of opinions. In the reduction stage, the mapping results are merged and the overall query response is output. The overall query response includes a global view of the attack event and potential attack paths; It adopts the local retrieval implementation method, uses vector search or keyword search technology, searches and parses keywords, identifies a set of initial nodes or document sets that are similar to the user's query concept, traverses and sorts the initial search results, and displays the most relevant documents to generate a response.

Images