Network threat behavior reasoning method and system based on large model retrieval enhancement

Abstract

The invention discloses a network threat behavior reasoning method and system based on large model retrieval enhancement, and the method comprises the following steps: a computer system executes the following steps: determining a query target and an associated scene or organization by receiving a network threat analysis query request of a user; retrieving related nodes and relationships from the threat intelligence knowledge graph, and generating an initial reasoning sub-graph; calling a GraphRAG framework, and carrying out semantic matching retrieval on a subgraph node context and an external threat intelligence knowledge base to obtain supplementary evidence; constructing a comprehensive prompt template, inputting the large language model subjected to LoRA fine tuning, and generating a conformity ATTamp; reasoning a conclusion of the CK framework; and finally, supplementing the newly added nodes and the relationship to the knowledge graph, and outputting a structured file or a visual result. According to the method, retrieval enhancement and large model reasoning technologies are combined, the problems of cross-document attack chain restoration and complex threat behavior logical reasoning are effectively solved, and the method has remarkable application value in the aspects of threat intelligence tracing, behavior prediction and defense decision assistance.

Description

Technical Field

[0001] This invention relates to the technical fields of network security and industrial internet security, specifically to technical directions such as network threat intelligence analysis, APT attack behavior reasoning, threat prediction and defense strategy generation in computer security, and particularly to a network threat behavior reasoning method, system, electronic device and storage medium based on large model retrieval enhancement. Background Technology

[0002] As cybersecurity threats continue to evolve, the number of attacks by transnational APT groups, zero-day exploits, and multi-stage attack chains continues to grow. The stealth and complexity of these cyberattacks have significantly increased, directly driving an explosive growth in the scale and complexity of Cyber ​​Threat Intelligence (CTI) data. Currently, APT analysis reports, vulnerability announcements, technical bulletins, and open-source intelligence are continuously generated from multiple sources, resulting in massive amounts of heterogeneous threat intelligence. This intelligence is not only scattered in origin, but also often uses aliases, abbreviations, or vague descriptions of the same attacking organization or tools. The upstream and downstream dependencies of attack behaviors often span multiple text paragraphs or even multiple reports. This directly leads to significant challenges for existing technologies in threat knowledge extraction, attack chain reconstruction, and behavioral reasoning.

[0003] While existing cyber threat behavior inference technologies have made initial explorations to address the aforementioned challenges, they still have significant limitations: on the one hand, the same threat entity lacks effective mapping across different intelligence sources, leading to information isolation and knowledge fragmentation; on the other hand, rule-based or keyword-based detection methods can only achieve shallow identification of IOCs and cannot characterize complex attack chains. Furthermore, while deep learning-based entity extraction and relationship recognition improve automated processing capabilities, they have shortcomings in cross-document inference and multi-source knowledge fusion. Meanwhile, graph neural network inference models rely on shallow associations such as entity co-occurrence and lack deep integration with authoritative external knowledge bases such as ATT&CK and CVE, resulting in insufficient consistency and completeness of inference results.

[0004] Specifically, among mainstream technologies, rule-based and ontology-based methods, while modeling and matching threat entities and attack patterns through predefined rules, regular expressions, or security standards, can efficiently standardize IOCs and ensure standardized expressions, they cannot support cross-document and multi-hop reasoning, making it difficult to characterize the complex attack logic of APT organizations. Secondly, while knowledge graph-based and graph reasoning methods abstract attack organizations and vulnerabilities into nodes to construct attack graphs or threat knowledge graphs, and utilize graph algorithms or graph neural networks (such as GCN and GAT) for relationship reasoning and link discovery, the lack of integration with authoritative knowledge bases leads to fragmented reasoning results.

[0005] Besides the two categories mentioned above, although behavior prediction methods based on machine learning and deep learning can improve the automation of reasoning and achieve attack step prediction and behavior chain reasoning, they rely on large-scale training data, have limited generalization ability, and have obvious shortcomings in interpretability and cross-knowledge base linkage.

[0006] In summary, while existing technologies have made some progress in the structuring and analysis of network threat intelligence, three core problems still exist: First, the semantic alignment capability is weak, making it difficult to unify the same attack entity under different aliases, abbreviations, or translations; second, there is a lack of cross-document and external knowledge base linkage reasoning capabilities, making it impossible to establish logical causal relationships between multi-source intelligence; and third, the reasoning results lack global consistency, limiting a comprehensive understanding of the evolution of APT attack patterns and complex attack chains. Summary of the Invention

[0007] The purpose of this invention is to provide a network threat behavior inference method, system, electronic device and storage medium based on large model retrieval enhancement, so as to solve the problems of cross-document information fragmentation, difficulty in inferring upstream and downstream links and lack of consistency of inference results in the existing technology in APT attack behavior analysis.

[0008] Based on a first major aspect of the present invention, a method for network threat behavior reasoning based on large model retrieval enhancement is provided, comprising the following steps performed by a computer system;

[0009] Receive network threat analysis query requests from users and determine the query target and its attack scenario or APT organization;

[0010] Based on the query target, relevant entity nodes and their semantic relationships are retrieved from the constructed threat intelligence knowledge graph to generate an initial inference subgraph containing multi-hop association paths;

[0011] The GraphRAG framework is invoked to vectorize the node context information of the initial inference subgraph, and then semantically retrieved and matched with an external threat intelligence knowledge base to obtain a supplementary evidence set related to the inference task.

[0012] The initial reasoning subgraph is merged with the supplementary evidence set to construct a comprehensive prompt template that includes reasoning task instructions, node relationship topology descriptions, and external evidence text.

[0013] The comprehensive prompt template is input into the large language model that has been lightly tuned by LoRA to generate inference conclusions that conform to the MITRE ATT&CK framework;

[0014] The newly added nodes and relationships in the reasoning conclusions are added to the threat intelligence knowledge graph, and a structured graph file or a visual reasoning result that supports interactive analysis is output.

[0015] As a further preferred embodiment, in the aforementioned method, determining the query target includes identifying the subject object, reasoning scope, and task intent of the query target through semantic parsing, and limiting boundary conditions through predefined context constraint rules;

[0016] The generation of the initial inference subgraph containing multi-hop association paths also includes performing relation constraint detection and redundant edge elimination, as well as using a node importance ranking and threshold filtering mechanism to remove nodes with low relevance to the query target;

[0017] The external threat intelligence knowledge base includes the MITRE ATT&CK tactical technology library, the CVE vulnerability database, the open-source APT intelligence analysis report library, and the STIX 2.1 format standardized threat intelligence dataset.

[0018] As a further preferred option, in the aforementioned method, the GraphRAG framework performs multi-round retrieval based on node semantic representation and selects the most relevant external evidence for the reasoning task through cosine similarity or vector similarity measures.

[0019] The step of vector encoding the node context information of the initial inference subgraph includes: calling the GraphRAG framework to partition the initial inference subgraph, and extracting subgraph information in units of topics or community structures;

[0020] The semantic retrieval and matching with the external threat intelligence knowledge base includes: using node semantic embedding to perform similarity matching with the text description of the external threat intelligence knowledge base.

[0021] As a further preferred option, in the aforementioned method, the comprehensive prompt template adopts a structured coding method to hierarchically organize the entity nodes, semantic relationships, and supplementary evidence sets. At the same time, weight control and constraint rules are introduced to place core nodes and high-confidence evidence in prominent positions, while low-confidence information serves as auxiliary background.

[0022] As a further preferred embodiment, in the aforementioned method, the fine-tuning method for the large language model after LoRA lightweight fine-tuning includes:

[0023] While freezing the original pre-trained weights, only the low-rank matrix parameters in the attention sublayer and the feedforward network are updated;

[0024] The LoRA lightweight fine-tuning is as follows:

[0025]

[0026] Where W is the original frozen weight matrix; A and B are low-rank trainable matrices;

[0027] The generation of reasoning conclusions conforming to the MITRE ATT&CK framework includes: using the large language model to analyze the causal relationships and potential dependencies between nodes layer by layer through a multi-hop logical deduction mechanism during the reasoning process, and generating reasoning conclusions conforming to the MITRE ATT&CK framework.

[0028] The reasoning conclusions include newly added attack stage nodes, attack tool nodes and their semantic relationships, and update the edge weights in the original graph to reflect the evolution of the attack chain.

[0029] As a further preferred embodiment, in the aforementioned method, the newly added nodes and relationships include the newly added attack stage nodes, attack tool nodes, and the semantic relationships between them;

[0030] When supplementing the threat intelligence knowledge graph, a consistency check and redundancy detection mechanism is used for verification and detection.

[0031] The output visualization reasoning results are based on a graphical interface that displays the attack chain reconstruction process. The interface supports node aggregation, path expansion, and interactive queries across document links.

[0032] Based on a second key aspect of the present invention, a network threat behavior inference system based on large model retrieval enhancement is provided, comprising:

[0033] The query module is used to receive network threat analysis query requests input by users and determine the query target and related scenarios through semantic parsing;

[0034] The graph retrieval module connects to the constructed threat intelligence knowledge graph and is used to retrieve relevant nodes and relationships and generate an initial inference subgraph.

[0035] The retrieval enhancement module is used to call the GraphRAG framework to perform semantic matching between the inference subgraph and an external threat intelligence knowledge base to obtain a supplementary evidence set.

[0036] The prompt generation module is used to integrate the initial reasoning subgraph with the supplementary evidence set to generate a comprehensive prompt template that includes task instructions, node relationships and external evidence;

[0037] The large model reasoning module has a built-in large language model that has been lightly tuned by LoRA, which is used to process the comprehensive prompt template and generate reasoning conclusions that conform to the MITRE ATT&CK framework.

[0038] The graph update and output module is used to update the inference conclusions to the threat intelligence knowledge graph and output a structured graph file or a visualized inference result.

[0039] As a further preferred embodiment, in the aforementioned system, the query module includes:

[0040] The semantic parsing unit is used to perform entity recognition, intent judgment, and ambiguity resolution on natural language queries; the keyword recognition unit is used to extract core keywords such as attack organizations, vulnerability numbers, and tool names from the query; and the scene mapping unit is used to associate the parsing results with a preset attack scene tag library.

[0041] Based on a third key aspect of the present invention, an electronic device is provided, comprising: at least one processor; a memory communicatively connected to the processor; the memory storing a computer program that, when executed by the processor, enables the implementation of the network threat behavior inference method based on large model retrieval enhancement as described above.

[0042] Based on a fourth key aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon that, when executed, implements the aforementioned network threat behavior inference method based on large model retrieval enhancement.

[0043] Compared with existing technologies, this invention addresses the problems of information fragmentation, missing cross-document links, and insufficient consistency of inference results in APT threat behavior analysis through a core approach that combines enhanced retrieval with large-scale model inference. Furthermore, this invention significantly improves the completeness and interpretability of inference through multi-stage semantic fusion and logical constraint mechanisms, effectively compensating for the technical shortcomings of existing technologies in complex threat technologies.

[0044] Secondly, experimental results verify the technical advantages of this invention. Firstly, the invention improves the inference integrity of real APT intelligence datasets by more than 20%, and secondly, it improves the cross-document consistency index by more than 15%. Furthermore, while ensuring inference performance, this invention achieves significant optimization in computational overhead, balancing analytical effectiveness and operational efficiency.

[0045] Finally, this invention can provide key technical support for network threat tracing, attack prediction, and defense strategy generation. It can directly serve the threat defense work of related industries, help improve the understanding and response capabilities to complex APT attacks, and has broad application prospects. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0047] Figure 1The following is an execution flowchart of a network threat behavior inference method based on large model retrieval enhancement in one embodiment of the present invention;

[0048] Figure 2 This illustrates a network threat behavior reasoning process based on RAG in one embodiment of the present invention;

[0049] Figure 3 This invention illustrates an embodiment of the use of LoRA for lightweight fine-tuning of a large model;

[0050] Figure 4 This paper illustrates the construction process of an intelligent question-answering model for APT attacks based on a large language model and knowledge graph in one embodiment of the present invention. Detailed Implementation

[0051] The preferred embodiments of the present invention will be described in detail below to provide a clearer understanding of the purpose, features, and advantages of the invention. It should be understood that the following embodiments are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the technical solution of the invention.

[0052] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known techniques associated with the invention may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0053] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0054] The specific meanings of the technical terms or English abbreviations that may be used in this invention are explained below:

[0055] APT: Advanced Persistent Threat, is a complex cyberattack model that targets specific objectives, combines advanced technical means, and aims at long-term infiltration and intelligence theft.

[0056] STIX 2.1: Structured Threat Intelligence Expression.

[0057] CVE: Common Vulnerability Disclosure.

[0058] CNVD: National Information Security Vulnerability Contribution Platform.

[0059] The GraphRAG framework (Graph Retrieval-Augmented Generation Framework) refers to an inference enhancement method that combines knowledge graph retrieval with external threat intelligence retrieval results. In this invention, GraphRAG performs topic partitioning and node semantic matching on the initial inference subgraph, jointly retrieves highly relevant evidence from the knowledge graph and external threat intelligence database, and integrates it with structured information from the inference task to generate a comprehensive prompt template adapted to a large language model, significantly improving the accuracy and contextual completeness of the inference results.

[0060] In-Context Learning Prompt Template (ICL) refers to a contextual prompt structure built before input to a large language model, typically including three parts: task instructions, example samples, and the current input. In this invention, the ICL template is automatically generated in conjunction with semantic similarity retrieval, supporting multi-task switching such as APT organization affiliation, vulnerability exploitation, and tool targeting, significantly improving semantic consistency and contextual guidance capabilities.

[0061] LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique for large language models. By inserting low-rank matrices into the attention layer and feedforward network, it achieves model adaptation tasks by updating only a small number of parameters. In this invention, LoRA is used to perform lightweight fine-tuning of large models, effectively enhancing their knowledge alignment and extraction capabilities in safe contexts and significantly reducing training resource consumption.

[0062] Dynamic Update Mechanism: This refers to automatically merging newly generated nodes and relationships into the existing network threat intelligence knowledge graph while retaining their source information. This invention synchronously maintains the relationship edges associated with new nodes during updates, ensuring the coherence of the knowledge graph structure and the traceability of the data.

[0063] Cross-Knowledge Source Semantic Fusion refers to the semantic alignment and fusion of textual information from knowledge graph node descriptions and external threat intelligence knowledge bases, eliminating issues such as naming differences, abbreviations, aliases, and translation inconsistencies. This invention employs a three-stage strategy combining coarse-grained clustering and fine-grained embedding similarity matching to ensure the uniformity and integrity of the inference input context.

[0064] APT Attack Chain Deduction: This refers to using comprehensive prompt templates and large language models to infer the attack stages, attack techniques, tool usage, and vulnerability exploitation elements of APT organizations, forming a complete attack chain path. This invention introduces the ATT&CK framework during the inference process to standardize the annotation of attack stages, supporting chronological playback and visualization.

[0065] APT attack chain reconstruction mechanism: This refers to reconstructing the attack paths and behavioral chains of APT organizations by performing graph-level fusion of entities and relationships extracted from multiple reports. This invention utilizes multi-dimensional features such as entity timestamps, context paragraph numbers, and similarity tags to construct attack event sequences, supporting time-series visualization and replay of the attack chain.

[0066] Graph-structured knowledge representation: Threat intelligence entities such as APT organizations, vulnerability IDs, attack tools, and malware families are modeled as nodes, and the abstract relationships between them, such as exploitation, attack, and target orientation, are modeled as edges to form a knowledge graph. This invention's graph supports exporting to multiple standard graph formats such as GraphML and Neo4j, and can be integrated into visualization platforms for situational analysis.

[0067] Dynamic updater for inference conclusions: This refers to automatically merging newly generated nodes and relationships into the existing network threat intelligence knowledge graph while retaining their source information. This invention synchronously maintains the relationship edges associated with new nodes during updates, ensuring the coherence of the knowledge graph structure and the traceability of the data.

[0068] Instruction-Example Dynamic Switching Mechanism: The ICL template design of this invention supports automatic switching of task instructions and example content based on the input paragraph topic (e.g., vulnerability number priority, APT tool priority). For example, in detecting the text "APT28 exploit CVE-2022-30190", examples of the "vulnerability exploitation" category will be prioritized to improve the model's ability to focus on relationship identification.

[0069] Multi-round semantic fusion clustering: This invention adopts a progressive similarity threshold setting. The first round captures generalized near-sense candidate entities (similarity > 0.7), the second round raises the threshold to accurately merge core nodes (similarity > 0.85), and finally combines the domain expert database for manual mapping verification to balance aggregation granularity and semantic accuracy.

[0070] APT tag auto-completion: During the knowledge graph construction phase, this invention performs cross-validation with the ATT&CK tactical library and the CVE vulnerability library. For example, after identifying "APT-C-23 attack T1059 script execution", it automatically completes the attack phase "Execution" and the known exploit tool "PowerShell", improving the completeness and interpretability of the knowledge graph.

[0071] Context-Aware Retrieval refers to the process of comprehensively considering the reasoning objective, node semantic similarity, relational path weights, and contextual topic relevance during evidence retrieval to select the evidence set that best matches the current reasoning task. In this invention, this mechanism is implemented throughout the GraphRAG retrieval stage, ensuring the relevance and specificity of the reasoning input.

[0072] Threat Intelligence Multi-Label Classifier: A weakly supervised multi-label classification mechanism is introduced during the fine-tuning of the large language model. The extracted entities are dynamically bound with labels (such as malware / tools / organizations / tactics, etc.), which supports the model to automatically determine the optimal category in semantically overlapping scenarios, effectively reducing misclassification and redundant entities.

[0073] The GraphML format export module exports the final constructed knowledge graph structure as a standardized GraphML file, which includes metadata such as entity node attributes, semantic relationship edges, and source document reference information. It can be directly connected to the Neo4j graph database or other graph engines to achieve efficient graph query and visualization interaction.

[0074] Attack chain visualization rendering engine: As one of the application components of this invention, this module automatically generates an APT attack chain path view after accepting the graph structure input. It supports interactive functions such as node highlighting, path tracing, timeline evolution, and multi-dimensional switching (by organization / industry / vulnerability / stage) to assist analysts in tracing and predicting the source.

[0075] Dependency Chain-Driven Policy Linkage Interface: This invention supports pushing discovered attack dependency chains to the defense system policy interface based on graph construction, and automatically generating defense rules by combining YARA rules, EDR engine, etc., to achieve an integrated closed loop from intelligence extraction to policy linkage.

[0076] MITRE ATT&CK Tactical and Technical Database: This is a knowledge base and classification system for cyberattack behavior maintained by the US non-profit organization MTIRE, focusing on depicting the tactical objectives and technical means of APT groups throughout the attack lifecycle.

[0077] The CVE vulnerability database is an internationally standardized cybersecurity vulnerability dictionary maintained by the US non-profit organization MTIRE. It assigns unique numbers to known vulnerabilities worldwide and provides standardized descriptions.

[0078] The Open Source APT Intelligence Analysis Report Repository is an open-source resource collection that aggregates real APT attack analysis reports and is contributed publicly by the cybersecurity research community.

[0079] The STIX 2.1 format standardized threat intelligence dataset is a machine-readable standardized collection of threat intelligence built on the STIX 2.1 protocol specification. Its core is to achieve cross-platform exchange, automated analysis and collaborative defense of threat intelligence through a unified data model and format.

[0080] like Figure 1 As shown, in one embodiment of the present invention, a network threat behavior inference method based on large model retrieval enhancement includes the following steps S110-S160 performed by a computer system:

[0081] S110, the computer system processor executes a computer program to receive a network threat analysis query request input by the user, determines the query target through semantic parsing, and identifies the attack scenario or APT organization associated with the query target;

[0082] In practical applications, users can input their analysis requirements through the system front end, such as "reasoning about the attack chain path of a certain APT group" or "analyzing recent attack behavior patterns related to a certain attack activity." The system parses the query request, extracts keywords, analyzes the target type (such as attacker, victim, vulnerability, tactics, etc.), and generates a description of the query target.

[0083] S120, Based on the query target, retrieve relevant entity nodes and semantic relationships between nodes from the constructed threat intelligence knowledge graph, and generate an initial inference subgraph containing multi-hop association paths;

[0084] The knowledge graph stores entity nodes (such as APT groups, malware, attack techniques, CVE vulnerabilities, IOC metrics, etc.) and their semantic relationships in threat intelligence. The system retrieves nodes and adjacent relationships related to the query target using graph database query languages ​​(such as Cypher and SPARQL) to obtain an initial inference subgraph containing core nodes and context nodes.

[0085] S130, The GraphRAG framework is invoked to vectorize the node context information of the initial inference subgraph and then perform semantic retrieval and matching with an external threat intelligence knowledge base to obtain a supplementary evidence set related to the inference task;

[0086] The GraphRAG framework first partitions the initial inference subgraph and extracts subgraph information by topic or community structure. Then, it uses node semantic embedding to perform similarity matching with text descriptions in external knowledge bases (such as the MITRE ATT&CK technical library, CVE vulnerability database, and IOC database). Finally, it retrieves external evidence that is highly relevant to the inference target, including attack technique descriptions, known attack cases, vulnerability exploitation methods, and associated IOC data.

[0087] S140, the structured information of the initial reasoning subgraph is fused with the supplementary evidence set to construct a comprehensive prompt template containing reasoning task instructions, node relationship topology descriptions and external evidence text;

[0088] This prompt template consists of four parts:

[0089] (1) Reasoning task instructions, used to clarify the reasoning objectives and output requirements;

[0090] (2) Node and relationship descriptions, used to convey structured information in the knowledge graph;

[0091] (3) External evidence texts, used to provide supplementary background for the reasoning process;

[0092] (4) Input parameter information, used to control the fine-grained features of the inference results (such as output format, attack chain resolution, etc.).

[0093] S150, the comprehensive prompt template is input into the large language model that has been lightly tuned by LoRA to generate inference conclusions that conform to the MITREATT&CK framework. The inference conclusions include potential attack chain paths, attack behavior patterns and implicit entity node relationships.

[0094] The LoRA-tuned large language model enhances its adaptability to threat intelligence tasks based on pre-training. During the inference phase, the model combines structured information from the knowledge graph with external evidence text for comprehensive analysis, inferring possible attack chain stages of APT groups, potential technological exploitation methods, and undocumented inter-entity relationships.

[0095] S160, the newly added nodes and relationships in the reasoning conclusion are added to the threat intelligence knowledge graph, and a structured graph file or a visual reasoning result that supports interactive analysis is output.

[0096] The system analyzes the reasoning conclusions and adds new entity nodes (such as new attack techniques, potential vulnerabilities, and unknown IOCs) and relationship edges to the knowledge graph, enabling dynamic expansion of the graph. The updated graph can be exported as a structured file or displayed through a front-end visualization interface to support subsequent analysis by security analysts.

[0097] In the above six steps of this embodiment, the target location, initial graph construction, external evidence supplementation, prompt template generation, large model reasoning, and graph update and result output are realized respectively. There are dependencies and logical progression between the steps, which ensures the accuracy and consistency of the overall reasoning process.

[0098] In the query target localization stage, the system first performs semantic parsing on the natural language query input by the user, identifies the subject object, reasoning scope and task intent of the query target, and limits the boundary conditions through predefined context constraint rules.

[0099] This stage not only performs keyword recognition but also combines entity mapping and scene modeling to ensure that the input query can be accurately transformed into structured task requirements. Through semantic disambiguation and relation normalization, it avoids inference target bias caused by input ambiguity, thereby providing high-quality target descriptions for subsequent graph retrieval.

[0100] In the initial graph construction phase, the system retrieves the set of nodes associated with the target entity from the knowledge graph and generates an initial subgraph based on the known relationships between the nodes. This subgraph not only includes the core entity but also introduces upstream and downstream adjacent nodes to ensure the integrity of the reasoning chain. During subgraph construction, the system performs relationship constraint detection and redundant edge elimination to ensure that the graph structure has high logical closure and topological rationality.

[0101] To avoid excessive expansion leading to increased graph noise, this stage also employs a node importance ranking and threshold filtering mechanism to remove nodes with low relevance to the query target, thus maintaining the compactness and relevance of the subgraph.

[0102] During the external evidence supplementation phase, the system invokes the GraphRAG framework to expand the initial subgraph. This framework vectorizes the subgraph nodes and their contextual information, and performs semantic retrieval with an external threat intelligence knowledge base to obtain a set of evidence highly relevant to the inference objective.

[0103] External evidence includes not only descriptions of threat behaviors but also contextual information such as vulnerability details, attack tool characteristics, and organizational attributes. This stage maps and fuses external knowledge with the subgraph through semantic similarity calculation and evidence credibility assessment, ensuring consistency and interpretability between the expanded evidence and the initial graph. In this way, the inference input can cover multi-dimensional threat information, thereby improving the completeness of cross-document information aggregation.

[0104] During the prompt template generation phase, the system integrates the initial subgraph, external evidence, and task instructions to form a comprehensive prompt template. The template uses a structured coding method to hierarchically organize nodes, relationships, and semantic evidence, ensuring that the large model can utilize both logical structure and semantic content during reasoning.

[0105] To improve inference stability, weight control and constraint rules are introduced during template construction, placing core nodes and high-confidence evidence in prominent positions, while low-confidence information serves as auxiliary background, thus preventing the inference results from being interfered with by noise. This template serves as a unified input interface, enabling large models to perform inference under the joint constraints of context, external knowledge, and logical relationships.

[0106] During the large model inference phase, the synthesized prompt template is input into the large language model, which has been lightweighted and fine-tuned using LoRA. Figure 3 As shown, the model uses a multi-hop logic deduction mechanism to analyze the causal relationships and potential dependencies between nodes layer by layer during the reasoning process, generating reasoning conclusions that conform to the ATT&CK framework.

[0107] Reasoning is not limited to the direct extension of existing relationships, but also includes implicit deductions of potential links and inductions of behavioral patterns, thereby revealing relationships in the attack chain that are not explicitly recorded. This stage uses contextual consistency detection and semantic confidence calculation to filter and optimize the generated results, ensuring that the reasoning conclusions are both logically complete and practically applicable.

[0108] During the knowledge graph update and output phase, the system adds new nodes and relationships derived through reasoning to the knowledge graph, forming a dynamically evolving knowledge structure. Consistency checks and redundancy detection mechanisms are employed during the update process to ensure logical compatibility between the new content and the existing knowledge graph.

[0109] The output stage supports the generation of structured graph files, which can be directly integrated with the backend analysis system. It also supports multi-dimensional visualization, presenting attack chains, behavioral patterns, and upstream and downstream dependencies in a graphical way, helping analysts quickly understand the reasoning conclusions and make security decisions accordingly.

[0110] In one of the following possible implementations, combining Figure 2 As shown, this invention proposes a network threat behavior reasoning method based on large model retrieval enhancement;

[0111] The following examples illustrate the above-described network threat behavior inference method based on large model retrieval enhancement:

[0112] Step 1: Inference Request Input and Parsing

[0113] (1) Inference request input:

[0114] Users can input reasoning query text through the reasoning system interface, such as "inferring the recent attack chain of APT29 and the vulnerabilities exploited";

[0115] The system receives the query and performs task parsing, extracting the target entity (APT29) and task type (attack chain reasoning).

[0116] (2) Request identifier generation:

[0117] The system generates a unique task identifier for each inference task to support subsequent tracing and task tracking.

[0118] Step 2: Relevant Evidence Retrieval and Knowledge Fusion

[0119] (1) Multi-source intelligence retrieval:

[0120] The Retriever module is invoked to retrieve evidence text related to the query from the internal APT knowledge graph (KG) and external threat intelligence databases.

[0121] External databases include, but are not limited to, the MITRE ATT&CK Tactical and Technical Library, the STIX 2.1 Intelligence Files Set, the APT Analysis Report Database, and the CVE / CNVD Vulnerability Announcement Database.

[0122] (2) Semantic vector encoding and similarity calculation:

[0123] Encode the query text into vectors using a pre-trained sentence vector model (such as SecureBERT). Encode candidate evidence text into vectors ;

[0124] Calculate similarity:

[0125]

[0126] Sort by similarity and select the top-K highly relevant pieces of evidence.

[0127] (3) Knowledge integration

[0128] The retrieved textual evidence is fused with relevant nodes and edges in the knowledge graph to form a comprehensive evidence set containing structured information and contextual content.

[0129] Step 3: Constructing Reasoning Hint Templates

[0130] (1) Template structure:

[0131] By combining task instructions, Top-K evidence text, and relevant graph subgraph data, inference prompt templates are generated.

[0132] (2) Task parameter injection:

[0133] Parameters such as time range, output format requirements, and key entities for reasoning are injected into the prompt template to ensure that the reasoning process is targeted and controllable.

[0134] Step 4: GraphRAG Inference Execution and Result Generation

[0135] (1) Model configuration

[0136] We adopted a large language model of LLaMA that was finely tuned with LoRA lightweighting, frozen the backbone Transformer parameters, and only updated the low-rank matrix parameters;

[0137] The model has the ability to fuse cross-document information and infer APT attack chains.

[0138] (2) Reasoning execution

[0139] Input the inference hint template into the GraphRAG model;

[0140] The model outputs structured reasoning triples and corresponding tactical phase markers.

[0141] (3) Tactical phase mapping:

[0142] The model, combined with the MITRE ATT&CK library, automatically labels the tactical order of relations.

[0143] Step 5: Integrating Reasoning Results and Updating the Knowledge Graph

[0144] (1) Adding new entities and relations:

[0145] Write the new entity nodes obtained through reasoning (such as vulnerability numbers and technical methods) and their semantic relationship edges into the knowledge graph;

[0146] (2) Traceability information record:

[0147] Each new node and relationship is associated with the original evidence source and confidence score to support subsequent verification;

[0148] (3) Structured and visual output:

[0149] Output structured inference results in STIX 2.1 format for easy system integration;

[0150] Outputs a visual graph of the attack chain, supporting node highlighting, path tracing, and timeline evolution.

[0151] In one possible implementation, the present invention verifies the inference results and evaluates their stability;

[0152] The following examples illustrate the methods for verifying inference results and evaluating stability in detail:

[0153] Step 1: Reasoning Task and Preliminary Result Generation

[0154] (1) Input query and inference execution:

[0155] Users input inference requests, such as "inferring the attack tools and related vulnerabilities recently used by APT29";

[0156] The system uses GraphRAG retrieval to enhance the large language model to perform inference and generate preliminary inference results, including new points (such as attack tools and vulnerability numbers) and corresponding relationships (such as "exploitation" and "execution").

[0157] (2) Preliminary results are structured:

[0158] The inference output is represented in the form of "entity-relationship-entity" triples and associated with the corresponding evidence source ID.

[0159] Step 2: Consistency check of multi-source evidence

[0160] (1) Exaggerated evidence comparison:

[0161] Vectorize the original evidence paragraphs associated with each reasoning conclusion;

[0162] Calculate the semantic similarity between different pieces of evidence and calculate the mean similarity. :

[0163]

[0164] in, Let be the vector representation of the i-th matrix.

[0165] (2) Consistency judgment:

[0166] like If all the evidence supports a consistent relationship type, the inference conclusion passes the consistency check; otherwise, it enters the manual review queue.

[0167] Step 3: Inference Confidence Score

[0168] (1) Confidence level calculation formula:

[0169]

[0170] in, Score for consistency of evidence; The authority of the evidence source is scored (e.g., official announcements and reports from mainstream security vendors score higher). The normalized value of the number of pieces of evidence supporting this conclusion; These are empirical weighting coefficients, with default values ​​of 0.5, 0.3, and 0.2.

[0171] (2) Confidence level classification:

[0172] High confidence level (directly entered into the spectrum);

[0173] Medium confidence level (marked as pending review);

[0174] Low confidence level (not to be updated for now).

[0175] Step 4: Reasoning Robustness Test

[0176] (1) Adversarial sample generation:

[0177] Insert 30% adversarial perturbations into the inference test set, including multiple aliases for the same entity, word order scrambling, and noisy text interference;

[0178] Perform inference tasks on the model and record the output.

[0179] (2) Stability assessment indicators:

[0180] Entity Consistency Score (ECS): Whether the system can correctly map entities to a unified node under different aliases and expression variants;

[0181] Inference stability difference (ΔF1): The difference between the F1 values ​​of the original sample and the perturbed sample is calculated. The smaller the value, the higher the robustness.

[0182] Step 5: Verification Results and Performance Comparison

[0183] In a test set containing 1,500 APT intelligence reasoning tasks, the method of this invention performs as follows:

[0184] The proportion of high-confidence inference conclusions reached 84.7%, and the consistency check pass rate reached 91.2%.

[0185] ECS score ≥ 0.92, ΔF1 controlled within 3.1%;

[0186] In adversarial perturbation samples, the F1 score remained above 88.5%, an improvement of approximately 14.6% compared to the version without verification mechanism.

[0187] The results show that the reasoning verification mechanism of the present invention can effectively filter out low-quality reasoning conclusions and significantly improve the credibility and stability of reasoning results.

[0188] The following embodiments provide a comprehensive verification of the inference performance comparison experiment of the present invention;

[0189] To verify the advantages of this invention in network threat behavior reasoning, several APT cases containing complete attack chains were selected as test scenarios, and the method of this invention was compared and analyzed with existing typical large model reasoning schemes.

[0190] The comparison methods include: (1) traditional RAG reasoning methods that rely solely on text retrieval and generation; and (2) threat intelligence knowledge graph reasoning systems based on large model fine-tuning.

[0191] In experiments, traditional RAG methods frequently encountered problems such as contextual fragmentation and difficulty in resolving entity ambiguities during multi-source intelligence fusion and cross-segment reasoning, resulting in incomplete reasoning chains. Especially in tasks requiring traversal of multiple intelligence segments to reconstruct a complete attack path, their accuracy and consistency were significantly insufficient. Existing knowledge graph reasoning systems have shown some improvement with the support of structured knowledge, but due to the lack of multi-granularity entity alignment mechanisms and graph augmentation retrieval capabilities, issues such as missing tactical or technical nodes, logical jumps, and incomplete ATT&CK mappings still occur during the reasoning process.

[0192] In contrast, this invention introduces a graph-enhanced retrieval strategy based on GraphRAG during the inference phase. This strategy deeply integrates the tactical-technical semantic mapping of the ATT&CK framework with the structured information of the knowledge graph. It can automatically eliminate ambiguity and redundancy in multi-source data during inference, accurately identifying and associating key tactics, techniques, and tools in the attack chain. Simultaneously, the multi-granularity knowledge alignment mechanism of this invention effectively integrates synonymous entities and cross-document information, making the inference results superior to the comparative methods in terms of logical consistency, completeness, and interpretability.

[0193] In real-world APT case verification, this invention can completely reconstruct multiple stages of the attack chain (such as the entire process from initial intrusion, persistence, lateral movement, and data theft), and accurately map them to the corresponding ATT&CK tactical and technical nodes. The generated inference chain not only has a wider coverage but also has a clear description of causal relationships and contextual explanations, significantly reducing information omissions and logical jumps in traditional methods.

[0194] Experimental results show that the present invention has stronger multi-hop reasoning capabilities, higher attack chain integrity, and better interpretability in network threat intelligence reasoning tasks, and can provide more accurate and reliable technical support for network security analysis and emergency response.

[0195] In one of the following possible implementations, combining Figure 4 As shown, the process of building an intelligent question-answering model for APT attacks based on a large language model and knowledge graph is demonstrated.

[0196] After a user raises a question about highly covert attacks, the question is first transformed into a vector representation through a vector embedding model. Based on this vector, relevant documents are searched from the knowledge base and knowledge graph. Then, prompt words are constructed using these documents, combined with Top-k knowledge prompts, and input into the fine-tuned model GPT4.0.

[0197] The model generates responses through a dialogue system based on a large language model, and finally outputs highly covert answers or knowledge, which are then fed back to the user to form a closed loop.

[0198] The technical terms, principles, or means related to the technical solutions of the present invention mentioned in the above embodiments, which are not described in detail above, are all well-known technologies or common practices that are known to those skilled in the art.

[0199] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A network threat behavior inference method based on large model retrieval enhancement, characterized in that, This includes the computer system performing the following steps; Receive network threat analysis query requests from users and determine the query target and its attack scenario or APT organization; Based on the query target, relevant entity nodes and their semantic relationships are retrieved from the constructed threat intelligence knowledge graph to generate an initial inference subgraph containing multi-hop association paths; The GraphRAG framework is invoked to vectorize the node context information of the initial inference subgraph and then perform semantic retrieval and matching with an external threat intelligence knowledge base to obtain a supplementary evidence set related to the inference task. The GraphRAG framework first divides the initial inference subgraph into graphs and extracts subgraph information by topic or community structure. Then, it uses node semantic embedding to perform similarity matching with the text description of the external knowledge base. Finally, it retrieves external evidence that is highly relevant to the inference target. The initial reasoning subgraph is merged with the supplementary evidence set to construct a comprehensive prompt template that includes reasoning task instructions, node relationship topology descriptions, and external evidence text. The comprehensive prompt template is input into the large language model that has been lightly tuned by LoRA to generate inference conclusions that conform to the MITRE ATT&CK framework; The newly added nodes and relationships in the reasoning conclusions are added to the threat intelligence knowledge graph, and a structured graph file or a visual reasoning result that supports interactive analysis is output.

2. The network threat behavior inference method based on large model retrieval enhancement according to claim 1, characterized in that, The determination of the query target includes identifying the subject object, reasoning scope and task intent of the query target through semantic parsing, and limiting the boundary conditions through predefined context constraint rules; The generation of the initial inference subgraph containing multi-hop association paths also includes performing relation constraint detection and redundant edge elimination, as well as using a node importance ranking and threshold filtering mechanism to remove nodes with low relevance to the query target; The external threat intelligence knowledge base includes the MITRE ATT&CK tactical technology library, the CVE vulnerability database, the open-source APT intelligence analysis report library, and the STIX 2.1 format standardized threat intelligence dataset.

3. The network threat behavior inference method based on large model retrieval enhancement according to claim 1, characterized in that, The GraphRAG framework performs multi-round retrieval based on node semantic representation and selects the most relevant external evidence for the reasoning task through cosine similarity or vector similarity measures. The step of vector encoding the node context information of the initial inference subgraph includes: calling the GraphRAG framework to partition the initial inference subgraph, and extracting subgraph information in units of topics or community structures; The semantic retrieval and matching with the external threat intelligence knowledge base includes: using node semantic embedding to perform similarity matching with the text description of the external threat intelligence knowledge base.

4. The network threat behavior inference method based on large model retrieval enhancement according to claim 1, characterized in that, The comprehensive prompt template adopts a structured coding method, which organizes the entity nodes, semantic relationships and supplementary evidence sets hierarchically. At the same time, it introduces weight control and constraint rules to place core nodes and high-confidence evidence in prominent positions, while low-confidence information serves as auxiliary background.

5. The network threat behavior inference method based on large model retrieval enhancement according to claim 1, characterized in that, The fine-tuning method for the large language model after LoRA lightweight fine-tuning includes: While freezing the original pre-trained weights, only the low-rank matrix parameters in the attention sublayer and the feedforward network are updated; The LoRA lightweight fine-tuning is as follows: ； Where W is the original frozen weight matrix; A and B are low-rank trainable matrices; The generation of reasoning conclusions conforming to the MITRE ATT&CK framework includes: using the large language model to analyze the causal relationships and potential dependencies between nodes layer by layer through a multi-hop logical deduction mechanism during the reasoning process, and generating reasoning conclusions conforming to the MITRE ATT&CK framework. The reasoning conclusions include newly added attack stage nodes, attack tool nodes and their semantic relationships, and update the edge weights in the original graph to reflect the evolution of the attack chain.

6. The network threat behavior inference method based on large model retrieval enhancement according to claim 5, characterized in that, The newly added nodes and relationships include the newly added attack phase nodes, attack tool nodes, and the semantic relationships between them; When supplementing the threat intelligence knowledge graph, a consistency check and redundancy detection mechanism is used for verification and detection. The output visualization reasoning results are based on a graphical interface that displays the attack chain reconstruction process. The interface supports node aggregation, path expansion, and interactive queries across document links.

7. A network threat behavior inference system based on large model retrieval enhancement, characterized in that, include: The query module is used to receive network threat analysis query requests input by users and determine the query target and related scenarios through semantic parsing; The graph retrieval module connects to the constructed threat intelligence knowledge graph and is used to retrieve relevant nodes and relationships and generate an initial inference subgraph. The retrieval enhancement module is used to call the GraphRAG framework to perform semantic matching between the inference subgraph and an external threat intelligence knowledge base to obtain a supplementary evidence set. The prompt generation module is used to integrate the initial reasoning subgraph with the supplementary evidence set to generate a comprehensive prompt template that includes task instructions, node relationships and external evidence; The large model reasoning module has a built-in large language model that has been lightly tuned by LoRA, which is used to process the comprehensive prompt template and generate reasoning conclusions that conform to the MITRE ATT&CK framework. The graph update and output module is used to update the inference conclusions to the threat intelligence knowledge graph and output a structured graph file or a visualized inference result.

8. The network threat behavior inference system based on large model retrieval enhancement according to claim 7, characterized in that, The query module includes: The semantic parsing unit is used to perform entity recognition, intent judgment, and ambiguity resolution on natural language queries; the keyword recognition unit is used to extract core keywords such as attack organizations, vulnerability numbers, and tool names from the query; and the scene mapping unit is used to associate the parsing results with a preset attack scene tag library.

9. An electronic device, characterized in that, include: At least one processor; The memory is communicatively connected to the processor; The memory stores a computer program that, when executed by the processor, enables the implementation of the network threat behavior reasoning method based on large model retrieval enhancement as described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed, the program implements the network threat behavior inference method based on large model retrieval enhancement as described in any one of claims 1-6.

Images