A method, system, storage medium and program product for automatically constructing a network threat report attack knowledge graph based on a large language model

Abstract

The present invention relates to a method, system, storage medium and program product for automatically constructing a network threat report attack knowledge graph based on a large language model. The method generates an initial attack knowledge graph by iteratively processing the extracted entities and relationships through a clustering method based on a large language model. At the same time, the present invention aggregates multiple technical example threat reports belonging to the same attack technology through an attack technology graph template generation mechanism, establishes a standardized template library for standardized attack technologies, and adopts an attack technology alignment method based on Word2Vec and WordNet to realize automatic attack technology labeling of new threat reports, construct a complete attack technology knowledge graph, facilitate security analysts to quickly understand attack paths and key threat points, and significantly improve the automation and accuracy of threat intelligence analysis.

Description

Technical Field

[0001] The present invention relates to computer network security and threat intelligence analysis technology, and in particular to a method, system, storage medium and program product for automatically constructing a threat report attack knowledge graph based on a large language model. Background Art

[0002] As cyberattacks become increasingly complex and specialized, the collection, analysis, and utilization of threat intelligence have become critical components of cybersecurity defense. Threat reports, as a crucial source of threat intelligence, contain a wealth of information on attack techniques, tactics, and procedures (TTPs), which are crucial for understanding attacker behavior patterns and formulating defense strategies. Furthermore, attack knowledge graphs, as models that formally describe attack paths, can intuitively demonstrate intrusion paths, vulnerability exploits, and potential impacts, supporting defense decision-making. However, most threat reports are written in natural language and lack a unified format and structure. Existing methods that rely on manual extraction of attack knowledge graphs are inefficient. Therefore, how to automatically extract key information from unstructured threat reports and construct attack knowledge graphs has become a pressing challenge in the cybersecurity field.

[0003] To address the above issues, there are mainly the following methods:

[0004] First, Attackg (building a technical knowledge graph from cyber threat intelligence) uses machine learning technology based on natural language processing technology to semi-automate tasks such as entity recognition and coreference resolution. It realizes threat intelligence processing and attack knowledge graph construction through three main steps: the natural language processing-based report parsing module processes cyber threat intelligence reports and extracts attack-related text, while extracting key indicator information through the indicator protection module; the attack entity extraction and attack dependency extraction modules identify entities and their dependencies from the parsing results, generate a preliminary attack knowledge graph, and optimize the attack knowledge graph simplification module.

[0005] Second, CN116094843B discloses a network threat assessment method based on a knowledge graph, including: manually labeling entities and relationships in advanced persistent threat (APT) reports; using BERT to train an entity relationship classification model to automatically extract the connections between entities; completing the construction of a network threat knowledge graph; and introducing a credibility assessment mechanism to use graph structure information and transformer structure to judge the correctness of entity relationships.

[0006] Third, CN118101346B discloses a method for automatically generating attack knowledge graph interaction rules for honey spot deployment, including: using LLM to extract attack entities from the natural language description of CVE vulnerabilities; using Word2Vec to process synonyms and BERT to process phrase extraction; using LLM to automatically generate attack knowledge graph interaction rules; and security experts provide feedback on the generated rules.

[0007] These technologies can achieve structured extraction of cyber threat information and the construction of attack knowledge graphs, which to some extent solve the problem of converting unstructured text into structured knowledge. However, the following problems still exist:

[0008] First, Attackg's core process relies on manually designed regular expression rules and predefined technical templates, resulting in poor adaptability to new attack patterns or complex attack scenarios. CN116094843B manually labels specific types of entity relationships (12 basic relationship types) and relies on a fixed model structure (BERT combined with a specific tag insertion strategy), which lacks adaptability to new attack scenarios or undefined relationship types. Although CN118101346B uses a large language model, it is fine-tuned using a specific CVE library and relies on preset interaction rule templates (such as fixed templates for the conclusion part and the enabling condition part), making it difficult to cope with complex and changing attack technology descriptions.

[0009] Second, Attackg entity recognition and relationship extraction require manual labeling of training data, increasing deployment and maintenance costs; CN116094843B requires manual labeling of 10,000 network threat records to construct a training set; and the interaction rules generated by CN118101346B require manual feedback, deletion, or modification by security experts. Both involve significant manual intervention, limiting process efficiency and making it difficult to cope with large-scale, frequently updated threat data.

[0010] Third, natural language processing technology and machine learning have limitations when parsing the semantic connotations of unstructured professional texts in the security field. They lack a deep understanding of the network security knowledge system and have limited support for coreference resolution and cross-paragraph behavioral associations. For example, traditional NLP models (such as BERT and Word2Vec) lack the ability to deeply understand the causal relationships, temporal dependencies, and domain-specific terminology (such as "APT" and "honeypot deployment") of attack steps when processing complex semantics in the network security field. They find it difficult to accurately parse complex associations between multiple entities (such as CN116094843B, which relies on graph structure information flow calculations and Transformer encoders and decoders, and CN118101346B, which relies on keyword extraction and cosine similarity matching). This leads to insufficient relationship recognition accuracy and attack logic modeling capabilities, and low attack path recognition accuracy.

[0011] Fourth, existing methods lack the ability to standardize and integrate multi-source reports on the same attack technology, and are unable to form a unified technical template for subsequent technical identification and classification, resulting in fragmented understanding of the same attack technology and difficulty in establishing a complete attack technology knowledge base system.

[0012] Fifth, existing technologies lack automated technology alignment and label assignment mechanisms when faced with new threat reports, and are unable to quickly and accurately associate attack behaviors in new reports with known attack techniques, affecting the timeliness and effectiveness of threat intelligence. Summary of the Invention

[0013] In response to the problems existing in the prior art, the purpose of the present invention is to provide a method, system, storage medium and program product for automatically constructing a network threat report attack knowledge graph based on a large language model, so as to improve the accuracy of attack path identification.

[0014] To achieve the above object, the technical solution adopted by the present invention is:

[0015] A method for automatically constructing a network threat report attack knowledge graph based on a large language model, the method comprising the following steps:

[0016] Step 1: Collect unstructured new cyber threat report texts from multiple channels;

[0017] Step 2: Use a large language model to process the new network threat report text, extract the subject-predicate-object report triple set, and generate the initial attack knowledge graph;

[0018] Step 3: Generate a standardized attack technology template library: Collect threat report template texts for multiple attack technologies, where one attack technology corresponds to multiple threat report template texts; aggregate multiple threat reports for the same attack technology to generate a standardized template for the attack technology; and the standardized templates for the multiple attack technologies form a standardized attack technology template library;

[0019] Step 4: Attack technology alignment: Compare the report triples output from the new threat report text in step 2 with the template triples in the standardized template obtained in step 3. Automatically assign technical labels by calculating the matching scores, and then construct the final attack technology knowledge graph based on the report triples containing technical labels.

[0020] The step 2 is specifically as follows:

[0021] Step 2.1 uses the collected new network threat report text as input, uses domain knowledge to design prompt words, and guides the large language model to process the input network threat report text, instructing the large language model to output the detected network attack related entities in a structured format to obtain a network security entity list;

[0022] Step 2.2: The second phase of the large language model is called. Based on the list of cybersecurity entities obtained in step 2.1 and the new cyberthreat report text collected in step 1, the large language model uses contextual understanding to identify causal, temporal, and conditional dependencies between entities, extract the relationships between entities, and output a subject-predicate-object triple attack behavior chain.

[0023] Step 2.3: After extracting the triples, all triples are aggregated into a source graph. Then, all unique entities and edges in the source graph are collected and normalized to reduce redundancy and integrate into a single attack knowledge graph.

[0024] Step 2.4: Utilize the semantic understanding capability of the large language model, use domain knowledge hints, and adopt an iterative clustering method based on the large language model to perform a series of consecutive identical clustering operations on the entity list and relationship list contained in the single attack knowledge graph generated in step 2.3. Generate an initial attack knowledge graph based on the entity cluster list and relationship cluster list obtained by clustering. The initial attack knowledge graph contains report triples.

[0025] Step 2.4 specifically includes the following steps:

[0026] Step 2.4.1: Pass the entity list and relationship list contained in the single attack knowledge graph from step 2.3 to the large language model in context. The large language model then performs single cluster extraction. The default prompt words should take into account synonyms, expression variations of cybersecurity terms in the report, and differences in professional terminology. It should identify semantically equivalent or highly similar items and use them to cluster items that refer to the same entities or express the same relationships, and classify them into the same cluster.

[0027] Step 2.4.2: Use the large language model as a judge to verify the single cluster. The evaluation criteria include similarity of cybersecurity concepts and relevance of attack techniques. If the cluster passes the verification, add it to the entity cluster list or relationship cluster list. Then, remove the clustered entities or relationships from the entity list or relationship list and assign a shared cybersecurity domain label to the clustered entities and relationships.

[0028] Step 2.4.3: Repeat steps 2.4.1-2.4.2 until no new clusters of network security entities and relationships are successfully extracted after n consecutive cycles, where n is the number of iterations required to complete the clustering goal.

[0029] Step 2.4.4: Check the remaining cybersecurity entities and relationships in the entity list and relationship list in batches of size b to assess whether they should be added to the existing attack technique or attack behavior cluster. The judgment criteria include similarity in the attack chain stages and consistency in functional purpose.

[0030] Step 2.4.5: For each new cybersecurity entity or relationship added to the entity cluster or relationship cluster list, use the large language model as a judge to verify again to ensure that the newly added entity or relationship is consistent with the existing cluster in terms of security concepts;

[0031] Step 2.4.6. Repeat steps 2.4.4-2.4.5 until there are no remaining network security entities and relationships to be checked, forming an entity cluster list and a relationship cluster list, and generating an initial attack knowledge graph based on the entity cluster list and the relationship cluster list.

[0032] The step 3 specifically includes:

[0033] Step 3.1, Initialize the aggregator: Create an empty aggregate knowledge base for a certain attack technology, including aggregate entity clusters and aggregate relationship clusters;

[0034] Step 3.2: Process multiple threat report templates: For each threat report template about attack techniques, run the complete extraction process of step 2 to obtain clustered subject-predicate-object triples.

[0035] Step 3.3, cluster merging: Place the triples output from each threat report template text into entity clusters and relationship clusters, and then merge them into the aggregated knowledge base to achieve cross-report knowledge integration;

[0036] Step 3.4, Entity Cluster and Relationship Cluster Count: During the merging process, keep track of the total number of times each entity cluster and relationship cluster appears in all threat report template texts;

[0037] Step 3.5, triple normalization: traverse each triple, find the clusters where the subject and object are located in the aggregate entity cluster and obtain the canonical labels, find the cluster where the predicate is located in the aggregate relationship cluster and obtain its canonical label;

[0038] Step 3.6, Normalized triple count: Count the frequency of occurrence of all normalized triples output by the report;

[0039] Step 3.7, Final template generation: Fill the aggregation and counting results into the customized JSON template format to form a complete standardized template. All standardized templates form a standardized template library of attack techniques.

[0040] The step 4 is specifically as follows:

[0041] Step 4.1. Execute the alignment algorithm: the outer loop traverses each standardized template in the attack technology standardized template library, and the inner loop traverses each template triple to be processed;

[0042] Step 4.2, calculate the matching score: use Word2Vec to calculate entity similarity, use WordNet to calculate relationship similarity, find the matching item in the template triple list of the current standardized template, and calculate the single report triple alignment score;

[0043] Step 4.3, score accumulation and normalization: Accumulate the scores for each matching report triple and finally perform normalization to obtain the final alignment score;

[0044] Step 4.4, technical label assignment: set the confidence threshold and assign the corresponding technical label to the report triples whose final alignment score is higher than the confidence threshold;

[0045] Step 4.5: Generate attack technology knowledge graph: Construct the final attack technology knowledge graph based on the report triples containing technology labels.

[0046] In step 4.2, for the report triplet Tr and the template triplet Tt, the similarity calculation formula is defined as:

[0047]

[0048]

[0049] in, 、 、 is the weight coefficient, represents the entity similarity function, represents the relationship similarity function, Represents the subject entity extracted from the threat report that needs to be processed. Represents the subject entity extracted from the template library, Represents the predicate relationship extracted from the threat report that needs to be processed, represents the predicate relation extracted from the template library, Represents the object entity extracted from the threat report that needs to be processed, Represents the object entity extracted from the template library;

[0050] The single report triplet alignment score is defined as:

[0051] Among them, occurrence(Tt) represents the frequency of occurrence of the template triple Tt in the template, and Template represents the attack technology standardized template library formed by collecting various attack technology threat report template texts from MITRE.

[0052] In step 4.3, the total template alignment score is defined as:

[0053]

[0054] The final alignment score is defined as follows:

[0055] .

[0056] A system for automatically constructing a network threat report attack knowledge graph based on a large language model includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method for automatically constructing a network threat report attack knowledge graph based on a large language model as described above.

[0057] A computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the method for automatically constructing a network threat report attack knowledge graph based on a large language model as described above.

[0058] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the method for automatically constructing a network threat report attack knowledge graph based on a large language model as described above.

[0059] After adopting the above scheme, the present invention uses a two-stage method to first extract entities and then identify relationships in combination with the original report. This can effectively filter out invalid text, improve the pertinence and accuracy of the extraction process, reduce noise interference, and enable the relationship identification stage to focus on the association between verified valid entities, thereby greatly improving the accuracy of attack path identification.

[0060] This paper uses a large language model-based clustering method to iteratively process extracted entities and relationships. This effectively addresses the coreference problem in cybersecurity reports, automatically identifying and merging semantically identical but distinct entities and their referents within threat reports. This reduces redundancy, improves the simplicity and readability of the attack knowledge graph, and facilitates security analysts' rapid understanding of attack paths and key threat points.

[0061] This invention leverages the large language model's advantage of pre-training on massive open knowledge data to achieve powerful contextual understanding and knowledge reasoning of network threat reports, improving the generalization capability of attack knowledge graph construction. Based on the large language model's instruction following and contextual learning capabilities, it achieves automated extraction of attack knowledge graphs and reduces manual intervention costs. It also applies the large language model's semantic understanding capabilities to accurately identify and parse attack steps, techniques, and relationships in threat reports, addressing the limited semantic understanding capabilities of traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a flow chart of the method of the present invention;

[0063] Figure 2 This is a prompt reference diagram of the present invention;

[0064] Figure 3 The JSON format of the technical template;

[0065] Figure 4 Schematic diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0066] like Figures 1-4 As shown, the present invention discloses a method for automatically constructing a network threat report attack knowledge graph based on a large language model, which includes the following steps:

[0067] Step 1: Collect unstructured new network threat report texts from multiple channels to provide the original data source.

[0068] Collection channels generally include existing technical reports, white papers, blogs, and news reports, as well as specialized websites such as the Cisco Intelligence Group and the Microsoft Security Intelligence Center. Cyber ​​threat reports contain descriptions of attacker behavior, vulnerability exploitation details, and attack targets.

[0069] Step 2: Use a large language model to process the new network threat report text, extract the subject-predicate-object report triple set, and generate the initial attack knowledge graph.

[0070] The step 2 specifically includes the following steps:

[0071] Step 2.1 uses the collected new network threat report text as input, uses domain knowledge to design prompt words, and guides the large language model to process the input network threat report text, instructing the large language model to output the detected network attack-related entities in a structured format to obtain a list of network security entities.

[0072] Entity extraction involves extracting the subject and object of an attack from a natural language description, outputting the results in a list format. For example, if "A executed B," the output would be "Entities:{"A","B"}."

[0073] Extract all entities related to attack activity from the given cyber threat report text and return a list of entities. Note: 1. Do not output any redundant explanations or groupings; return only a flat list of entity strings. 2. Do not omit any technical indicators, abstract roles, or attack phase descriptions. 3. For references, try to restore the exact entity name within the context. If in doubt, retain the original text.

[0074] The extracted entity types include at least the following:

[0075] 1. Threat actor / attack group (e.g., APT group name);

[0076] 2. Malware / tools (e.g., Trojan horse, ransomware name);

[0077] 3. Vulnerabilities (e.g., CVE numbers, zero-day vulnerabilities);

[0078] 4. Attack techniques (such as phishing, watering hole attacks, and Distributed Denial of Service (DDoS) attacks);

[0079] 5. Target of victim (e.g., industry, organization, country);

[0080] 6. Assets (such as servers, databases, and network equipment);

[0081] 7. Attack phase (e.g., initial access, privilege escalation, data theft);

[0082] 8. Indicators of Compromise (IoC) (e.g., IP addresses, domain names, hash values);

[0083] 9. Defensive measures (e.g., patches, firewall rules).

[0084] Step 2.2: Call the large language model in the second phase. Based on the list of network security entities obtained in step 2.1 and the new network threat report text collected in step 1, the large language model identifies the causal, temporal, and conditional dependencies between entities based on contextual understanding, extracts the relationships between entities, and outputs a subject-predicate-object triple attack behavior chain.

[0085] When extracting relationships between entities, keep in mind the following: 1. Only output relationships related to the known entity list. If the subject or object is not in the known entity list, the triple is not output. 2. For nested or multi-action statements, split them into multiple independent triples to preserve the complete attack chain logic. 3. Do not output duplicate triples with the same meaning unless the text is indeed repeated with slightly different meanings.

[0086] When extracting relationships between entities, the predicate should be concise (1-3 words) and clearly define the relationship between the subject and the entity. Relationship types include at least the following:

[0087] 1. Usage relationships: "use," "deploy," "exploit" (e.g., APT groups use specific malware);

[0088] 2. Attack relationships: “attack,” “invasion,” “targeting” (e.g., malware attacks a specific asset);

[0089] 3. Exploitation relationship: "Exploit", "Trigger" (e.g., attacker exploits a specific vulnerability);

[0090] 4. Temporal relations: “before,” “after,” “lead to” (the order between attack steps);

[0091] 5. Belong to relationship: "belongs to", "comes from" (e.g. malware belongs to a specific APT group);

[0092] 6. Purpose relationship: "Purpose is", "aims to" (e.g., the attack is aimed at data theft);

[0093] 7. Defense relationships: “defense,” “mitigation,” and “detection” (e.g., patches fix specific vulnerabilities).

[0094] Step 2.3: After extracting the triples, all triples are formed into an aggregated source graph. Then, all unique entities and edges in the aggregated source graph are collected and normalized to reduce redundancy and integrate into a single attack knowledge graph.

[0095] Step 2.4: Utilize the semantic understanding capability of the large language model, use domain knowledge hints, and adopt an iterative clustering method based on the large language model to perform a series of consecutive identical clustering operations on the entity list and relationship list contained in the single attack knowledge graph generated in step 2.3. Generate the initial attack knowledge graph based on the entity cluster list and relationship cluster list obtained by clustering.

[0096] It should be noted that the clustering processing of the entity list and the clustering processing of the relationship list are performed independently.

[0097] Step 2.4 specifically includes the following steps:

[0098] Step 2.4.1: Pass the entity list and relationship list contained in the single attack knowledge graph in step 2.3 to the large language model in context, so that the large language model can perform single cluster extraction. The default prompt words need to consider synonyms and expression variants of cybersecurity terms in the report and differences in professional terminology. Identify semantically equivalent or highly similar items, use them to cluster entities that refer to the same entity or express the same relationship, and classify them into the same cluster.

[0099] Step 2.4.2: Use the large language model as a judge to validate the individual clusters. Criteria include similarity of cybersecurity concepts and relevance of attack techniques. If the cluster passes validation, add it to the entity or relationship cluster list. Then, remove the clustered entities or relationships from the entity or relationship list and assign them a shared cybersecurity domain label.

[0100] Step 2.4.3: Repeat steps 2.4.1-2.4.2 until no new clusters of network security entities and relationships are successfully extracted after n consecutive cycles. n is the number of iterations required to complete the clustering goal, which depends on the complexity of the network threat report to be processed. In most cases, it is 6-10 times.

[0101] Step 2.4.4: Check the remaining cybersecurity entities and relationships in the entity list and relationship list in batches of size b to evaluate whether they should be added to the existing attack technology or attack behavior cluster. The judgment criteria include similarity of attack chain stages and consistency of functional purpose.

[0102] Step 2.4.5: For each new cybersecurity entity and relationship added to the entity clustering or relationship clustering list, use the large language model as a judge to verify again to ensure that the newly added entities and relationships are consistent with the existing clustering in terms of security concepts.

[0103] Step 2.4.6. Repeat steps 2.4.4-2.4.5 until there are no remaining network security entities and relationships to be checked, forming an entity cluster list and a relationship cluster list, and generating an initial attack knowledge graph based on the entity cluster list and the relationship cluster list. The initial attack knowledge graph contains the report triples.

[0104] Step 3. Generate a standardized template library for attack technologies: Collect threat report template texts for multiple attack technologies from MITRE, where one attack technology corresponds to multiple threat report template texts. Aggregate multiple threat reports for the same attack technology to generate a standardized template for the attack technology. The standardized templates for multiple attack technologies form a standardized template library for attack technologies.

[0105] Specifically include:

[0106] Step 3.1. Initialize the aggregator: Create an empty aggregation knowledge base for a certain attack technology, including agg_entity_clusters (aggregate entity clusters) and agg_edge_clusters (aggregate relationship clusters).

[0107] Step 3.2: Process multiple threat report template texts: For each threat report template text collected from MITRE about the attack technology, run the complete extraction process of step 2 to obtain clustered subject-predicate-object triples.

[0108] Step 3.3, Merge Clusters: Place the triples output from each threat report template text into the entity cluster entity_clusters and the relationship cluster edge_clusters and merge them into the aggregate knowledge base to achieve cross-report knowledge integration.

[0109] Step 3.4, Entity cluster and relationship cluster count: During the merging process, keep tracking the total number of times each entity cluster and relationship cluster appears in all threat report template texts, recorded as instance_count.

[0110] Step 3.5, triple normalization: Traverse each triple (s, r, o), find the clusters where s and o are located in the aggregate entity cluster agg_entity_clusters and obtain the canonical labels, and find the cluster where r is located in the aggregate relationship cluster agg_edge_clusters and obtain its canonical label.

[0111] Step 3.6, normalized triple count: Count the occurrence frequency of all normalized triples produced by the report, recorded as occurrence_count.

[0112] Step 3.7, Final template generation: Fill the aggregation and counting results into the custom JSON template format (such as Figure 3 As shown in the figure), a complete standardized template is formed, and all standardized templates form an attack technology standardized template library.

[0113] Step 4: Attack technology alignment: Compare the report triples output from the new threat report text in step 2 with the template triples in the standardized template obtained in step 3, and automatically assign technical labels by calculating the matching score. This includes:

[0114] Step 4.1, execute the alignment algorithm: the outer loop traverses each standardized template in the attack technology standardized template library, and the inner loop traverses each template triple to be processed.

[0115] Step 4.2. Calculate the matching score: Use Word2Vec to calculate entity similarity and WordNet to calculate relationship similarity, and find the matching item in the template triple list of the current standardized template.

[0116] For the report triplet Tr and the template triplet Tt, the similarity calculation formula is defined as:

[0117]

[0118]

[0119] in, 、 、 is the weight coefficient, represents the entity similarity function, Represents the relationship similarity function, calculated based on Word2Vec and WordNet respectively, Represents the subject entity extracted from the threat report that needs to be processed. Represents the subject entity extracted from the template library, Represents the predicate relationship extracted from the threat report that needs to be processed, represents the predicate relation extracted from the template library, Represents the object entity extracted from the threat report that needs to be processed, Represents the object entity extracted from the template library.

[0120] The single report triplet alignment score is defined as:

[0121] Among them, occurrence(Tt) represents the frequency of occurrence of the template triple Tt in the template, and Template represents the attack technology standardized template library formed by collecting various attack technology threat report template texts from MITRE.

[0122] In the above formula, S represents subject, r represents relation, O represents object, subscript r represents report, and subscript t represents template.

[0123] Step 4.3, score accumulation and normalization: Accumulate the scores for each matching report triple, and finally perform normalization to obtain the final alignment score.

[0124] The total template alignment score is defined as:

[0125]

[0126] The final normalized alignment score is defined as follows:

[0127]

[0128] Step 4.4, technical label assignment: set the confidence threshold and normalize the alignment score Report triplets above the confidence threshold are assigned corresponding technical labels.

[0129] Step 4.5: Generate attack technology knowledge graph: Construct the final attack technology knowledge graph based on the report triples containing technology labels.

[0130] It should be noted that, in the present invention, report triples refer to subject-predicate-object triples extracted from network threat reports that describe attack behaviors. For example, a simple triple might be: "The attacker (subject) uses (predicate) malware (object)." Template triples refer to triples extracted from standardized attack technology templates. These templates are usually aggregated from multiple threat reports and describe the common features and behavior patterns of the same attack technology. For example, a certain attack technology may have multiple threat report templates, and the triples of all these templates are called template triples. A triple generally refers to a basic data unit that describes the relationship between entities in a knowledge graph. It consists of a subject, a predicate, and an object, and is used to represent the specific relationship between entities.

[0131] Figure 4 This is a final implementation example. The first layer of unstructured text comes from the cyber threat report to be processed. The nodes and arrows in the second and third layers represent the report triples extracted in step 2. The bottom layer contains the attack technique labels corresponding to the template triples after executing step 3. The technical labels in the middle two layers demonstrate the technical alignment achieved with the knowledge graph after executing step 4—presenting the entire attack landscape and the corresponding MITRE techniques for each attack step.

[0132] This invention leverages the large language model's advantage of pre-training on massive open knowledge data to achieve powerful contextual understanding and knowledge reasoning of network threat reports, improving the generalization capability of attack knowledge graph construction. Based on the large language model's instruction following and contextual learning capabilities, it achieves automated extraction of attack knowledge graphs and reduces manual intervention costs. It also applies the large language model's semantic understanding capabilities to accurately identify and parse attack steps, techniques, and relationships in threat reports, addressing the limited semantic understanding capabilities of traditional methods.

[0133] This method uses domain knowledge (entity / relationship categories) to design prompts to guide large language models in understanding specific cybersecurity concepts. By incorporating cybersecurity domain knowledge as prompts, the accuracy of entity and relationship extraction is significantly improved, enabling large language models to accurately identify specialized terminology and technical concepts, reducing misidentifications and missed identifications, and improving the quality and reliability of attack knowledge graph construction.

[0134] This invention uses a two-stage approach to first extract entities and then identify relationships based on the original report. This effectively filters out invalid text, improving the pertinence and accuracy of the extraction process, reducing noise interference, and allowing the relationship identification phase to focus on the connections between verified valid entities, significantly improving the accuracy of attack path identification.

[0135] This paper uses a large language model-based clustering method to iteratively process extracted entities and relationships. This effectively addresses the coreference problem in cybersecurity reports, automatically identifying and merging semantically identical but distinct entities and their referents within threat reports. This reduces redundancy, improves the simplicity and readability of the attack knowledge graph, and facilitates security analysts' rapid understanding of attack paths and key threat points.

[0136] The present invention realizes the standardized integration of multi-source reports on the same attack technology through a technology template generation mechanism, forms a unified technology knowledge base, and improves the integrity and consistency of attack technology understanding.

[0137] This invention adopts an automated technology alignment method, which can quickly and accurately assign technical tags to new threat reports, significantly improving the timeliness and accuracy of threat intelligence processing and reducing manual analysis costs.

[0138] Through weighted similarity calculation and frequency weighting mechanism, the present invention can accurately quantify the matching degree between new reports and known attack techniques, ensure fair comparison between templates of different sizes through normalization processing, and significantly improve the accuracy and reliability of technology alignment.

[0139] In scenarios such as facing new attack technologies, requiring a deep understanding of complex attack narratives, pursuing highly automated security analysis, needing to quickly respond to new threats, and dealing with semantically complex or ambiguous security descriptions, this invention uses large language models and domain knowledge prompts to extract entities and relationships, which is suitable for processing unstructured threat reports. It can effectively make up for the limitations of traditional rule and template methods in generalization ability, degree of automation, and semantic understanding, and provide security analysts with more efficient threat understanding and assessment tools.

[0140] The present invention also discloses a system for automatically constructing a network threat report attack knowledge graph based on a large language model, which includes a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the method in the above embodiment.

[0141] The present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device.

[0142] The computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments.

[0143] The above description is merely an embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any minor modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for automatically constructing a network threat report attack knowledge graph based on a large language model, characterized in that: The method comprises the following steps: Step 1: Collect unstructured new cyber threat report texts from multiple channels; Step 2: Use a large language model to process the new network threat report text, extract the subject-predicate-object report triple set, and generate the initial attack knowledge graph; Step 3: Generate a standardized attack technology template library: Collect threat report template texts of multiple attack technologies, where one attack technology corresponds to multiple threat report template texts; aggregate the threat report template texts for the same attack technology to generate a standardized template for the attack technology; the standardized templates of the multiple attack technologies form a standardized attack technology template library; Step 4: Attack technology alignment: Compare the report triples output from the new threat report text in step 2 with the template triples in the standardized template obtained in step 3. Automatically assign technical labels by calculating the matching score. Then, construct the final attack technology knowledge graph based on the report triples containing the technical labels. The step 3 specifically includes: Step 3.1, Initialize the aggregator: Create an empty aggregate knowledge base for a certain attack technology, including aggregate entity clusters and aggregate relationship clusters; Step 3.2: Process multiple threat report templates: For each threat report template about attack techniques, run the complete extraction process of step 2 to obtain clustered subject-predicate-object triples. Step 3.3, cluster merging: Place the triples output from each threat report template text into entity clusters and relationship clusters, and then merge them into the aggregated knowledge base to achieve cross-report knowledge integration; Step 3.4, Entity Cluster and Relationship Cluster Count: During the merging process, keep track of the total number of times each entity cluster and relationship cluster appears in all threat report template texts; Step 3.5, triple normalization: traverse each triple, find the clusters where the subject and object are located in the aggregate entity cluster and obtain the canonical labels, find the cluster where the predicate is located in the aggregate relationship cluster and obtain its canonical label; Step 3.6, Normalized triple count: Count the frequency of occurrence of normalized triples generated by all threat report template texts; Step 3.7, Final template generation: Fill the aggregation and counting results into the customized JSON template format to form a complete standardized template. All standardized templates form a standardized template library of attack techniques.

2. The method for automatically constructing a network threat report attack knowledge graph based on a large language model according to claim 1 is characterized in that: The step 2 is specifically as follows: Step 2.1 uses the collected new network threat report text as input, uses domain knowledge to design prompt words, and guides the large language model to process the input network threat report text, instructing the large language model to output the detected network attack related entities in a structured format to obtain a network security entity list; Step 2.2: The second phase of the large language model is called. Based on the list of cybersecurity entities obtained in step 2.1 and the new cyberthreat report text collected in step 1, the large language model uses contextual understanding to identify causal, temporal, and conditional dependencies between entities, extract the relationships between entities, and output a subject-predicate-object triple attack behavior chain. Step 2.3: After extracting the triples, all triples are formed into an aggregated source graph. Then, all unique entities and edges in the aggregated source graph are collected and normalized to reduce redundancy and integrate into a single attack knowledge graph. Step 2.4: Utilize the semantic understanding capability of the large language model, use domain knowledge hints, and adopt an iterative clustering method based on the large language model to perform a series of consecutive identical clustering operations on the entity list and relationship list contained in the single attack knowledge graph generated in step 2.

3. Generate an initial attack knowledge graph based on the entity cluster list and relationship cluster list obtained by clustering. The initial attack knowledge graph contains report triples.

3. The method for automatically constructing a network threat report attack knowledge graph based on a large language model according to claim 2 is characterized in that: Step 2.4 specifically includes the following steps: Step 2.4.1: Pass the entity list and relationship list contained in the single attack knowledge graph from step 2.3 to the large language model in context. The large language model then performs single cluster extraction. The default prompt words should take into account synonyms, expression variations of cybersecurity terms in the report, and differences in professional terminology. It should identify semantically equivalent or highly similar items and use them to cluster items that refer to the same entities or express the same relationships, and classify them into the same cluster. Step 2.4.2: Use the large language model as a judge to verify the single cluster. The evaluation criteria include similarity of cybersecurity concepts and relevance of attack techniques. If the cluster passes the verification, add it to the entity cluster list or relationship cluster list. Then, remove the clustered entities or relationships from the entity list or relationship list and assign a cybersecurity domain label with shared meaning to the clustered entities or relationships. Step 2.4.3: Repeat steps 2.4.1-2.4.2 until no new clusters of network security entities and relationships are successfully extracted after n consecutive cycles, where n is the number of iterations required to complete the clustering goal. Step 2.4.4: Check the remaining cybersecurity entities and relationships in the entity list and relationship list in batches of size b to assess whether they should be added to the existing attack technique or attack behavior cluster. The judgment criteria include similarity in the attack chain stages and consistency in functional purpose. Step 2.4.5: For each new cybersecurity entity or relationship added to the entity cluster or relationship cluster list, use the large language model as a judge to verify again to ensure that the newly added entity or relationship is consistent with the existing cluster in terms of security concepts; Step 2.4.

6. Repeat steps 2.4.4-2.4.5 until there are no remaining network security entities and relationships to be checked, forming an entity cluster list and a relationship cluster list, and generating an initial attack knowledge graph based on the entity cluster list and the relationship cluster list.

4. The method for automatically constructing a network threat report attack knowledge graph based on a large language model according to claim 1 is characterized in that: The step 4 is specifically as follows: Step 4.

1. Execute the alignment algorithm: the outer loop traverses each standardized template in the attack technology standardized template library, and the inner loop traverses each template triple to be processed; Step 4.2, calculate the matching score: use Word2Vec to calculate entity similarity, use WordNet to calculate relationship similarity, find the matching item in the template triple list of the current standardized template, and calculate the single report triple alignment score; Step 4.3, score accumulation and normalization: Accumulate the scores for each matching report triple and finally perform normalization to obtain the final alignment score; Step 4.4, technical label assignment: set the confidence threshold and assign the corresponding technical label to the report triples whose final alignment score is higher than the confidence threshold; Step 4.5: Generate attack technology knowledge graph: Construct the final attack technology knowledge graph based on the report triples containing technology labels.

5. The method for automatically constructing a network threat report attack knowledge graph based on a large language model according to claim 4 is characterized in that: In step 4.2, for the report triplet Tr and the template triplet Tt, the similarity calculation formula is defined as: in, 、 、 is the weight coefficient, represents the entity similarity function, represents the relationship similarity function, Represents the subject entity extracted from the threat report that needs to be processed. Represents the subject entity extracted from the template library, Represents the predicate relationship extracted from the threat report that needs to be processed, represents the predicate relation extracted from the template library, Represents the object entity extracted from the threat report that needs to be processed, Represents the object entity extracted from the template library; The single report triplet alignment score is defined as: Among them, occurrence(Tt) represents the frequency of occurrence of template triple Tt in the template, and Template represents the standardized template library of attack technology.

6. The method for automatically constructing a network threat report attack knowledge graph based on a large language model according to claim 5 is characterized in that: In step 4.3, the total template alignment score is defined as: The final alignment score is defined as follows: 。 7. A network threat report attack knowledge graph automatic construction system based on a large language model, characterized by: It includes a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of a method for automatically constructing a network threat report attack knowledge graph based on a large language model as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of a method for automatically constructing a network threat report attack knowledge graph based on a large language model as described in any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of a method for automatically constructing a network threat report attack knowledge graph based on a large language model as described in any one of claims 1 to 6 are implemented.

Images