APT intelligence triple extraction method and system based on multiple units of large language model

Abstract

The invention discloses an APT (Advanced Persistent Threat) information triple extraction method and system based on multiple units of a large language model. The method comprises the following steps: receiving original text data containing advanced persistent threat information and preprocessing the original text data to obtain standardized text data; inputting the data into a multi-unit cooperation framework containing entity identification, relation identification and arbitration units; the entity recognition unit extracts entities to generate a preliminary entity set, and the relation recognition unit extracts a relation to generate a preliminary relation set based on the set; and the arbitration unit performs consistency verification and conflict resolution on the two data sets, and generates and outputs a final triple data set for threat intelligence analysis or knowledge graph construction. The system correspondingly comprises a data receiving and preprocessing module, a multi-agent module, an entity recognition module, a relation recognition module, an arbitration integration module and a result output module, and APT information triples can be efficiently extracted.

Description

Technical Field

[0001] This invention relates to the field of industrial internet security technology, specifically providing a method and system for extracting APT intelligence triples based on a large language model with multiple units. Background Technology

[0002] Currently, techniques for extracting triples from APT threat intelligence texts mainly fall into two categories: traditional Natural Language Processing (NLP) driven and Large Language Model (LLM) assisted systems. Traditional NLP driven systems are centered on rule templates and basic models, identifying threat entities and their relationships through models trained with pre-set keyword libraries, regular expressions, or labeled data. However, CTI texts contain numerous technical terms and complex attack logic, making it difficult for rules to cover all scenarios. Furthermore, traditional methods process each sentence individually, lacking global contextual understanding and failing to address the fragmentation and association of scattered information. LLM assisted systems rely on their semantic understanding capabilities to extract triples. Some solutions optimize prompts or verification formats, but they suffer from context length limitations, making it difficult to handle long CTI reports. Simple segmentation easily breaks down attack logic, and general models lack CTI domain knowledge, leading to misjudgments. Existing processes are mostly linear structures, lacking collaborative optimization and error correction mechanisms, resulting in high illusion rates and poor consistency in the extracted triples. Summary of the Invention

[0003] To address the technical problems in existing technologies for extracting triples from APT threat intelligence (CTI) text, such as the inability of traditional methods to cover complex scenarios, lack of global context understanding and referential resolution capabilities, and the tendency of large language models to be misjudged due to the limitation of context length and lack of CTI domain knowledge, this invention provides a method and system for extracting APT intelligence triples based on a multi-unit large language model.

[0004] This invention provides a method for extracting APT intelligence triples based on a large language model with multiple units, the method comprising: The network threat intelligence text is obtained from an open-source intelligence platform, and the network threat intelligence text is preprocessed to obtain several text fragments; Deploy a group of working agents, which process the text fragments, extract an initial set of triples from the text fragments, and construct a short-term memory. The initial set of triples is processed sequentially by the integration agent, the refinement agent, and the verification agent. The integration agent performs semantic integration and deduplication on the initial set of triples. The refinement agent performs format correction, referential resolution, and compound relation splitting on the integrated triples. The verification agent verifies the correctness of the refined triples. If the proportion of erroneous triples exceeds a preset threshold, the working agent is triggered to perform a trial extraction. The working agent is invoked to construct long-term memory based on the verified triples. The working agent unifies entity representation, associates relationships across text segments, and performs secondary verification in conjunction with the verification agent to output a complete set of triples. Furthermore, the preprocessing step for the network threat intelligence text includes: removing non-textual information from the network threat intelligence text; performing data cleaning and normalization on the removed text; and splitting the normalized text into several text segments according to preset logic or character length rules. Furthermore, the working agent is built based on the same large language model, and its prompts contain an example set covering multiple network threat scenarios. The working agent is configured to extract only network threat intelligence domain-specific entities and output triples conforming to a preset format. Furthermore, the refined intelligent agent performs format correction, referential resolution, and compound relation decomposition on the integrated triples, including: At least one entity pair is extracted from the initial triple set; a context text fragment containing the entity pair is input into a preset large language model; the relationship type between the entity pairs is determined based on the output of the large language model, thereby forming the initial triple preliminary relationship set.

[0005] Furthermore, the entity pair refers to a pair consisting of two core information units (entities) extracted from the text that may have a semantic relationship. Furthermore, the verification agent performs correctness checks on the refined triples, including: Detect whether there are inconsistencies between the entity references in the preliminary triplet set and the entity references in the preliminary relation set; When an inconsistency is detected, the correct entity and / or relationship is determined by re-analysis based on the contextual semantic information of the standardized text data. Eliminate logical conflicts between the output results and integrate them to obtain the final set of triples.

[0006] Furthermore, it also includes: generating a set of triplet data and storing the set of triplet data in a non-volatile storage medium. Furthermore, the method also includes: formatting the triplet data set before outputting it to conform to a preset data exchange standard. Furthermore, it also includes: fine-tuning the parameters of the large language model components based on feedback information from the triplet dataset.

[0007] This invention also provides an APT intelligence triple extraction system based on a large language model with multiple units, the system comprising: The text preprocessing module is used to obtain unstructured network threat intelligence text from an open-source intelligence platform and preprocess the network threat intelligence text to obtain multiple text fragments. The short-term memory construction module includes a set of working agents that process each text segment in parallel and extract an initial set of triples to construct short-term memory. The triplet processing module includes an integration agent, a refinement agent, and a verification agent connected in sequence. The integration agent performs semantic integration and deduplication on the initial triplet set, the refinement agent performs format correction, referential resolution, and compound relation splitting, and the verification agent performs correctness verification and triggers a retrial of the working agent when the error rate exceeds the standard. The memory construction output module includes a merging agent, which unifies entity representations, associates relationships across text fragments, and performs secondary verification in conjunction with a validation agent, ultimately outputting a complete set of triples. Furthermore, the system also includes: The model optimization module is used to fine-tune the parameters of the large language model component in the multi-unit collaboration module based on the feedback information from the final triplet data set.

[0008] Beneficial effects This invention provides a method and system for extracting APT intelligence triples based on a large language model with multiple units. This method effectively improves the accuracy and reliability of APT intelligence triple extraction. Through a multi-unit collaborative framework, the entity recognition unit and the relationship recognition unit have clearly defined roles, accurately extracting entities and relationships respectively, reducing the bias of single-unit processing. The consistency verification and conflict resolution steps of the arbitration unit eliminate inconsistencies and logical conflicts in the outputs of different units, avoiding the retention of erroneous information and ensuring the correctness of the final triple dataset. Simultaneously, the preprocessing step cleans, structures, and vectorizes the original text data, providing a high-quality data foundation for subsequent extraction and improving extraction efficiency. The modular design of the system makes the functions of each module clear, facilitating maintenance and expansion. Furthermore, the parameters of the large language model components can be fine-tuned based on feedback information to continuously optimize extraction performance. The generated triple dataset effectively supports threat intelligence analysis and knowledge graph construction, providing strong structured knowledge support for network security protection, adapting to diverse CTI processing scenarios, and meeting practical application needs. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating the working steps of the APT intelligence triple extraction method based on a large language model with multiple units provided in this embodiment of the invention. Figure 2 A schematic diagram of a framework for multi-agent collaborative work provided in another embodiment of the present invention; Figure 3 A schematic diagram of an APT intelligence triple extraction system module based on a large language model multi-unit provided in another embodiment of the present invention; Figure 4 The diagram shows the structure of an APT intelligence triple extraction device based on a large language model multi-unit, which is provided in another embodiment of the present invention.

[0011] Figure labels: 1. Text preprocessing module; 2. Short-term memory construction module; 3. Triple processing module; 4. Memory construction output module. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0013] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the use of "and / or" or "and / or" throughout the text implies three parallel solutions. For example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied. Furthermore, the technical solutions of the various embodiments can be combined, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0014] In existing technologies, the field of cyber threat intelligence analysis has long relied on traditional natural language processing techniques to extract threat entities and their relationships through preset rule templates or basic models. Because cyber threat intelligence texts contain a large amount of technical terminology and complex attack logic, traditional methods struggle to cover all scenarios, and sentence-by-sentence processing lacks global contextual understanding, leading to difficulties in dereference resolution and missing cross-textual information connections. With the development of large language models, some solutions attempt to utilize their semantic understanding capabilities for triple extraction; however, limited by context length, they struggle to effectively handle long threat reports. Simple segmentation easily fragments the attack logic chain, general models lack domain knowledge and are prone to misjudgments, and existing linear processes lack collaborative optimization mechanisms, resulting in high illusion rates and poor consistency in the extraction results.

[0015] To address the aforementioned issues, existing technologies cannot effectively balance the conflict between processing efficiency and information integrity, necessitating the exploration of new methods that can maintain contextual coherence while accurately extracting proprietary entities. Analysis reveals that traditional methods suffer from semantic coverage blind spots in the entity recognition stage, and single-model processing cannot achieve error self-correction. This leads to the concept of introducing multi-agent collaboration into triple extraction. By constructing a multi-agent system with domain knowledge, text fragments are processed in stages, and a memory mechanism is established, achieving cross-fragment information association while ensuring parallel efficiency.

[0016] refer to Figures 1-2 This embodiment provides a method for extracting APT intelligence triples based on a large language model with multiple units. The method includes: S110. Obtain unstructured network threat intelligence text from an open-source intelligence platform, and preprocess the network threat intelligence text to obtain several text fragments. S120. Deploy a group of working agents, which process each text segment in parallel, extract an initial set of triples from each text segment, and construct short-term memory. S130. The integration agent, refinement agent, and verification agent are sequentially invoked to process the initial set of triples. The integration agent performs semantic integration and deduplication on the initial set of triples. The refinement agent performs format correction, referential resolution, and compound relation splitting on the integrated triples. The verification agent performs correctness verification on the refined triples. If the proportion of erroneous triples exceeds a preset threshold, the working agent is triggered to perform a trial extraction. S140. Invoke the merging agent to build long-term memory based on the verified triples. The merging agent unifies entity representation, associates relationships across text segments, and performs secondary verification in conjunction with the verification agent. Finally, it outputs the complete set of triples corresponding to a single network threat intelligence text.

[0017] Preferably, the step of preprocessing the network threat intelligence text includes: removing non-text information from the network threat intelligence text, performing data cleaning and standardization on the removed text, and splitting the standardized text into several text segments according to preset logic or character length rules.

[0018] It should be noted that, in this embodiment, the specific technical solution includes the following steps: obtaining unstructured network threat intelligence (CTI) text from an open-source intelligence platform; preprocessing the CTI text to obtain multiple text fragments; deploying a group of worker agents (hereinafter referred to as Worker agents) and an integrator agent (hereinafter referred to as Integrator agent); the worker agents process each text fragment in parallel, extracting an initial set of triples from each text fragment to construct short-term memory; sequentially calling the refiner agent (hereinafter referred to as Refiner agent) and the checker agent (hereinafter referred to as Checker agent) to process the initial set of triples. The Integrator agent performs semantic integration and deduplication on the initial set of triples. The Refiner agent performs format correction, referential resolution, and compound relation splitting on the integrated triples. The Checker agent verifies the correctness of the refined triples. If the proportion of incorrect triples exceeds a preset threshold, the Worker agent is triggered to re-extract the triples. The Merger agent is called to build long-term memory based on the verified triples. The Merger agent unifies entity representations, associates relationships across text segments, and performs secondary verification in conjunction with the Checker agent. Finally, the complete set of triples corresponding to a single CTI text is output.

[0019] Preprocessing refers to the structural processing of the original text, which can be achieved by stripping non-textual information, cleaning data, and segmenting it according to logical rules, providing standardized input for subsequent intelligent agent processing. Worker agents are parallel processing units built on a large language model, which can be implemented by configuring differentiated temperature parameters and using prompt words to define task scenarios, improving the efficiency of fragment-level information extraction through multi-agent parallel processing. Integrator agents are functional modules that perform semantic integration, which can be implemented by recognizing synonyms, preserving original logic, and selecting standardized terms, eliminating redundant information in the output of multiple Worker agents. Refiner agents are processing units that optimize the triple structure, which can be implemented by relation splitting, entity referencing association, and format standardization, resolving issues of nested complex relations and ambiguous expressions. Checker agents are quality verification modules, which can be implemented by illusion detection, format verification, and domain relevance screening, using error rate threshold control to trigger retry mechanisms to ensure result reliability. Merger agents are the core components for building a global knowledge base, which can be implemented by entity unification, cross-fragment relationship linking, and conflict marking, forming long-term memory with contextual consistency.

[0020] Specifically, unstructured text is preprocessed to form a logically coherent set of text fragments. Multiple Worker agents simultaneously process different fragments to generate initial triples, which are stored in short-term memory (SSM). The Integrator agent performs semantic clustering on the triples in SSM, merging synonyms and removing duplicates. The Refiner agent performs structural transformation on the integrated triples, breaking down complex relationships into atomic relation chains and correcting entity referencing biases. The Checker agent performs triple verification on the refinement results, sending a retry command to the corresponding Worker agent when the error rate exceeds a preset threshold. Triples that pass the verification enter the Merger agent's processing stage. This module establishes an entity alias mapping table, topologically connects the relationships scattered across different fragments, marks conflicting relationships, and finally outputs a globally consistent set of triples.

[0021] Stripping non-text information refers to removing non-text elements such as images, tables, and hyperlinks from CTI text. This can be achieved using regular expression matching or HTML tag parsing techniques. This operation can eliminate the interference of unstructured data on subsequent semantic analysis. Data cleaning and normalization refers to text spelling correction, redundant space removal, and encoding format standardization. This can be achieved using natural language processing toolchains. This operation can improve text quality and reduce the complexity of agent parsing. Segmenting text according to preset logic refers to segmenting based on chapter titles, paragraph themes, or semantic coherence. For example, segmenting based on logical units such as threat behavior descriptions or attack tool descriptions. Character length rule segmentation refers to cutting text exceeding a set length into segments with a fixed number of characters, such as limiting each segment to 500 characters. This operation can balance processing efficiency and contextual integrity.

[0022] In some embodiments, the Worker agents are built based on the same large language model, and the prompts for each Worker agent limit the task scenario to network threat intelligence analysis, extracting only CTI domain-specific entities and outputting triples that conform to a preset format.

[0023] It should be noted that the temperature parameter is a hyperparameter that controls the randomness of the generated results of the large language model. Specifically, it can be set differently using different numerical ranges (e.g., 0.2 to 0.8). By adjusting this parameter, the diversity and accuracy of the generated results can be balanced.

[0024] Preferably, the large language model is constructed using the same basic model architecture to implement each Worker agent. This can be achieved by sharing model weights or fine-tuning parameters, ensuring that each agent has consistent semantic understanding capabilities.

[0025] Preferably, task scenario limitation refers to constraining the analysis scope of the intelligent agent through prompts. Specifically, natural language instructions can be used to explicitly require the intelligent agent to focus only on specific domain entities such as attack behaviors and vulnerability exploits in network threat intelligence.

[0026] Preferably, the triples in the preset format refer to the data organization form that conforms to the "head entity-relationship-tail entity" structure. Specifically, they can be standardized and output in JSON or CSV format to ensure that subsequent processing modules can parse them directly.

[0027] Preferably, when worker agents process text fragments in parallel, differentiated temperature parameters are used to adjust the generation strategies of each agent. For example, agents with lower temperature parameters tend to generate more deterministic triples, while agents with higher temperature parameters may capture more potential associations. All agents are built on the same large language model to avoid semantic understanding biases caused by model differences. The network threat intelligence analysis task scenario embedded in the prompts limits the extraction process, enabling agents to automatically filter general entities (e.g., conventional network devices) and retain only proprietary entities related to APT attacks (e.g., specific malware variants). Predefined triple output requirements are implemented through structured templates, such as mandating that triple fields include attack tactics, technical names, and related indicators.

[0028] The prompts for the Worker agent include a set of Few-Shot examples. These examples cover malware behavior scenarios, vulnerability exploitation scenarios, and IOC association scenarios. Furthermore, the entities in the examples are real CTI objects, and the relationships conform to the tactical relationships defined in the ATT&CK framework.

[0029] Preferably, the Few-Shot example set refers to embedding a small number of typical scenario input / output examples in the prompt words. Specifically, this can be implemented using structured text fragments containing descriptions of malware propagation behavior, exploit chain descriptions, and attack indicator correlations, guiding the large language model to quickly understand task requirements. Real CTI objects refer to entity names in the examples derived from malicious organizations, attack tools, or vulnerability numbers appearing in publicly available threat intelligence reports. Specifically, this can be implemented using entity names registered in the MITREATT&CK technology library, ensuring a high degree of alignment between the examples and the target domain. Tactical relationships defined by the ATT&CK framework refer to relationship types in the examples strictly following the behavioral associations of tactical phases such as initial access, execution, and persistence. Specifically, this can be implemented using the tactical-technical correspondences listed in the ATT&CK matrix, constraining the relationship types output by the model to conform to threat intelligence analysis specifications. When the Worker agent processes text fragments, the pre-set Few-Shot example set in the prompt words demonstrates the entity relationship extraction process in malware behavior scenarios, such as "Emoteta Trojan spreads to the victim's host via malicious email attachments," guiding the model to identify head entities, relationships, and tail entities in similar attack chains. For vulnerability exploitation scenarios, the example set provides templates that associate vulnerability numbers with attack techniques, such as "CVE-2021-44228 vulnerability was exploited to deploy a WebShell backdoor," guiding the model to accurately extract the logical relationship between vulnerabilities and attack behaviors. In IOC association scenarios, the example set uses instances that associate IP addresses with malicious domain names, such as "IP address 192.168.1.1 resolves to the malicious domain example.com," to help the model distinguish between ordinary network behavior and attack indicators. By limiting the entities in the examples to real CTI objects, the model is prevented from generating fictitious entities; and by constraining the relationships to conform to the ATT&CK framework, the model is prevented from outputting generic relationships unrelated to threat intelligence.

[0030] In some embodiments, the processing steps of the Integrator agent include identifying the contents with the same meaning but different expressions in multiple sets of input triples, retaining the entity and relation logic in the original triples, and prioritizing the selection of triples with concise expressions and standardized terminology as the integration result. Preferably, semantic integration refers to merging triples from different text fragments that have different expressions but the same semantic meaning. This can be achieved using semantic similarity-based clustering algorithms or large language model embedding vector comparison, eliminating the problem of duplicate triples caused by text fragment splitting. Expression difference processing refers to identifying different expressions of the same entity or relation. This can be achieved by building a thesaurus or using a large language model for contextual reasoning, addressing the issue of diverse expressions such as attacker aliases and vulnerability number abbreviations in threat intelligence. Terminology standardization priority refers to selecting expressions that conform to CTI domain standards from multiple synonyms. This can be achieved using the ATT&CK framework terminology library or the MITRE CVE numbering system as reference standards, ensuring the universality of the output triples within the industry. Preferably, after receiving the initial set of triples from multiple worker agents, the Integrator agent first identifies triples describing the same attack behavior in different text fragments through semantic similarity calculation. For example, when detecting two triples, "APT29 uses CVE-2021-1234 vulnerability" and "Hacker group CozyBear exploits SolarWinds vulnerability," the entity link confirms that APT29 and CozyBear are the same attack group, and CVE-2021-1234 is the official number of the SolarWinds vulnerability. The two triples are then merged into a standardized representation. In cases where multiple reasonable representations exist, the standardized form containing the CVE number and MITRE ATT&CK tactical number is retained first. The entity relationships of the original triples are strictly preserved during the integration process; only the representation form is optimized, and the final output is a deduplicated integrated triple set. In some embodiments, the correctness verification steps of the Checker agent include: checking whether the triples have an illusion problem, i.e., verifying the consistency between the triples and the original text fragment content; checking whether the triples have a format error, i.e. confirming that the triples meet the structural requirements of head entity-relation-tail entity; and checking whether the triples have a task misunderstanding problem, i.e. filtering out triples that only contain attack intelligence. It's important to note that the "illusion problem" refers to the phenomenon where the triples generated by the large language model have no semantic connection to the original text fragments or exceed the scope of the original description. This can be verified by comparing whether the entities and relations in the triples contain keywords or synonyms that appear in the corresponding text fragments, thus eliminating errors caused by the model's arbitrary interpretation. The "formatting error" refers to triples not being output in a predefined structured format. This can be detected by matching the delimiters and field integrity of the head entity, relation, and tail entity using regular expressions, ensuring compatibility for subsequent knowledge graph construction. The "task misunderstanding problem" refers to triples containing generic entities or relations unrelated to network attacks. This can be filtered using a predefined list of CTI domain entity types and an ATT&CK framework tactical relation list, eliminating interference from non-threat intelligence data. Preferably, the Checker agent performs three verification operations on the refined triples sequentially. First, each triple is semantically aligned with its corresponding original text segment. For example, word vector similarity calculations are used to verify whether the head entity is mentioned in the original text. If an entity or relation is detected as not having a corresponding description in the original text, it is judged as a hallucination error. Second, regular expression rules are used to check the triple structure. For example, it identifies whether there is a missing separator between the head entity and the relation, or whether the tail entity has extra characters. If it does not conform to the structural specifications, it is marked as a format error. Finally, triples are filtered based on the CTI domain knowledge base. For example, if a relation is detected as a general verb rather than an ATT&CK tactical term, or an entity belongs to a non-threat intelligence object, it is judged as a mission misunderstanding error. When the proportion of erroneous triples exceeds a preset threshold, the Worker agent is triggered to reprocess the corresponding text segment. Preferably, the steps for the Merger agent to handle conflicts include calling the Checker agent to backtrack the original text fragments corresponding to the conflicting triples and retaining the triples with strong original textual evidence; if both conflicting triples are supported by text, then marking them as "multiple actions" in long-term memory to retain all triples; if entity referencing conflicts are detected, then updating the prompt words to guide the Merger agent to correct entity referencing. In some embodiments, calling the Checker agent to trace back the original text fragment corresponding to the conflicting triple refers to utilizing the Checker agent's correctness verification capability to verify the reliability of the conflicting triple's source by retrieving the original text fragment. This can be implemented using an indexing mechanism based on text fragment numbers, ensuring that the original data context is always associated during conflict resolution. Marking "multiple behaviors" refers to establishing special identifiers in long-term memory, which can be implemented using metadata annotation to distinguish multiple attack behaviors exhibited by the same entity in different text fragments, avoiding intelligence loss due to simple deletion. Updating prompt words to guide the Merger agent in correcting entity referencing refers to dynamically adjusting the agent's processing rules. This can be implemented using a keyword replacement strategy based on conflict type; for example, when a conflict is detected between "C2 server" and "command and control infrastructure," standardized terminology is enforced in the prompt words. Preferably, when the Merger agent detects contradictory relationships of the same entity during the construction of long-term memory, it first extracts the conflicting triples and their associated text fragment numbers, and calls the Checker agent to retrieve the original content of the corresponding fragments for semantic verification. For triples with clear original text support, their content is retained first; if multiple conflicting triples have textual basis, they are added with "multi-behavior" tags in the long-term memory and stored completely. For example, an APT organization is described as "using spear phishing" in text fragment A and as "deploying Cobalt Strike" in fragment B; both are retained and marked as related. When a difference in entity description is detected, entity alias mapping rules are added by updating the Merger agent's prompt words, such as establishing an equivalence relationship between "TA505" and "threat organization 123", and term substitution is automatically performed in subsequent processing.

[0031] Preferably, the Checker agent's backtracking of the original text fragment corresponding to the conflicting triples refers to verifying the reliability of the conflicting triples' source by retrieving the original text content. Specifically, this can be achieved using text fragment hash value matching technology to ensure that the correspondence between the conflicting triples and the original text is traceable. This step addresses the problem of insufficient evidence in conflict handling in traditional methods, enhancing the credibility of decisions through backtracking of the original text. Preferably, marking multiple behaviors refers to establishing multiple associations for the same entity in long-term memory. This can be achieved using a multi-sided connection structure in a relational graph, such as establishing behavioral relationships for different attack stages for the same attacker entity in a knowledge graph. This step preserves the complex attack behavior characteristics present in real threat intelligence, avoiding information loss due to simple overwriting. Updating prompt words guides the Merger agent to correct entity referencing, which involves dynamically adjusting entity association rules. This can be achieved using context-aware entity resolution algorithms, such as identifying different representations of the same entity in different text fragments based on a coreference resolution model. This step addresses the problem of inconsistent entity referencing across fragments, improving the overall consistency of the triple set. Preferably, when the Merger agent detects contradictory relationships between the same entity during the construction of long-term memory, it first triggers the Checker agent to verify the original text of both conflicting parties. For example, if an APT organization has two conflicting descriptions: "deploying a backdoor" and "establishing a C2 channel," and if the two behaviors appear in different text fragments and are both supported by the original text, then a multi-behavior tag is added to the entity in the knowledge graph. For entity referencing conflicts, such as malware being abbreviated as "sample A" in the first half of the text but fully named "XZ_Backdoor_v3" in the second half, the system will update the entity association rules and automatically identify the equivalence of the two expressions in subsequent processing. The resulting long-term memory retains the integrity of the original intelligence while achieving a standardized expression of entity relationships.

[0032] refer to Figure 3 The present invention also provides an APT intelligence triple extraction system based on a large language model multi-unit, the system comprising: The text preprocessing module is used to obtain unstructured network threat intelligence text from an open-source intelligence platform and preprocess the network threat intelligence text to obtain multiple text fragments. The short-term memory construction module includes a set of working agents that process each text segment in parallel and extract an initial set of triples to construct short-term memory. The triplet processing module includes an integration agent, a refinement agent, and a verification agent connected in sequence. The integration agent performs semantic integration and deduplication on the initial triplet set, the refinement agent performs format correction, referential resolution, and compound relation splitting, and the verification agent performs correctness verification and triggers a retrial of the working agent when the error rate exceeds the standard. The memory construction output module includes a merging agent, which unifies entity representation, associates relationships across text fragments, and performs secondary verification in conjunction with a verification agent, ultimately outputting a complete set of triples. Preferably, the system further includes a model optimization module, used to fine-tune the parameters of the large language model component in the multi-unit collaboration module based on feedback information from the final triplet data set.

[0033] refer to Figure 4 Another embodiment of the present invention provides an APT intelligence triple extraction device based on a large language model multi-unit, comprising: one or more processors; and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any of the APT intelligence triple extraction methods based on a large language model multi-unit.

[0034] Figure 4 This is a schematic diagram of the structure of the APT intelligence triple extraction device based on a large language model multi-unit provided in the embodiments of this application, with reference to... Figure 4 The APT intelligence triplet extraction device based on a large language model multi-unit includes: a processor 31, a memory 32, a communication device 33, an input device 34, and an output device 35. The number of processors 31 and the number of memories 32 in the APT intelligence triplet extraction device based on a large language model multi-unit can be one or more. The processor 31, memory 32, communication device 33, input device 34, and output device 35 of the APT intelligence triplet extraction device based on a large language model multi-unit can be connected via a bus or other means.

[0035] Memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the APT intelligence triple extraction method based on a large language model multi-unit in any embodiment of this application. Memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system and at least one program required for a function; the data storage area may store data created according to the device's operation, etc. In addition, memory 32 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0036] The communication device 33 is used for data transmission.

[0037] The processor 31 executes various functions of the device and performs data processing by running software programs, instructions, and modules stored in the memory 32, thereby realizing the above-mentioned APT intelligence triple extraction method based on a large language model multi-unit.

[0038] Input device 34 can be used to receive input digital or character information, and to generate key signal inputs related to the device's user settings and function control. Output device 35 may include display devices such as a display screen.

[0039] The APT intelligence triple extraction device based on a large language model multi-unit provided above can be used to execute the APT intelligence triple extraction method based on a large language model multi-unit provided in the above embodiments, and has corresponding functions and beneficial effects.

[0040] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made based on the description and drawings of the present invention under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for extracting APT intelligence triplets based on a large language model multi-unit, characterized in that, The method comprises: obtaining network threat intelligence text from an open source intelligence platform, and preprocessing the network threat intelligence text to obtain a plurality of text segments; deploying a group of working agents, the working agents processing the plurality of text segments, extracting an initial triple set from the plurality of text segments, and constructing a short-term memory; sequentially invoking an integration agent, a refining agent and a verification agent to process the initial triple set, wherein the integration agent performs semantic integration and deduplication on the initial triple set, the refining agent performs format correction, coreference resolution and complex relationship splitting on the integrated triple, the verification agent performs correctness verification on the refined triple, and if the proportion of incorrect triples exceeds a preset threshold, the working agent is triggered to retry extraction; calling the working agent to construct a long-term memory based on the verified triple, the working agent unifying entity expressions, associating cross-text segment relationships, and performing secondary verification in combination with the verification agent, and outputting a complete triple set.

2. The method according to claim 1, wherein, The preprocessing step of the network threat intelligence text comprises stripping non-text information in the network threat intelligence text, data cleaning and normalization processing of the stripped text, and splitting the normalized text into a plurality of text segments according to the character length rule.

3. The method according to claim 1, wherein, The working agent is constructed based on the same large language model, and the prompt words contain an example set covering multiple network threat scenarios, and the working agent is configured to extract only network threat intelligence domain-specific entities and output triples in a preset format.

4. The method according to claim 3, wherein, The refining agent performs format correction, coreference resolution and complex relationship splitting on the integrated triple, which comprises: extracting at least one entity pair from the initial triple set; inputting the context text segment containing the entity pair into a preset large language model; determining the relationship type between the entity pair based on the output of the large language model, thereby forming a preliminary relationship set of the initial triple.

5. The method according to claim 1, wherein, The verification agent performs correctness verification on the refined triple, which comprises: detecting whether there is inconsistency between the entities in the preliminary triple set and the entity references in the preliminary relationship set; when inconsistency is detected, reanalyzing based on the context semantic information of the standardized text data to determine the correct entity and / or relationship; eliminating logical conflicts between output results and integrating to obtain the final triple set.

6. The method according to claim 1, wherein, Further comprising: generating a triple data set and storing the triple data set in a non-volatile storage medium.

7. The method according to claim 1, wherein, The method further comprises: before outputting the triple data set, formatting the triple data set to conform to a preset data exchange standard.

8. The method according to any one of claims 1 to 7, wherein, Further comprising: based on the feedback information of the triple data set, parameter fine-tuning of the large language model component.

9. A large language model multi-unit based APT intelligence triple extraction system, characterized in that, The system comprises: a text preprocessing module configured to obtain unstructured network threat intelligence text from an open source intelligence platform and preprocess the network threat intelligence text to obtain a plurality of text segments; The short-term memory construction module comprises a group of working agents which process each text segment in parallel to extract an initial triple set to construct short-term memory; The triple processing module comprises an integration agent, a refining agent and a verification agent connected in sequence, the integration agent performs semantic integration and deduplication on the initial triple set, the refining agent performs format correction, coreference resolution and complex relationship splitting, and the verification agent performs correctness verification and triggers the working agent to retry when the error rate exceeds the standard; The memory construction output module comprises a merging agent which unifies entity expressions, associates cross-text segment relationships, and performs secondary verification in combination with the verification agent to finally output a complete triple set.

10. The large language model-based multi-unit APT intelligence triple extraction system according to claim 9, wherein, The system further comprises: A model optimization module for fine-tuning the parameters of the large language model component in the multi-unit cooperative module based on the feedback information of the final triple data set.

Images