Safety traceability knowledge graph construction method based on large language model

Through a method based on a large language model, the entities and relationships in secure data are automatically extracted, which solves the problem of relying on manual annotation data in the existing technology, and improves the accuracy and efficiency of the construction of a secure traceability knowledge graph.

CN119047549APending Publication Date: 2024-11-29YANCHENG POWER SUPPLY CO STATE GRID JIANGSU ELECTRIC POWER CO

Patent Information

Application Number
CN202410901029.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-05
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

When building a secure traceability knowledge graph, the prior art relies on a large amount of manual labeling data, making it difficult to effectively extract secure entities in a variety of log information, and there are fuzzy entity boundaries and domain special words, which increases the difficulty.

Method used

Using a method based on a large language model, through four parts: data acquisition, data set generation, model fine-tuning and knowledge graph construction, using open general model and LoRA for fine-tuning instructions, automatically extracting entities and relationships in secure data, and building security Trace the knowledge graph.

Benefits of technology

It reduces the cost of manually labeling data, improves the recognition accuracy of the model, realizes more effective construction of secure traceability knowledge graphs, and automatically extracts secure entities and relationships.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119047549A_ABST
    Figure CN119047549A_ABST
Patent Text Reader

Abstract

The invention provides a security traceability knowledge graph construction method based on a large language model, which comprises the following steps of: 1, collecting related security data, and analyzing text content; step 2, generating a data set for fine adjustment of the model by using the open general large model; step 3, performing instruction fine tuning on the local large language model based on LoRA, and analyzing the collected textualized security data by using the trained model; and 4, according to the extracted information, carrying out fusion on the entity and the relationship, and constructing a security traceability knowledge graph. The invention provides a security traceability mapping knowledge domain construction method based on a large language model, which can automatically extract entities and relationships in security data, improve the recognition accuracy of the model and more effectively construct a security traceability mapping knowledge domain.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of cyberspace security technology, and in particular relates to a method for constructing a security traceability knowledge graph based on a large language model. Background Art

[0002] Logs are recorded information generated by computer systems, applications or services, recording various activities, events and states of the system. Log analysis technology refers to the collection, storage, parsing and query of system logs to obtain information about system operation status, performance monitoring and security analysis. There are many business systems and middleware in the process of secure operation of information systems. Attackers may exploit vulnerabilities in the operating system to invade, or exploit vulnerabilities in middleware such as Apache, Tomcat, Nginx, etc. to attack the system. In order to completely restore the real attack scenario, in addition to the data in the alarm log, it is also necessary to perform necessary processing on the massive alarm data obtained from the information system based on the real and complete operating system logs and middleware logs for analysis.

[0003] Knowledge graph is an effective information expression method. By building a security traceability knowledge graph, the purpose of attack detection and traceability can be achieved through the knowledge graph. At the same time, graphical display technology can present complex traceability relationships in an intuitive and easy-to-understand way, helping users better understand the process of attack events and the attacker's behavior patterns.

[0004] In order to extract entity objects from logs, most methods use machine learning methods such as LSTM or BERT to extract entity objects from log files. A common method is to use natural language processing techniques, such as word segmentation and part-of-speech tagging, to convert log files into word sequences. Then, deep learning models such as BERT can be used to analyze these word sequences and extract entity objects from them. However, this method relies on a large amount of labeled data and requires a lot of manual sequence labeling. At the same time, security entities have typical characteristics of the security field, such as fuzzy entity boundaries and special domain-specific words. These problems increase the difficulty of extracting security entities from a variety of log information to build a knowledge graph. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a method for constructing a security traceability knowledge graph based on a large language model, which can automatically extract entities and relationships in security data, improve the recognition accuracy of the model, and more effectively construct a security traceability knowledge graph.

[0006] In view of the shortcomings of the above-mentioned prior art, the present invention provides a method for constructing a security traceability knowledge graph based on a large language model, which includes four parts: data acquisition, data set generation, model fine-tuning, and knowledge graph construction, and includes the following steps:

[0007] Step 1: Collect relevant security data and parse text content;

[0008] Step 2: Generate a dataset for model fine-tuning using an open general-purpose large model;

[0009] Step 3: Fine-tune the local large language model based on LoRA, and use the trained model to analyze the collected textual security data;

[0010] Step 4: Integrate entities and relationships based on the extracted information to build a security traceability knowledge graph.

[0011] Furthermore, the step 1 specifically includes the following sub-steps:

[0012] Step 1.1: Collect relevant security data according to the application scenario, including device information, security event logs, alarm logs, middleware logs, etc.

[0013] Step 1.2, according to the data format of each type of security data, parse out the valid text content in the data;

[0014] Step 1.3: Remove the original blank lines in the text and store the text content in the database as the data source for security traceability analysis. Determine the entities and relationships of the security traceability knowledge graph, such as entity categories such as devices, users, IP addresses, files, and relationships such as device associations, user-device associations, and device-IP address associations.

[0015] Furthermore, the step 2 specifically includes the following sub-steps:

[0016] Step 2.1, construct task instructions and output examples for named entity recognition and relationship extraction as instructions for the open general large model input, where the task description is a natural language description of the processing target of the input text, and the output example is one or several examples of "input text + output result" for the task description, and the examples are used to further explain the task content and specify the corresponding output structure;

[0017] Step 2.2: Using the contextual learning capability of the open general large model and the input content structure of "instruction + input text", obtain the output result corresponding to the model;

[0018] Step 2.3: Combine instructions, input text, and output results to form a dataset for model fine-tuning, where the model input consists of instructions and input text, and the model output refers to the output result of the general large model.

[0019] Furthermore, the step 3 specifically includes the following sub-steps:

[0020] Step 3.1: Use the constructed dataset and LoRA-based instruction fine-tuning method to perform local fine-tuning on the local large language model with smaller parameters;

[0021] Step 3.2: Use the trained model to analyze the collected security data and extract the security entities and relationships between entities in the text.

[0022] Furthermore, the step 4 specifically includes the following sub-steps:

[0023] Step 4.1, fuse security entities and relations according to the extracted information through rule and hierarchical agglomerative clustering methods;

[0024] Step 4.2: Use the fused information to construct a knowledge graph and store it graphically.

[0025] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0026] 1. Utilize the contextual learning ability of the large language model to generate the required data set through task description and output cases, reducing the cost of manual data annotation.

[0027] 2. Utilize the text understanding ability and instruction fine-tuning method of the large language model, and fine-tune the large language model so that it can automatically extract entities and relationships in security data, improve the recognition accuracy of the model, and more effectively build a security traceability knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a structural framework diagram of a method for constructing a security traceability knowledge graph based on a large language model in the present invention;

[0029] Figure 2 Schematic diagram of the instruction format used to generate the data set. DETAILED DESCRIPTION

[0030] The following is a detailed description of a specific implementation method of a security traceability knowledge graph construction method based on a large language model of the present invention in conjunction with the accompanying drawings.

[0031] This paper proposes a method for constructing a security traceability knowledge graph based on a large language model. Figure 1As shown in the figure, it is divided into four parts: data acquisition, data set generation, model fine-tuning, and knowledge graph construction. First, collect relevant security data, including device information, security event logs, alarm logs, middleware logs and other data, and parse the text content; then, determine the entities and relationships of the security traceability knowledge graph, construct task instructions and output examples for named entity recognition and relationship extraction, and use the context learning ability of the open general large model GPT model to obtain the data set required by this method; use the LoRA-based instruction fine-tuning method to fine-tune the local large language model so that it can extract security entities and relationships between entities in the text; finally, fuse the entities and relationships based on the above extracted information to build a security traceability knowledge graph.

[0032] Specifically, the security traceability knowledge graph construction method of the present invention includes the following steps:

[0033] Step 1: Collect relevant security data and parse the text content. The specific process is as follows:

[0034] Step 1.1: Collect relevant security data according to the application scenario, including device information, security event logs, alarm logs, middleware logs, etc.

[0035] Step 1.2, according to the data format of each type of security data, parse out the valid text content in the data;

[0036] Step 1.3, remove the original blank lines in the text, store the text content in the database as the data source for security traceability analysis, and determine the entities and relationships of the security traceability knowledge graph, such as entity categories such as devices, users, IP addresses, files, device associations, user-device associations, device-IP address associations, etc., as shown in Table 1 and Table 2.

[0037] Table 1

[0038]

[0039]

[0040] Table 2

[0041]

[0042] Step 2: Use the open general large model to generate a data set for model fine-tuning. The specific process is as follows:

[0043] Step 2.1, construct the task instructions and output examples of named entity recognition and relationship extraction as the instructions for the open general large model input, where the task description is a natural language description of the processing target of the input text, and the output example is one or several examples of "input text + output result" for the task description. The examples further explain the task content and specify the corresponding output structure. The corresponding instruction format example used to generate the data set is as follows Figure 2 As shown;

[0044] The task description of named entity recognition adopts an instruction-based information extraction method, requiring the model to output entities and relations in the input text, and specifying the types of security entities and relations to be extracted;

[0045] Step 2.2: Use the contextual learning ability of the open general large model and the input content structure of "command + input text" to obtain the output result corresponding to the model. The general large model selected here is GPT-3.5-turbo;

[0046] Step 2.3, combine instructions, input text, and output results to form a data set for model fine-tuning. Each data in the data set includes two parts: model input and model output. The model input consists of instructions and input text, and the model output refers to the output result of the general large model GPT.

[0047] Step 3: Fine-tune the local large language model based on LoRA, and use the trained model to analyze the collected textual security data. The specific process is as follows:

[0048] Step 3.1, use the constructed dataset and LoRA-based instruction fine-tuning method to fine-tune the large language model with smaller parameters. The fine-tuned large language model selected here is LLaMA-7B, and the hyperparameters used for fine-tuning training are shown in Table 3;

[0049] Table 3

[0050]

[0051] Step 3.2: Use the trained model to analyze the collected security data and extract the security entities and relationships between entities in the text.

[0052] Step 4: Based on the extracted information, entities and relationships are integrated to build a security traceability knowledge graph. The specific process is as follows:

[0053] Step 4.1: Fusion of entities and relations based on the extracted information, fusion of security entities through rules, retention of words with proper names, such as IP addresses, user names, etc., without merging, merging identical entities through absolute and relative paths for file names, and merging of security alerts through retained keywords; clustering and fusion of relational words through hierarchical agglomerative clustering method, calculation of word embedding representation for each word, use of cosine similarity as distance metric, and selection of the word with the highest TF-IDF weight value as the representative word of a cluster;

[0054] Step 4.2: Use the fused information to build a knowledge graph and use the Neo4j database for graphical storage.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit the present invention. A person skilled in the art should understand that the specific implementation of the present invention can be modified or replaced by equivalents, but these modifications or changes are within the scope of protection of the pending claims.

Claims

1. A method for constructing a security traceability knowledge graph based on a large language model, characterized in that: It includes four parts: data acquisition, data set generation, model fine-tuning, and knowledge graph construction, including the following steps: Step 1: Collect relevant security data and parse text content; Step 2: Generate a dataset for model fine-tuning using an open general-purpose large model; Step 3: Fine-tune the local large language model based on LoRA, and use the trained model to analyze the collected textual security data; Step 4: Integrate entities and relationships based on the extracted information to build a security traceability knowledge graph.

2. According to claim 1, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The step 1 specifically includes the following sub-steps: Step 1.1: Collect relevant security data according to the application scenario, including device information, security event logs, alarm logs, and middleware log data; Step 1.2, according to the data format of each type of security data, parse out the valid text content in the data; Step 1.3: Remove the original blank lines in the text and store the text content in the database as the data source for security traceability analysis to determine the entities and relationships of the security traceability knowledge graph.

3. According to claim 2, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The step 2 specifically includes the following sub-steps: Step 2.1, construct task instructions and output examples for named entity recognition and relationship extraction as instructions for input of the open general large model; Step 2.2: Utilize the contextual learning capability of the open general large model and the input content structure of "command + input text" to obtain the output result corresponding to the model; Step 2.3: Combine instructions, input text, and output results to form a dataset for model fine-tuning.

4. According to claim 3, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The task description in step 2.1 is a natural language description of the processing target of the input text, and the output example is one or several examples of "input text + output result" for the task description, which further explains the task content and specifies the corresponding output structure through examples.

5. According to claim 4, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The task description of named entity recognition in step 2.1 adopts an instruction-based information extraction method, requiring the model to output entities and relationships in the input text, and specifies the types of security entities and relationships that need to be extracted.

6. According to claim 3, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: Each piece of data in the data set described in step 2.3 includes two parts: model input and model output, where the model input consists of instructions and input text, and the model output refers to the output result of the GPT model.

7. According to claim 1, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The step 3 specifically includes the following sub-steps: Step 3.1: Use the constructed dataset and LoRA-based instruction fine-tuning method to perform localized fine-tuning on the large language model with smaller parameters; Step 3.2: Use the trained model to analyze the collected security data and extract the security entities and relationships between entities in the text.

8. According to claim 1, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The step 4 specifically includes the following sub-steps: Step 4.1, integrating security entities and relationships based on the extracted information; Step 4.2: Use the fused information to construct a knowledge graph and store it graphically.

9. A method for constructing a security traceability knowledge graph based on a large language model according to claim 8, characterized in that: The step 4.1 integrates security entities and relationships through rule and hierarchical agglomerative clustering methods.

Citation Information

Patent Citations

  • Knowledge graph generation type question answering method and system based on large language model

    CN117033608A

  • Fine adjustment method, system and equipment based on large language model and medium

    CN117290480A

  • Knowledge graph-based method for enhancing dual-carbon domain service of large language model

    CN117370580A

  • Legal assistance method and system based on large language model

    CN117743590A

  • Knowledge graph construction method and device based on pre-trained large language model

    CN117851610A

Cited By

  • Space-time knowledge graph construction method

    CN119377420A

  • Tracing method and system based on power enterprise production information

    CN119904012A

  • Knowledge-driven multi-mode large model lung cancer postoperative rehabilitation guidance method and system

    CN120674011A

  • Emergency response standard knowledge extraction method based on large language model

    CN120821858A

  • Urban highway tunnel operation period toughness safety evaluation method

    CN121563312A