Safety traceability knowledge graph construction method based on large language model
Through a method based on a large language model, the entities and relationships in secure data are automatically extracted, which solves the problem of relying on manual annotation data in the existing technology, and improves the accuracy and efficiency of the construction of a secure traceability knowledge graph.
Patent Information
- Application Number
- CN202410901029.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-05
- Publication Date
- 2024-11-29
AI Technical Summary
When building a secure traceability knowledge graph, the prior art relies on a large amount of manual labeling data, making it difficult to effectively extract secure entities in a variety of log information, and there are fuzzy entity boundaries and domain special words, which increases the difficulty.
Using a method based on a large language model, through four parts: data acquisition, data set generation, model fine-tuning and knowledge graph construction, using open general model and LoRA for fine-tuning instructions, automatically extracting entities and relationships in secure data, and building security Trace the knowledge graph.
It reduces the cost of manually labeling data, improves the recognition accuracy of the model, realizes more effective construction of secure traceability knowledge graphs, and automatically extracts secure entities and relationships.
Smart Images

Figure CN119047549A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of cyberspace security technology, and in particular relates to a method for constructing a security traceability knowledge graph based on a large language model. Background Art
[0002] Logs are recorded information generated by computer systems, applications or services, recording various activities, events and states of the system. Log analysis technology refers to the collection, storage, parsing and query of system logs to obtain information about system operation status, performance monitoring and security analysis. There are many business systems and middleware in the process of secure operation of information systems. Attackers may exploit vulnerabilities in the operating system to invade, or exploit vulnerabilities in middleware such as Apache, Tomcat, Nginx, etc. to attack the system. In order to completely restore the real attack scenario, in addition to the data in the alarm log, it is also necessary to perform necessary processing on the massive alarm data obtained from the information system based on the real and complete operating system logs and middleware logs for analysis.
[0003] Knowledge graph is an effective information expression method. By building a security traceability knowledge graph, the purpose of attack detection and traceability can be achieved through the knowledge graph. At the same time, graphical display technology can present complex traceability relationships in an intuitive and easy-to-understand way, helping users better understand the process of attack events and the attacker's behavior patterns.
[0004] In order to extract entity objects from logs, most methods use machine learning methods such as LSTM or BERT to extract entity objects from log files. A common method is to use natural language processing techniques, such as word segmentation and part-of-speech tagging, to convert log files into word sequences. Then, deep learning models such as BERT can be used to analyze these word sequences and extract entity objects from them. However, this method relies on a large amount of labeled data and requires a lot of manual sequence labeling. At the same time, security entities have typical characteristics of the security field, such as fuzzy entity boundaries and special domain-specific words. These problems increase the difficulty of extracting security entities from a variety of log information to build a knowledge graph. Summary of the invention
[0005] In order to solve the above problems, the present invention provides a method for constructing a security traceability knowledge graph based on a large language model, which can automatically extract entities and relationships in security data, improve the recognition accuracy of the model, and more effectively construct a security traceability knowledge graph.
[0006] In view of the shortcomings of the above-mentioned prior art, the present invention provides a method for constructing a security traceability knowledge graph based on a large language model, which includes four parts: data acquisition, data set generation, model fine-tuning, and knowledge graph construction, and includes the following steps:
[0007] Step 1: Collect relevant security data and parse text content;
[0008] Step 2: Generate a dataset for model fine-tuning using an open general-purpose large model;
[0009] Step 3: Fine-tune the local large language model based on LoRA, and use the trained model to analyze the collected textual security data;
[0010] Step 4: Integrate entities and relationships based on the extracted information to build a security traceability knowledge graph.
[0011] Furthermore, the step 1 specifically includes the following sub-steps:
[0012] Step 1.1: Collect relevant security data according to the application scenario, including device information, security event logs, alarm logs, middleware logs, etc.
[0013] Step 1.2, according to the data format of each type of security data, parse out the valid text content in the data;
[0014] Step 1.3: Remove the original blank lines in the text and store the text content in the database as the data source for security traceability analysis. Determine the entities and relationships of the security traceability knowledge graph, such as entity categories such as devices, users, IP addresses, files, and relationships such as device associations, user-device associations, and device-IP address associations.
[0015] Furthermore, the step 2 specifically includes the following sub-steps:
[0016] Step 2.1, construct task instructions and output examples for named entity recognition and relationship extraction as instructions for the open general large model input, where the task description is a natural language description of the processing target of the input text, and the output example is one or several examples of "input text + output result" for the task description, and the examples are used to further explain the task content and specify the corresponding output structure;
[0017] Step 2.2: Using the contextual learning capability of the open general large model and the input content structure of "instruction + input text", obtain the output result corresponding to the model;
[0018] Step 2.3: Combine instructions, input text, and output results to form a dataset for model fine-tuning, where the model input consists of instructions and input text, and the model output refers to the output result of the general large model.
[0019] Furthermore, the step 3 specifically includes the following sub-steps:
[0020] Step 3.1: Use the constructed dataset and LoRA-based instruction fine-tuning method to perform local fine-tuning on the local large language model with smaller parameters;
[0021] Step 3.2: Use the trained model to analyze the collected security data and extract the security entities and relationships between entities in the text.
[0022] Furthermore, the step 4 specifically includes the following sub-steps:
[0023] Step 4.1, fuse security entities and relations according to the extracted information through rule and hierarchical agglomerative clustering methods;
[0024] Step 4.2: Use the fused information to construct a knowledge graph and store it graphically.
[0025] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0026] 1. Utilize the contextual learning ability of the large language model to generate the required data set through task description and output cases, reducing the cost of manual data annotation.
[0027] 2. Utilize the text understanding ability and instruction fine-tuning method of the large language model, and fine-tune the large language model so that it can automatically extract entities and relationships in security data, improve the recognition accuracy of the model, and more effectively build a security traceability knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 This is a structural framework diagram of a method for constructing a security traceability knowledge graph based on a large language model in the present invention;
[0029] Figure 2 Schematic diagram of the instruction format used to generate the data set. DETAILED DESCRIPTION
[0030] The following is a detailed description of a specific implementation method of a security traceability knowledge graph construction method based on a large language model of the present invention in conjunction with the accompanying drawings.
[0031] This paper proposes a method for constructing a security traceability knowledge graph based on a large language model. Figure 1As shown in the figure, it is divided into four parts: data acquisition, data set generation, model fine-tuning, and knowledge graph construction. First, collect relevant security data, including device information, security event logs, alarm logs, middleware logs and other data, and parse the text content; then, determine the entities and relationships of the security traceability knowledge graph, construct task instructions and output examples for named entity recognition and relationship extraction, and use the context learning ability of the open general large model GPT model to obtain the data set required by this method; use the LoRA-based instruction fine-tuning method to fine-tune the local large language model so that it can extract security entities and relationships between entities in the text; finally, fuse the entities and relationships based on the above extracted information to build a security traceability knowledge graph.
[0032] Specifically, the security traceability knowledge graph construction method of the present invention includes the following steps:
[0033] Step 1: Collect relevant security data and parse the text content. The specific process is as follows:
[0034] Step 1.1: Collect relevant security data according to the application scenario, including device information, security event logs, alarm logs, middleware logs, etc.
[0035] Step 1.2, according to the data format of each type of security data, parse out the valid text content in the data;
[0036] Step 1.3, remove the original blank lines in the text, store the text content in the database as the data source for security traceability analysis, and determine the entities and relationships of the security traceability knowledge graph, such as entity categories such as devices, users, IP addresses, files, device associations, user-device associations, device-IP address associations, etc., as shown in Table 1 and Table 2.
[0037] Table 1
[0038]
[0039]
[0040] Table 2
[0041]
[0042] Step 2: Use the open general large model to generate a data set for model fine-tuning. The specific process is as follows:
[0043] Step 2.1, construct the task instructions and output examples of named entity recognition and relationship extraction as the instructions for the open general large model input, where the task description is a natural language description of the processing target of the input text, and the output example is one or several examples of "input text + output result" for the task description. The examples further explain the task content and specify the corresponding output structure. The corresponding instruction format example used to generate the data set is as follows Figure 2 As shown;
[0044] The task description of named entity recognition adopts an instruction-based information extraction method, requiring the model to output entities and relations in the input text, and specifying the types of security entities and relations to be extracted;
[0045] Step 2.2: Use the contextual learning ability of the open general large model and the input content structure of "command + input text" to obtain the output result corresponding to the model. The general large model selected here is GPT-3.5-turbo;
[0046] Step 2.3, combine instructions, input text, and output results to form a data set for model fine-tuning. Each data in the data set includes two parts: model input and model output. The model input consists of instructions and input text, and the model output refers to the output result of the general large model GPT.
[0047] Step 3: Fine-tune the local large language model based on LoRA, and use the trained model to analyze the collected textual security data. The specific process is as follows:
[0048] Step 3.1, use the constructed dataset and LoRA-based instruction fine-tuning method to fine-tune the large language model with smaller parameters. The fine-tuned large language model selected here is LLaMA-7B, and the hyperparameters used for fine-tuning training are shown in Table 3;
[0049] Table 3
[0050]
[0051] Step 3.2: Use the trained model to analyze the collected security data and extract the security entities and relationships between entities in the text.
[0052] Step 4: Based on the extracted information, entities and relationships are integrated to build a security traceability knowledge graph. The specific process is as follows:
[0053] Step 4.1: Fusion of entities and relations based on the extracted information, fusion of security entities through rules, retention of words with proper names, such as IP addresses, user names, etc., without merging, merging identical entities through absolute and relative paths for file names, and merging of security alerts through retained keywords; clustering and fusion of relational words through hierarchical agglomerative clustering method, calculation of word embedding representation for each word, use of cosine similarity as distance metric, and selection of the word with the highest TF-IDF weight value as the representative word of a cluster;
[0054] Step 4.2: Use the fused information to build a knowledge graph and use the Neo4j database for graphical storage.
[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit the present invention. A person skilled in the art should understand that the specific implementation of the present invention can be modified or replaced by equivalents, but these modifications or changes are within the scope of protection of the pending claims.
Claims
1. A method for constructing a security traceability knowledge graph based on a large language model, characterized in that: It includes four parts: data acquisition, data set generation, model fine-tuning, and knowledge graph construction, including the following steps: Step 1: Collect relevant security data and parse text content; Step 2: Generate a dataset for model fine-tuning using an open general-purpose large model; Step 3: Fine-tune the local large language model based on LoRA, and use the trained model to analyze the collected textual security data; Step 4: Integrate entities and relationships based on the extracted information to build a security traceability knowledge graph.
2. According to claim 1, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The step 1 specifically includes the following sub-steps: Step 1.1: Collect relevant security data according to the application scenario, including device information, security event logs, alarm logs, and middleware log data; Step 1.2, according to the data format of each type of security data, parse out the valid text content in the data; Step 1.3: Remove the original blank lines in the text and store the text content in the database as the data source for security traceability analysis to determine the entities and relationships of the security traceability knowledge graph.
3. According to claim 2, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The step 2 specifically includes the following sub-steps: Step 2.1, construct task instructions and output examples for named entity recognition and relationship extraction as instructions for input of the open general large model; Step 2.2: Utilize the contextual learning capability of the open general large model and the input content structure of "command + input text" to obtain the output result corresponding to the model; Step 2.3: Combine instructions, input text, and output results to form a dataset for model fine-tuning.
4. According to claim 3, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The task description in step 2.1 is a natural language description of the processing target of the input text, and the output example is one or several examples of "input text + output result" for the task description, which further explains the task content and specifies the corresponding output structure through examples.
5. According to claim 4, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The task description of named entity recognition in step 2.1 adopts an instruction-based information extraction method, requiring the model to output entities and relationships in the input text, and specifies the types of security entities and relationships that need to be extracted.
6. According to claim 3, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: Each piece of data in the data set described in step 2.3 includes two parts: model input and model output, where the model input consists of instructions and input text, and the model output refers to the output result of the GPT model.
7. According to claim 1, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The step 3 specifically includes the following sub-steps: Step 3.1: Use the constructed dataset and LoRA-based instruction fine-tuning method to perform localized fine-tuning on the large language model with smaller parameters; Step 3.2: Use the trained model to analyze the collected security data and extract the security entities and relationships between entities in the text.
8. According to claim 1, a method for constructing a security traceability knowledge graph based on a large language model is characterized in that: The step 4 specifically includes the following sub-steps: Step 4.1, integrating security entities and relationships based on the extracted information; Step 4.2: Use the fused information to construct a knowledge graph and store it graphically.
9. A method for constructing a security traceability knowledge graph based on a large language model according to claim 8, characterized in that: The step 4.1 integrates security entities and relationships through rule and hierarchical agglomerative clustering methods.
Citation Information
Patent Citations
Knowledge graph generation type question answering method and system based on large language model
CN117033608A
Fine adjustment method, system and equipment based on large language model and medium
CN117290480A
Knowledge graph-based method for enhancing dual-carbon domain service of large language model
CN117370580A
Legal assistance method and system based on large language model
CN117743590A
Knowledge graph construction method and device based on pre-trained large language model
CN117851610A
Cited By
Space-time knowledge graph construction method
CN119377420A
Tracing method and system based on power enterprise production information
CN119904012A
Knowledge-driven multi-mode large model lung cancer postoperative rehabilitation guidance method and system
CN120674011A
Emergency response standard knowledge extraction method based on large language model
CN120821858A
Urban highway tunnel operation period toughness safety evaluation method
CN121563312A
