Large language model corpus construction method and system, electronic equipment and storage medium

Abstract

The invention discloses a large language model corpus construction method and system, electronic equipment and a storage medium, and the method comprises the steps: judging whether a newly generated instruction is a classification task instruction through a first large language model based on an initially set network security task seed pool, and if not, determining that the instruction is a non-classification task instruction; inputting the classification task instruction into a first large language model according to an input priority mode, generating an input-output pair, and forming a classification task corpus set; inputting the non-classification task instruction into a first large language model according to an output priority mode, generating an input-output pair, and forming a non-classification task corpus set; and merging the classified task corpus set and the non-classified task corpus set, and filtering low-quality and repeated instructions to form a large language model instruction fine adjustment corpus set in the network security field. According to the method, the diversity of instruction fine adjustment samples can be enriched, so that the performance of the network security large model in the instruction fine adjustment stage is further improved.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a large language model corpus construction method, system, electronic device and storage medium. Background Art

[0002] In recent years, large language models have made significant progress in the field of natural language processing, and their powerful language understanding and generation capabilities have brought new opportunities to the field of network security. By learning from massive amounts of text data, large language models can capture the complex characteristics of language and provide rich semantic information for network security analysis. However, there are some challenges in directly applying large language models to the field of network security, such as the lack of adaptability of the model to specific network security fields and the lack of specialized corpus for network security tasks.

[0003] In order to overcome these challenges, it is necessary to build a large language model suitable for the field of network security and improve the model's adaptability to network security tasks through instruction fine-tuning. Instruction fine-tuning is a method of corpus construction and model fine-tuning based on specific tasks. By building a corpus related to network security tasks and using this corpus to fine-tune the model, the model can better understand the professional terms and concepts in the field of network security and improve its performance on network security tasks.

[0004] At present, there are relatively few studies on the construction method and system of fine-tuning corpus for large language model instructions in the field of network security. Most of the existing methods are fine-tuned based on general corpora, lacking specialized corpora for the field of network security. Therefore, developing a method and system for constructing fine-tuning corpus for large language model instructions specifically for the field of network security is of great significance for improving network security protection capabilities. Summary of the invention

[0005] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] In view of the above existing problems, the present invention is proposed.

[0007] Therefore, the present invention provides a method, system, electronic device and storage medium for constructing a large language model corpus to solve the problem that the existing methods are fine-tuned based on a general corpus and lack specialized corpus for the field of network security.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] In a first aspect, the present invention provides a method for constructing a large language model corpus, comprising:

[0010] Based on the initially set network security task seed pool, the first language model is used to determine whether the newly generated instruction is a classification task instruction;

[0011] If so, the classification task instruction is input into the first language model in an input-first manner to generate input-output pairs to form a classification task corpus;

[0012] If not, the non-classification task instructions are input into the first language model in an output-first manner to generate input-output pairs to form a non-classification task corpus;

[0013] Merge the classification task corpus and the non-classification task corpus, filter out low-quality and repeated instructions, and form a large language model instruction fine-tuning corpus in the field of network security.

[0014] As a preferred solution of the method for constructing a large language model corpus described in the present invention, wherein:

[0015] The initially set network security task seed pool includes the following steps:

[0016] Set 6 cybersecurity task seeds, and assign 1 instruction and 1 <input, output> instance to each security task;

[0017] In each step, 4 safety tasks are randomly sampled from the seed pool, 2 safety tasks are generated through the first large language model, and the 6 safety tasks are input into the first large language model according to the instruction generation prompt template to generate new task instructions, and the new task instructions are placed in the seed pool.

[0018] As a preferred solution of the method for constructing a large language model corpus described in the present invention, wherein:

[0019] The six safety tasks include a first safety task, a second safety task, a third safety task, a fourth safety task, a fifth safety task and a sixth safety task;

[0020] The first security task includes integrating log data from various security devices and applications, automatically parsing and analyzing them, and extracting key security event information;

[0021] The second security task includes performing static and dynamic security analysis on the given code, identifying and reporting potential vulnerabilities and security risks therein;

[0022] The third security task includes analyzing and integrating intelligence data from multiple sources, identifying connections and similarities and categorizing them in an orderly manner;

[0023] The fourth security task includes conducting in-depth analysis based on the diversity and complexity of ransomware threat intelligence to predict the attacker’s next action target;

[0024] The fifth security task includes using appropriate data and technical tools to conduct a comprehensive assessment of the organization’s external attack surface and identify potential attack entry points and attack paths;

[0025] The sixth security task includes simulating various attack methods and their corresponding defense strategies according to the specified attack and defense drill requirements.

[0026] As a preferred solution of the method for constructing a large language model corpus described in the present invention, wherein:

[0027] The step of determining whether the newly generated instruction is a classification task instruction by using the first language model comprises the following steps:

[0028] After generating new task instructions and putting the new task instructions into the seed pool, extracting all task instructions from the seed pool;

[0029] Designing a classification prompt template, wherein the classification prompt template includes 3 classification task instructions and 3 non-classification task instructions;

[0030] Input the classification prompt template and the task instruction to be classified into the first language model. The model will classify the new task instruction based on the example and determine whether it is a classification task;

[0031] The three classification task instructions include a first classification task instruction, a second classification task instruction and a third classification task instruction;

[0032] The three non-classified task instructions include a first non-classified task instruction, a second non-classified task instruction and a third non-classified task instruction;

[0033] The first classification task instruction includes analyzing a given code to determine whether it is malicious code;

[0034] The second classification task instruction includes analyzing whether there is malicious behavior from a given security log;

[0035] The third classification task instruction includes analyzing whether there is malicious traffic from a given traffic log;

[0036] The first unclassified task instruction includes conducting a comprehensive assessment of the organization’s external attack surface to identify potential attack entry points and attack paths;

[0037] The second unclassified task instruction includes integrating log data from multiple sources, performing automatic parsing and analysis, and extracting key information;

[0038] The third unclassified tasking directive includes the requirement to analyze and integrate intelligence data from multiple sources, identify connections and similarities, and organize them into orderly categories.

[0039] As a preferred solution of the method for constructing a large language model corpus described in the present invention, wherein:

[0040] The step of inputting the classification task instruction into the first language model in an input-first manner includes generating a sample according to an input-first mode for determining the first language model as an instruction of the classification task, inputting the instruction into the first language model to generate a corresponding input, and then generating a corresponding output according to the instruction and the input, and putting <instruction, input, output> into the classification task corpus as a sample;

[0041] The method of inputting the non-classification task instructions into the first largest language model in an output-first manner includes generating samples according to an output-first mode for instructions that are determined by the first largest language model as non-classification tasks, inputting the instructions into the first largest language model, generating corresponding outputs, and then generating corresponding inputs based on the instructions and outputs, and putting <instructions, inputs, outputs> as a sample into the non-classification task corpus.

[0042] As a preferred solution of the method for constructing a large language model corpus described in the present invention, wherein:

[0043] The filtering low-quality and repeated instructions include calculating the similarity between the newly generated corpus and the existing corpus in the corpus set, taking ROUGE-L as an indicator, and adding the corpus with a similarity lower than 0.7 to the corpus set.

[0044] In a second aspect, the present invention provides a large language model corpus construction system, comprising:

[0045] An instruction discrimination module is used to discriminate whether a newly generated instruction is a classification task instruction through a first language model based on an initially set network security task seed pool;

[0046] A first corpus generation module is used for inputting the classification task instruction into the first language model in an input-first manner to generate input-output pairs to form a classification task corpus set;

[0047] A second corpus generation module is used for, if not, inputting the non-classification task instruction into the first language model in an output-first manner to generate input-output pairs to form a non-classification task corpus set;

[0048] The corpus merging module is used to merge the classification task corpus and the non-classification task corpus, filter out low-quality and repeated instructions, and form a large language model instruction fine-tuning corpus in the field of network security.

[0049] As a preferred solution of the large language model corpus construction system of the present invention, wherein:

[0050] The setting module is used to set 6 types of network security task seeds, each of which is assigned 1 instruction and 1 <input, output> instance;

[0051] The sampling module is used to randomly sample 4 safety tasks from the seed pool in each step, generate 2 safety tasks through the first large language model, input the 6 safety tasks into the first large language model according to the instruction generation prompt template, generate new task instructions, and put the new task instructions into the seed pool.

[0052] In a third aspect, the present invention provides a computing device, comprising:

[0053] Memory, used to store programs;

[0054] A processor is used to execute the computer executable instructions, which, when executed by the processor, implement the steps of the method for constructing a large language model corpus.

[0055] In a fourth aspect, the present invention provides a computer-readable storage medium, comprising: when the program is executed by a processor, the steps of implementing the method for constructing a large language model corpus are implemented.

[0056] Beneficial effects of the present invention: The present invention ensures that the generated corpus is highly relevant and professional by designing a specific network security task seed pool and using high-quality <input, output> examples as the basis. At the same time, by filtering low-quality and repeated instructions, the overall quality of the corpus is further improved; by using two different generation methods, classified task instructions and non-classified task instructions, the corpus construction is more flexible and efficient, and can better meet the needs of different types of network security tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0058] Figure 1 A schematic diagram of the basic flow of a method for constructing a large language model corpus provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0061] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0062] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0063] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0064] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0065] Example 1

[0066] Reference Figure 1, is an embodiment of the present invention, and provides a method for constructing a large language model corpus, such as Figure 1 As shown, Figure 1 As shown, the following steps are included:

[0067] S1: Based on the initially set network security task seed pool, the first language model is used to determine whether the newly generated instruction is a classification task instruction, including the following steps:

[0068] Set up 6 network security task seed pools, and assign 1 instruction and 1 <input, output> instance to each security task.

[0069] In each step, 4 safety tasks are randomly sampled from the seed pool, 2 safety tasks are generated through the first large language model, and the 6 safety tasks are input into the first large language model according to the instruction generation prompt template to generate new task instructions, and the new task instructions are placed in the seed pool.

[0070] In the embodiment of the present invention, the six types of security task seed pools and their corresponding instructions include:

[0071] The first security task: security log analysis;

[0072] Instructions: Integrate log data from multiple security devices and applications and automatically parse and analyze to extract key information; specific steps include data collection, log standardization, event identification and key information extraction to ensure that potential security threats can be discovered and responded to in a timely manner.

[0073] The second security task: code vulnerability mining;

[0074] Instructions: Please perform static or dynamic security analysis on the following code to discover potential vulnerabilities or risks in the code; specific steps include code review, static analysis, dynamic testing, and vulnerability verification to provide detailed repair suggestions and enhance the security of the code.

[0075] The third security task: multi-source intelligence integration;

[0076] Instructions: Please find the connections and similarities between the following intelligence, classify them in order, and realize the automation of intelligence sorting and integration; specific steps include data collection, intelligence analysis, correlation identification and automatic classification to provide reliable intelligence support and assist decision-making.

[0077] Fourth security task: extortion intelligence mining;

[0078] Instructions: Please analyze the diversity of ransomware threat intelligence sources and complex threat characteristics to predict the attacker's possible next attack target; specific steps include intelligence collection, feature analysis, behavior prediction and prevention measures to reduce the risk of being ransomed.

[0079] Fifth security task: external attack surface assessment;

[0080] Instructions: Please use appropriate data and technical tools to evaluate the external attack surface of the organization and identify potential attack points and attack paths; specific steps include data collection, attack surface identification, path analysis and risk assessment to develop effective defense strategies and improve overall security.

[0081] The sixth security task: intelligent attack and defense confrontation;

[0082] Instructions: Please simulate the attack methods and corresponding defense strategies of various attacks according to the following attack and defense drill requirements. The specific steps include demand analysis, attack simulation, defense strategy formulation and effect evaluation, so as to continuously optimize and improve security measures and enhance the system's anti-attack capabilities.

[0083] In an optional embodiment, the security task may also include malware detection, identity authentication and access control, and data encryption and decryption;

[0084] In an optional embodiment, malware detection includes analyzing the following files, determining whether they are malware, and providing a detailed detection report. This task requires analyzing a given file, determining whether it is malware, and providing a detailed detection report, including file characteristics, behavior analysis, and detection results.

[0085] In an optional embodiment, authentication and access control includes designing an authentication and access control system to ensure that only authorized users can access sensitive resources. This task requires designing an authentication and access control system to ensure that only authorized users can access sensitive resources, including user authentication, permission management, and access logging.

[0086] In an optional embodiment, data encryption and decryption includes encrypting the following data and providing corresponding decryption methods to ensure the security of data during transmission. This task requires encrypting the given data and providing corresponding decryption methods to ensure the security of data during transmission, including selecting appropriate encryption algorithms and key management schemes.

[0087] It should be noted that the purpose of setting up a 6-task seed pool is to ensure the simplicity and manageability of the project in the initial stage while covering the core tasks of network security. As the project progresses, more tasks can be gradually added as needed to make the seed pool richer and more comprehensive. This approach helps to quickly start the project, verify model performance, and lay a solid foundation for subsequent expansion and optimization.

[0088] In an embodiment of the present invention, the instruction generates a prompt template including:

[0089] “

[0090] Several instruction examples are provided below. Please refer to their content and format to generate an instruction and stop generating after this instruction is generated.

[0091] Instructions: Consolidate and automatically parse and analyze log data from multiple security devices and applications to extract key information.

[0092] Instructions: Please perform static or dynamic security analysis on the following code to find potential vulnerabilities or risks in the code.

[0093] Instructions: Please find the connections and similarities between the following intelligence, classify them in order, and realize the automation of intelligence sorting and integration.

[0094] Instructions: Please analyze the diversity of ransomware threat intelligence sources and complex threat characteristics to predict the attacker's next possible attack target.

[0095] Instructions: Please use appropriate data and technical tools to assess the external attack surface of the organization and identify potential attack entry points and attack paths.

[0096] Instructions: Please simulate various attack methods and corresponding defense strategies according to the following attack and defense drill requirements.

[0097] instruction:{}.

[0098] ”

[0099] In an optional embodiment, the first language model may be BERT, T5, or GPT4;

[0100] In an optional embodiment, BERT is a bidirectional Transformer model that is widely used in natural language processing tasks such as text classification and named entity recognition. It improves the accuracy and robustness of the model by understanding text through bidirectional context, and is suitable for tasks that require understanding the context, such as discrimination of classification tasks and generation of input-output pairs.

[0101] In an optional embodiment, T5 converts all NLP tasks into text-to-text conversion tasks, unifies the task format, and is applicable to multiple types of NLP tasks, including classification tasks and generation tasks.

[0102] In the embodiment of the present invention, GPT4 is used to determine whether the newly generated instruction is a classification task instruction. If it is not a classification task instruction, it is a non-classification task instruction;

[0103] It should be noted that GPT-4 inherits the powerful generation capabilities of the GPT series of models and is able to generate high-quality text, including complex instructions and input-output pairs. This is critical for building a high-quality security task instruction library, and is able to understand complex contextual information, which makes it more accurate and reliable when handling classification tasks and generation tasks. It can capture subtle semantic differences and improve the accuracy of tasks, supports multimodal input, and can handle multiple types of data such as text and images. This makes it more flexible and powerful when handling complex network security tasks. It has been trained on large-scale training data, which gives it extensive knowledge and a rich corpus, and is able to handle various types of tasks, including rare and complex tasks. It can be highly customized through fine-tuning and prompt engineering, so that it can adapt to specific network security tasks and application scenarios. This makes it more flexible and efficient in practical applications, has strong reasoning capabilities, and is able to perform complex logical reasoning in generation tasks and classification tasks, improving the accuracy and reliability of tasks.

[0104] In an optional embodiment, the classification prompt template may include 1 classification instruction and 1 non-classification instruction, or may be a single task type template, or may include 3 classification instructions and 3 non-classification instructions;

[0105] In an optional embodiment, one classification instruction and one non-classification instruction are included, and the number of examples is insufficient. This small sample learning method may affect the learning effect and classification accuracy of the model because the model does not have enough examples for effective learning and generalization.

[0106] In an optional embodiment, the single task type template classification template mainly focuses on code analysis and traffic log analysis, lacking other types of tasks. This single task type example may lead to insufficient generalization ability of the model because it does not cover other important tasks in the field of network security.

[0107] In an embodiment of the present invention, determining whether a newly generated instruction is a classification task instruction includes:

[0108] The classification prompt template contains 3 classification instructions and 3 non-classification instructions. According to the classification prompt template, GPT4 is used to classify the instructions in a few-shot manner.

[0109] In the embodiment of the present invention, the classification prompt template is:

[0110] “

[0111] First classification instruction: Please analyze whether the following code is malicious code. Is it a classification task: Yes.

[0112] First non-classified instruction: Please use appropriate data and technical tools to evaluate the external attack surface of the organization and identify potential attack entry points and attack paths. Is it a classified task: No.

[0113] Second classification instruction: Please analyze the following security logs to see if they contain malicious behavior. Is this a classified task: Yes.

[0114] Second unclassified instruction: Integrate log data from multiple security devices and applications and automatically parse and analyze to extract key information. Is it a classified task: No.

[0115] Third non-classified instruction: Please find the connections and similarities between the following intelligence, classify them in order, and realize the automation of intelligence sorting and integration. Is it a classified task: No.

[0116] Third classification instruction: Please analyze the following traffic logs to see if it is malicious traffic. Is it a classification task: Yes.

[0117] Instruction: {}. Whether it is a classification task: {}.

[0118] ”

[0119] It should be noted that the classification prompt template of the present invention contains 3 classification tasks and 3 non-classification tasks, and each task is clearly marked whether it is a classification task. These tasks cover various types in the field of network security, such as malware detection, traffic analysis, log parsing, security configuration, etc. By providing diverse examples, the model can better understand and distinguish between classification tasks and non-classification tasks, and improve the accuracy and generalization ability of classification.

[0120] S2: If yes, the classification task instruction is input into the first language model in an input-first manner to generate input-output pairs to form a classification task corpus;

[0121] In an embodiment of the present invention, the classification task instruction is input into GPT4 in an input-first manner, an input-output pair is generated, and a classification task corpus is formed, including: the instruction determined by GPT4 as a classification task is generated as an example in an input-first mode. The instruction is input into GPT4, and it is allowed to generate the corresponding input first, and then the corresponding output is generated according to the instruction and the input, and <instruction, input, output> is put into the classification task corpus as a sample.

[0122] In an embodiment of the present invention, the classification task instruction is input into the first language model in an input-first manner to generate an input-output pair to form a classification task corpus, including: the first language model is judged as an instruction of the classification task to generate a sample in an input-first mode. The instruction is input into the first language model to generate a corresponding input first, and then the corresponding output is generated according to the instruction and the input, and <instruction, input, output> is put into the classification task corpus as a sample.

[0123] In an embodiment of the present invention, the classification task instruction is input into GPT4 in an input-first manner, an input-output pair is generated, and a classification task corpus is formed, including: the instruction determined by GPT4 as a classification task is generated as an example in an input-first mode. The instruction is input into GPT4, and it is allowed to generate the corresponding input first, and then the corresponding output is generated according to the instruction and the input, and <instruction, input, output> is put into the classification task corpus as a sample.

[0124] S3: If not, the non-classification task instructions are input into the first language model in an output-first manner to generate input-output pairs to form a non-classification task corpus;

[0125] In an embodiment of the present invention, a non-classified task instruction is input into the first language model in an output-first manner to generate an input-output pair to form a non-classified task corpus. This includes: generating samples according to an output-first mode for instructions that are determined by the first language model as non-classified tasks. The instruction is input into the first language model to generate the corresponding output first, and then the corresponding input is generated according to the instruction and the output, and <instruction, input, output> is put into the non-classified task corpus as a sample.

[0126] In an embodiment of the present invention, non-classified task instructions are input into GPT4 in an output-first manner to generate input-output pairs to form a non-classified task corpus. This includes: generating samples according to an output-first mode for instructions that GPT4 determines as non-classified tasks. Input the instructions into GPT4, let it generate the corresponding output first, and then generate the corresponding input based on the instructions and output, and put <instructions, input, output> as a sample into the non-classified task corpus.

[0127] S4: Merge the classification task corpus and the non-classification task corpus, filter out low-quality and repeated instructions, and form a large language model instruction fine-tuning corpus in the field of network security.

[0128] In an embodiment of the present invention, a classification task corpus and a non-classification task corpus are merged, low-quality and repeated instructions are filtered out, and a large language model instruction fine-tuning corpus in the field of network security is formed, including:

[0129] Merge the classification task corpus and the non-classification task corpus, calculate the similarity between the newly generated corpus and the existing corpus in the corpus, and use ROUGE-L as the indicator to add the corpus with a score lower than 0.7 to the corpus. Fusion of the input and output corresponding to the classification task instructions and the input and output corresponding to the non-classification task instructions to form a large language model instruction fine-tuning corpus in the field of network security.

[0130] This embodiment also provides a large language model corpus construction system, including:

[0131] An instruction creation module is used to build a security task instruction library based on the initially set network security task seed pool;

[0132] An instruction discrimination module, used to discriminate whether the newly generated instruction is a classification task instruction through the first language model, and if it is not a classification task instruction, it is a non-classification task instruction;

[0133] A first corpus generation module is used to input the classification task instructions into the first language model in an input-first manner to generate input-output pairs to form a classification task corpus set;

[0134] The second corpus generation module is used to input the non-classification task instructions into the first language model in an output-first manner to generate input-output pairs to form a non-classification task corpus set;

[0135] The corpus merging module is used to merge the classification task corpus and the non-classification task corpus, filter out low-quality and repeated instructions, and form a large language model instruction fine-tuning corpus in the field of network security.

[0136] Furthermore, it also includes:

[0137] The setting module is used to set 6 types of network security task seeds, each of which is assigned 1 instruction and 1 <input, output> instance;

[0138] The sampling module is used to randomly sample 4 safety tasks from the seed pool in each step, generate 2 safety tasks through the first large language model, input the 6 safety tasks into the first large language model according to the instruction generation prompt template, generate new task instructions, and put the new task instructions into the seed pool.

[0139] Furthermore, it also includes:

[0140] Memory, used to store programs;

[0141] A processor is used to load the program to execute the method for constructing a large language model corpus.

[0142] This embodiment also provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the method for constructing a large language model corpus.

[0143] The storage medium proposed in this embodiment and the large language model corpus construction method proposed in the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0144] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (RandomAccess Memory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.

[0145] Example 2

[0146] This is another embodiment of the present invention, which provides a large language model corpus construction system, including an instruction discrimination module, a corpus generation module and a corpus merging module;

[0147] In an embodiment of the present invention, the instruction discrimination module includes a network security task seed pool based on an initial setting, and determines whether the newly generated instruction is a classification task instruction through a first language model; In an embodiment of the present invention, the instruction discrimination module includes a setting module and a sampling module;

[0148] In the embodiment of the present invention, the setting module includes setting 6 kinds of network security task seeds, each security task is assigned 1 instruction and 1 <input, output> instance;

[0149] In an embodiment of the present invention, the sampling module includes randomly sampling 4 security tasks from the seed pool in each step, generating 2 security tasks through the first large language model, inputting 6 security tasks into the first large language model according to the instruction generation prompt template, generating new task instructions, and placing the new task instructions into the seed pool. Among them, the 6 security task seed pools and their corresponding instructions include: Security Task 1: Security Log Parsing, Instruction: Integrate log data from multiple security devices and applications and automatically parse and analyze them to extract key information; Security Task 2: Code Vulnerability Mining, Instruction: Please perform static or dynamic security analysis on the following code to discover potential vulnerabilities or risks in the code; Security Task 3: Multi-source Intelligence Integration, Instruction: Please find the connection points and similarities between the following intelligence, classify them in order, and realize the automation of intelligence sorting and integration; Security Task 4: Ransomware Intelligence Mining, Instruction: Please analyze the diversity and complex threat characteristics of the ransomware threat intelligence sources to predict the attacker's possible next attack target; Security Task 5: External Attack Surface Assessment, Instruction: Please use appropriate data and technical tools to assess the organization's external attack surface and identify potential attack entries and attack paths; Security Task 6: Intelligent Attack and Defense Confrontation, Instruction: Please simulate the attack methods and corresponding defense strategies of various attacks according to the following attack and defense drill requirements.

[0150] In an embodiment of the present invention, determining whether a newly generated instruction is a classified task instruction by using the first language model includes setting a classified prompt template, wherein the classified prompt template includes 3 classified instructions and 3 non-classified instructions. According to the classified prompt template, the instructions are classified using GPT4 in a few-shot manner.

[0151] In an embodiment of the present invention, the corpus generation module includes a first corpus generation module and a second corpus generation module;

[0152] In an embodiment of the present invention, the first corpus generation module includes: if it is a classification task instruction, inputting the classification task instruction into the first large language model in an input-first manner, generating an input-output pair, and forming a classification task corpus set;

[0153] In an embodiment of the present invention, inputting the classification task instruction into the first language model in an input-first manner includes generating samples for the instruction determined by GPT4 as a classification task in an input-first mode. Input the instruction into GPT4, let it generate the corresponding input first, and then generate the corresponding output according to the instruction and the input, and put <instruction, input, output> as a sample into the classification task corpus.

[0154] In an embodiment of the present invention, the second corpus generation module includes: if it is not a classification task instruction, inputting the non-classification task instruction into the first large language model in an output-first manner, generating an input-output pair, and forming a non-classification task corpus set;

[0155] In an embodiment of the present invention, inputting the non-classification task instruction into the first language model in an output-first manner includes generating samples for the instruction determined by GPT4 as a non-classification task in an output-first mode. Input the instruction into GPT4, let it generate the corresponding output first, and then generate the corresponding input according to the instruction and the output, and put <instruction, input, output> as a sample into the non-classification task corpus.

[0156] In an embodiment of the present invention, the corpus merging module includes merging a classification task corpus and a non-classification task corpus, filtering low-quality and repeated instructions, and forming a large language model instruction fine-tuning corpus in the field of network security.

[0157] In the embodiment of the present invention, the corpus merging module also includes calculating the similarity between the newly generated corpus and the existing corpus in the corpus set, taking ROUGE-L as the index, and adding the corpus with a similarity lower than 0.7 to the corpus set. The input and output corresponding to the classification task instructions and the input and output corresponding to the non-classification task instructions are integrated to form a large language model instruction fine-tuning corpus set in the field of network security.

[0158] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for constructing a large language model corpus, characterized in that: include: Based on the initially set network security task seed pool, the first language model is used to determine whether the newly generated instruction is a classification task instruction; If so, the classification task instruction is input into the first language model in an input-first manner to generate input-output pairs to form a classification task corpus; If not, the non-classification task instructions are input into the first language model in an output-first manner to generate input-output pairs to form a non-classification task corpus; Merge the classification task corpus and the non-classification task corpus, filter out low-quality and repeated instructions, and form a large language model instruction fine-tuning corpus in the field of network security.

2. The method for constructing a large language model corpus according to claim 1, wherein: The initially set network security task seed pool includes the following steps: Set 6 cybersecurity task seeds, and assign 1 instruction and 1 <input, output> instance to each security task; In each step, 4 safety tasks are randomly sampled from the seed pool, 2 safety tasks are generated through the first large language model, and the 6 safety tasks are input into the first large language model according to the instruction generation prompt template to generate new task instructions, and the new task instructions are placed in the seed pool.

3. The method for constructing a large language model corpus according to claim 2, wherein: The six safety tasks include a first safety task, a second safety task, a third safety task, a fourth safety task, a fifth safety task and a sixth safety task; The first security task includes integrating log data from various security devices and applications, automatically parsing and analyzing them, and extracting key security event information; The second security task includes performing static and dynamic security analysis on the given code, identifying and reporting potential vulnerabilities and security risks therein; The third security task includes analyzing and integrating intelligence data from multiple sources, identifying connections and similarities and categorizing them in an orderly manner; The fourth security task includes conducting in-depth analysis based on the diversity and complexity of ransomware threat intelligence to predict the attacker’s next action target; The fifth security task includes using appropriate data and technical tools to conduct a comprehensive assessment of the organization’s external attack surface and identify potential attack entry points and attack paths; The sixth security task includes simulating various attack methods and their corresponding defense strategies according to the specified attack and defense drill requirements.

4. The method for constructing a large language model corpus according to claim 3, wherein: The step of determining whether the newly generated instruction is a classification task instruction by using the first language model comprises the following steps: After generating new task instructions and putting the new task instructions into the seed pool, extracting all task instructions from the seed pool; Designing a classification prompt template, wherein the classification prompt template includes 3 classification task instructions and 3 non-classification task instructions; Input the classification prompt template and the task instruction to be classified into the first language model. The model will classify the new task instruction based on the example and determine whether it is a classified task instruction; The three classification task instructions include a first classification task instruction, a second classification task instruction and a third classification task instruction; The three non-classified task instructions include a first non-classified task instruction, a second non-classified task instruction and a third non-classified task instruction; The first classification task instruction includes analyzing a given code to determine whether it is malicious code; The second classification task instruction includes analyzing whether there is malicious behavior from a given security log; The third classification task instruction includes analyzing whether there is malicious traffic from a given traffic log; The first unclassified task instruction includes conducting a comprehensive assessment of the organization’s external attack surface to identify potential attack entry points and attack paths; The second unclassified task instruction includes integrating log data from multiple sources, performing automatic parsing and analysis, and extracting key information; The third unclassified tasking directive includes the requirement to analyze and integrate intelligence data from multiple sources, identify connections and similarities, and organize them into orderly categories.

5. The method for constructing a large language model corpus according to claim 4, wherein: The step of inputting the classification task instruction into the first language model in an input-first manner includes: The first largest language model is judged as the instruction of the classification task to generate samples in the input-first mode, the instruction is input into the first largest language model to generate the corresponding input, and then the corresponding output is generated according to the instruction and input, and <instruction, input, output> is put into the classification task corpus as a sample; The step of inputting the non-classification task instructions into the first language model in an output-first manner comprises: The instructions determined by the first largest language model as non-classification tasks generate samples in an output-first mode, the instructions are input into the first largest language model, the corresponding output is generated, and then the corresponding input is generated based on the instructions and output, and <instruction, input, output> is put into the non-classification task corpus as a sample.

6. The method for constructing a large language model corpus according to claim 5, characterized in that: The filtering low-quality and repeated instructions include calculating the similarity between the newly generated corpus and the existing corpus in the corpus set, taking ROUGE-L as an indicator, and adding the corpus with a similarity lower than 0.7 to the corpus set.

7. A system for implementing a method for constructing a large language model corpus, characterized in that: An instruction discrimination module is used to discriminate whether a newly generated instruction is a classification task instruction through a first language model based on an initially set network security task seed pool; A first corpus generation module is used for inputting the classification task instruction into the first language model in an input-first manner to generate input-output pairs to form a classification task corpus set; A second corpus generation module is used for, if not, inputting the non-classification task instruction into the first language model in an output-first manner to generate input-output pairs to form a non-classification task corpus set; The corpus merging module is used to merge the classification task corpus and the non-classification task corpus, filter out low-quality and repeated instructions, and form a large language model instruction fine-tuning corpus in the field of network security.

8. The large language model corpus construction system according to claim 7, characterized in that: The setting module is used to set 6 types of network security task seeds, each of which is assigned 1 instruction and 1 <input, output> instance; The sampling module is used to randomly sample 4 safety tasks from the seed pool in each step, generate 2 safety tasks through the first large language model, input the 6 safety tasks into the first large language model according to the instruction generation prompt template, generate new task instructions, and put the new task instructions into the seed pool.

9. An electronic device, characterized in that: include: Memory, used to store programs; A processor, configured to load the program to execute the steps of the method for constructing a large language model corpus as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the steps of the method for constructing a large language model corpus as described in any one of claims 1 to 6 are implemented.

Images