Construction method and device for fine tuning data set of large language model and electronic equipment
By using localized security knowledge base and self-guided generation methods in the fine tuning of large language models in the field of network security, the instruction set is filtered and expanded, and diversity and quality assessment is carried out, and the problems of insufficient data and low quality are solved, and the problem of insufficient data is achieved. Efficient and secure fine-tuning dataset construction.
Patent Information
- Application Number
- CN202411404405.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-02-07
AI Technical Summary
The prior art faces the problems of insufficient data and low quality in fine-tuning of large language models in the field of network security, resulting in limited generalization capabilities of the model and risks of data leakage and privacy.
By calling the security knowledge base built on local network security text data, the query vectors of the initial seed instruction set are retrieved, the candidate vector set with high similarity is selected, and the instruction set is generated by self-guided through a large language model to expand the instruction set. Then, the diversity and quality of the extended instruction set is measured, evaluated using Rouge_L scores, and finally a high-quality fine-tuned data set is obtained.
Reliance on external data sources is reduced through a localized security knowledge base, data breaches and privacy risks, data scarcity issues are addressed, and high quality and accuracy of fine-tuned data sets are ensured.
Smart Images
Figure CN119397270A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network security technology, and in particular to a method, device and electronic device for constructing a large language model fine-tuning data set. Background Art
[0002] Currently, large language models (LLMs) are increasingly used in different fields, but fine-tuning models for specific fields (such as network security) still faces the problem of insufficient data. Although studies have shown that data quality is more important than data quantity, there is currently a lack of high-quality training data for the field of network security. Summary of the invention
[0003] This application provides a method, device and electronic device for constructing a large language model fine-tuning dataset. The specific scheme is as follows:
[0004] The present application embodiment provides a method for constructing a large language model fine-tuning dataset, including:
[0005] Calling a security knowledge base to retrieve a query vector corresponding to the initial seed instruction set, and screening out a candidate vector set having a similarity with the query vector greater than a first preset threshold from a plurality of vector sets stored in the security knowledge base, wherein the security knowledge base is a knowledge base pre-built based on local network security text data;
[0006] Inputting a candidate instruction set corresponding to the candidate vector set into a large language model, and expanding the candidate instruction set by a self-guided generation method to obtain an expanded instruction set;
[0007] Measuring the diversity of the extended instruction set, and screening out a distributed instruction set having an entropy value less than a second preset threshold from the extended instruction set, wherein the entropy value is inversely proportional to the diversity;
[0008] The Rouge_L score is used to evaluate the quality of the distributed instruction set, and instructions that do not meet the quality standards are filtered out to obtain a fine-tuning dataset, which is used to train the large language model.
[0009] Optionally, in the embodiment of the present application, the Rouge_L score is used to perform quality assessment on the scattered instruction set, and instructions with substandard quality are filtered out to obtain a fine-tuning data set, including:
[0010] Obtaining the longest common subsequence length of the scattered instruction set and the reference instruction set;
[0011] Obtaining the Rouge_L score of the scattered instruction set according to the longest common subsequence length and the reference instruction set;
[0012] From the scattered instruction set, instructions whose Rouge_L scores are greater than or equal to a preset minimum score threshold are filtered out to obtain a fine-tuning data set.
[0013] Optionally, in the embodiment of the present application, measuring the diversity of the extended instruction set includes:
[0014] Obtaining a probability distribution of the extended instruction set;
[0015] Obtaining an entropy value of the extended instruction set according to the probability distribution;
[0016] The diversity of the extended instruction set is measured according to the entropy value.
[0017] Optionally, in the embodiment of the present application, before calling the security knowledge base to retrieve the query vector corresponding to the initial seed instruction set, the apparatus further includes a construction unit, wherein the construction unit is specifically configured to:
[0018] Acquire local network security text data, and perform vectorization processing on the network security text data to obtain a corresponding vector set;
[0019] A security knowledge base is constructed based on the vector set and the original data corresponding to the vector set.
[0020] Optionally, in the embodiment of the present application, the expansion unit is specifically used for:
[0021] Determining the instruction set to be selected corresponding to the candidate vector set;
[0022] The candidate instruction set is input into the large language model, and the candidate instruction set is expanded through multiple rounds of iterations by calling the generation function of the prompt word project to generate the expanded instruction set.
[0023] Accordingly, an embodiment of the present application provides a device for constructing a large language model fine-tuning dataset, including:
[0024] A retrieval unit, configured to call a security knowledge base to retrieve a query vector corresponding to an initial seed instruction set, and select a candidate vector set having a similarity with the query vector greater than a first preset threshold from a plurality of vector sets stored in the security knowledge base, wherein the security knowledge base is a knowledge base pre-constructed based on local network security text data;
[0025] An expansion unit, configured to input a candidate instruction set corresponding to the candidate vector set into a large language model, and expand the candidate instruction set by a self-guided generation method to obtain an expanded instruction set;
[0026] A measuring unit, used for measuring the diversity of the extended instruction set, and screening out a distributed instruction set having an entropy value less than a second preset threshold from the extended instruction set, wherein the entropy value is inversely proportional to the diversity;
[0027] The quality assessment unit is used to use the Rouge_L score to perform quality assessment on the scattered instruction set, filter out instructions that do not meet the quality standards, and obtain a fine-tuning data set, where the fine-tuning data set is used to train the large language model.
[0028] Optionally, in an embodiment of the present application, the quality assessment unit is specifically used to:
[0029] Obtaining the longest common sequence length between the distributed instruction set and the reference instruction set;
[0030] Obtaining the Rouge_L score of the scattered instruction set according to the longest common sequence length and the reference instruction set;
[0031] From the scattered instruction set, instructions whose Rouge_L scores are greater than or equal to a preset minimum score threshold are filtered out to obtain a fine-tuning data set.
[0032] Optionally, in the embodiment of the present application, the measurement unit is specifically used for:
[0033] Obtaining a probability distribution of the extended instruction set;
[0034] Obtaining an entropy value of the extended instruction set according to the probability distribution;
[0035] The diversity of the extended instruction set is measured according to the entropy value.
[0036] Optionally, in the embodiment of the present application, the expansion unit is specifically used for:
[0037] Determining the instruction set to be selected corresponding to the candidate vector set;
[0038] The candidate instruction set is input into the large language model, and the candidate instruction set is expanded through multiple rounds of iterations by calling the generation function of the prompt word project to generate the expanded instruction set.
[0039] Accordingly, an embodiment of the present application provides an electronic device, including:
[0040] Memory, used to store computer programs;
[0041] The processor is used to implement the method steps described in any one of the above items when executing the computer program stored in the memory.
[0042] Accordingly, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method steps described in any one of the above items are implemented.
[0043] The beneficial effects of the embodiments of the present application are as follows:
[0044] The embodiment of the present application provides a method, device and electronic device for constructing a large language model fine-tuning data set, wherein, through the construction method, first, the security knowledge base is called to retrieve the query vector corresponding to the initial seed instruction set, and the candidate vector set whose similarity with the query vector is greater than the first preset threshold is screened out from the multiple vector sets stored in the security knowledge base; wherein the security knowledge base is a knowledge base pre-constructed based on local network security text data. In this way, through the localized security knowledge base, the dependence on external data sources is reduced, and the data leakage and privacy risks are reduced. Then, the candidate instruction set corresponding to the candidate vector set is input into the large language model, and the candidate instruction set is expanded by the self-guided generation method to obtain the extended instruction set. In this way, the problem of data scarcity is solved.
[0045] Then, the diversity of the extended instruction set is measured, and the scattered instruction set whose entropy value is less than the second preset threshold is screened out from the extended instruction set, wherein the smaller the entropy value, the better the diversity of the extended instruction set. In this way, the instruction set that meets the diversity requirements can be screened out. Then, the Rouge_L score is used to evaluate the quality of the scattered instruction set, and the instructions that do not meet the quality standards are filtered out to obtain a data set for fine-tuning the large language model. In this way, the high quality and accuracy of the data set are ensured. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 A schematic diagram of a method for generating fine-tuning data based on a public dataset in the related art;
[0047] Figure 2 A method flow chart of a method for constructing a large language model fine-tuning dataset provided in an embodiment of the present application;
[0048] Figure 3 for Figure 2 The method flow chart of step S104;
[0049] Figure 4 for Figure 2 The method flow chart of step S103;
[0050] Figure 5 For Figure 2 One of the method flow charts before step S101;
[0051] Figure 6 for Figure 2 The method flow chart of step S102 in FIG.
[0052] Figure 7 A flowchart of one of the methods for constructing a large language model fine-tuning dataset provided in an embodiment of the present application;
[0053] Figure 8 A structural block diagram of a device for constructing a large language model fine-tuning dataset provided in an embodiment of the present application;
[0054] Fig. 9 This is a structural block diagram of one of the electronic devices provided in the embodiments of the present application. DETAILED DESCRIPTION
[0055] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be clearly and completely described below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. And in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. Based on the described embodiments of the present application, all other embodiments obtained by ordinary technicians in the field without creative work belong to the scope of protection of this application.
[0056] Unless otherwise defined, the technical terms or scientific terms used in this application should be understood by people with ordinary skills in the field to which this application belongs. The words "include" or "comprise" and the like used in this application mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, but do not exclude other elements or objects.
[0057] In the related art, the application of large language models in the field of network security mainly relies on public data sets for fine-tuning. Figure 1The figure shows a schematic diagram of a fine-tuning data generation method based on a public dataset; first, collect common datasets; then, preprocess the data; then, label and preprocess the data; specifically, label and preprocess the preprocessed data; then, generate the dataset; thereby generating a dataset for fine-tuning. Among them, common datasets include Advanced Persistent Threats (APT) attack reports, public vulnerability databases, network traffic analysis data, etc. Exemplarily, the public vulnerability database can be Common Vulnerabilities & Exposures (CVE). These datasets are usually used to train models to detect and defend against known network threats. The methods of data labeling and preprocessing mainly include: (1) Manual labeling: security experts are usually required to manually label the data, and the labeling content includes attack type, victim object, attack method, etc.; this process is time-consuming and labor-intensive and is susceptible to human errors; (2) Automated tools: Some datasets are labeled by automated tools. Exemplarily, natural language processing (NLP) technology can be used to extract key information from text, but the accuracy and precision are still limited.
[0058] As a result, the method of building a secure large model fine-tuning dataset based on public data has the following main problems:
[0059] (1) Data scarcity: In the field of cybersecurity, high-quality datasets required to train large language models are extremely scarce. Public datasets are usually limited in size and do not cover a comprehensive range of threat types and protection scenarios, which limits the generalization ability of the model.
[0060] (2) Data security: The sources of some data sets are unknown or even come from unreliable channels. Local data lacks a secure processing method, which poses a risk of user data being abused or leaked.
[0061] (3) Lack of data screening and evaluation mechanisms: Currently, the fine-tuning datasets of large language models in the security field lack strict screening and evaluation mechanisms, resulting in the inability to control data quality and diversity.
[0062] In view of this, the embodiments of the present application provide a method, device and electronic device for constructing a large language model fine-tuning dataset, which are used to improve the efficiency of data generation, enhance data privacy and security, effectively improve the task processing capabilities of large language models in the field of network security, and provide high-quality data support for practical applications.
[0063] like Figure 2 As shown, the embodiment of the present application provides a method for constructing a large language model fine-tuning dataset, including:
[0064] S101: calling a security knowledge base to retrieve a query vector corresponding to an initial seed instruction set, and selecting a candidate vector set having a similarity with the query vector greater than a first preset threshold from a plurality of vector sets stored in the security knowledge base, wherein the security knowledge base is a knowledge base pre-built based on local network security text data;
[0065] In the specific implementation process, the security knowledge base is a knowledge base pre-built based on local network security text data. Exemplarily, the local network security text data can be an expert knowledge base, threat intelligence, network traffic analysis data, or system-on-chip (SoC) alarm information, which is not limited here. In this way, the security knowledge base is a localized knowledge base, which reduces dependence on external data sources and reduces data leakage and privacy risks.
[0066] In one exemplary embodiment, first, an initial seed instruction set is generated. For example, key categories and subcategories in network security tasks are defined, and an initial seed instruction set S is generated for each category. 0 , which can be written as: S 0 ={s 1 ,s 2 ,…,s k}, k is an integer greater than 2. For example, the key network security categories are "intrusion detection" and "vulnerability scanning", and the initial seed instruction set S 0 Then, the security knowledge base is called to retrieve the query vector corresponding to the initial seed instruction set, and a candidate vector set whose similarity with the query vector is greater than a first preset threshold is selected from the multiple vector sets stored in the security knowledge base. 0 , generate a query vector vq, and call the security knowledge base for retrieval. Exemplarily, an artificial neural network (ANN) algorithm can be used for vector retrieval to find the k vectors most similar to the query vector vq and obtain a candidate vector set. The specific value of the first preset threshold can be set according to actual application needs and is not limited here.
[0067] S102: inputting the candidate instruction set corresponding to the candidate vector set into the large language model, and expanding the candidate instruction set through a self-guided generation method to obtain an expanded instruction set;
[0068] In the specific implementation process, a large language model (LLM) application programming interface (Application Programming Interface, API) can be used to generate a candidate instruction set based on the candidate vector set. The candidate instruction set is expanded by a self-instruct generation method to obtain an extended instruction set. For example, the current candidate instruction set is S i , generate a new instruction set S through LLM i+1 , satisfying the formula: S i+1 =S i ∪Generate(S i ), the new instruction set S i+1 is the extended instruction set after expansion. Among them, the generation function Generate(S i ) can be generated by multiple rounds of iterations: Generate(S i )=(LLM(s j )|s j ∈S i ,j∈[1,n]). Through multiple rounds of iterative generation, the size of the instruction set is gradually expanded, the problem of data scarcity is solved, and a guarantee is provided for the generation of high-quality instruction data.
[0069] S103: measuring the diversity of the extended instruction set, and selecting a distributed instruction set having an entropy value less than a second preset threshold from the extended instruction set, wherein the entropy value is inversely proportional to the diversity;
[0070] In the specific implementation process, after obtaining the extended instruction set, the diversity of the extended instruction set can be measured to obtain an instruction set with an entropy value less than the second preset threshold, and the instruction set is used as a distributed instruction set, thereby providing a guarantee for the diversity of the instruction set. Among them, the smaller the entropy value, the better the diversity, and the larger the entropy value, the worse the diversity. Exemplarily, the second preset threshold can be set according to actual application needs, which is not limited here.
[0071] S104: Use the Rouge_L score to evaluate the quality of the distributed instruction set, filter out instructions that do not meet the quality standards, and obtain a fine-tuning dataset, where the fine-tuning dataset is used to train the large language model.
[0072] In the specific implementation process, after obtaining the scattered instruction set, the Rouge_L score can be used to evaluate the quality of the scattered instruction set, filter out instructions that do not meet the quality standards, and obtain a fine-tuning data set. In other words, the Rouge_L score can be used as an evaluation criterion. Among them, the Rouge_L score is defined as a similarity measure based on the longest common subsequence (LCS). In this way, the quality and accuracy of the fine-tuning data set are guaranteed. The obtained fine-tuning data set can be used to train the language model in the future, which effectively improves the task processing capabilities of the large language model in the field of network security and provides high-quality data support for practical applications.
[0073] In the embodiments of the present application, Figure 3 As shown, step S104: using the Rouge_L score to evaluate the quality of the distributed instruction set, filtering out instructions that do not meet the quality standards, and obtaining a fine-tuning data set, including:
[0074] S201: Obtaining the longest common subsequence length of the scattered instruction set and the reference instruction set;
[0075] S202: Obtaining the Rouge_L score of the scattered instruction set according to the longest common subsequence length and the reference instruction set;
[0076] S203: Filter out instructions whose Rouge_L scores are greater than or equal to a preset minimum score threshold from the distributed instruction set to obtain a fine-tuning data set.
[0077] In the specific implementation process, the specific implementation process of step S201 to step S203 is as follows:
[0078] After obtaining the scattered instruction set, first, obtain the length of the longest common subsequence (LCS) between the scattered instruction set and the reference instruction set; illustratively, the longest common subsequence is recorded as LCS (X, Y), where X represents the scattered instruction set and Y represents the reference instruction set. In practical applications, the similarity between the generated text and the reference text can be measured by calculating the length of the longest common subsequence. In an embodiment of the present application, the similarity between an instruction in the scattered instruction set and an instruction in the reference instruction set can be determined by obtaining the length of the longest common subsequence between the scattered instruction set and the reference instruction set. Exemplarily, an instruction y in the reference instruction set is "How to write a crawler in Python?", and an instruction x in the distributed instruction set is "How to write a Python crawler?". The similarity between the two instructions can be determined by calculating LCS(x, y); the LCS algorithm will first convert x and y into sequences, where y: ['how', 'use', 'Python', 'write', 'one', 'crawler'], x: ['Python', 'crawler', 'how', 'write']; then, the corresponding sequence is converted into a two-dimensional array, and the length of the longest common subsequence of the two-dimensional array is calculated, and the value of LCS(x, y) can be obtained as 0.8. Then, based on the length of the longest common subsequence and the reference instruction set, the Rouge_L score of the distributed instruction set is obtained. Among them, the Rouge_L score is used to characterize the similarity measure based on the longest common subsequence, and the Rouge_L score is proportional to the similarity; the larger the Rouge_L score, the greater the similarity; conversely, the smaller the Rouge_L score, the smaller the similarity. The calculation formula of the Rouge_L score is:
[0079]
[0080] By comparing the scattered instruction set with the reference instruction set, the Rouge_L score of the scattered instruction set is calculated to evaluate the quality of the scattered instruction set.
[0081] Then, from the distributed instruction set, the instructions whose Rouge_L scores are greater than or equal to the preset minimum score threshold are filtered out to obtain the fine-tuning data set. For example, the minimum score threshold is denoted as T Rouge_L(X,Y) , the instructions that are filtered out and do not meet the quality standards are:
[0082] S filtered ={s∈S generated |Rouge_L(X,Y)≥T Rouge_L}
[0083] For example, the minimum score threshold value may be 0.7. Of course, the minimum score threshold value may also be set according to actual application needs, which is not limited here. That is, the minimum score threshold value T may be used. Rouge_L(X,Y) The instructions that meet the quality standards are selected from the scattered instruction sets. In this way, it is ensured that the selected instruction sets meet the requirements in terms of accuracy and coverage.
[0084] In the embodiments of the present application, Figure 4 As shown, step S103: measuring the diversity of the extended instruction set includes:
[0085] S301: Obtaining the probability distribution of the extended instruction set;
[0086] S302: Obtaining an entropy value of the extended instruction set according to the probability distribution;
[0087] S303: Measure the diversity of the extended instruction set according to the entropy value.
[0088] In the specific implementation process, the specific implementation process of step S301 to step S303 is as follows:
[0089] After obtaining the extended instruction set, first, obtain the probability distribution of the extended instruction set. Then, obtain the entropy value of the extended instruction set according to the probability distribution; wherein the entropy value is inversely proportional to the diversity; that is, the larger the entropy value of the extended instruction set, the smaller the diversity of the extended instruction set; conversely, the smaller the entropy value of the extended instruction set, the greater the diversity of the extended instruction set. The relevant calculation formula is as follows:
[0090]
[0091] Among them, S i Indicates extended instruction set, H(S i ) represents the entropy value of the extended instruction set, p(S j ) represents the probability distribution of the extended instruction set.
[0092] In this way, the diversity of the extended instruction set can be measured, thereby providing a guarantee for screening out extended instruction sets with better diversity.
[0093] In the embodiments of the present application, Figure 5 As shown, before step S101: calling the security knowledge base to retrieve the query vector corresponding to the initial seed instruction set, the method further includes:
[0094] S401: Acquire local network security text data, and perform vectorization processing on the network security text data to obtain a corresponding vector set;
[0095] S402: Construct a security knowledge base according to the vector set and the original data corresponding to the vector set.
[0096] In the specific implementation process, the specific implementation process of step S401 to step S402 is as follows:
[0097] First, obtain network security text data from the local. Exemplarily, the network security text data can be a local expert knowledge base, threat intelligence, network traffic analysis data, SoC alarm information, etc. Then, the network security text data is vectorized. Exemplarily, the obtained network security text data is standardized, such as word segmentation, stop word removal, stem extraction, etc.; then, the standardized text data is mapped to a high-dimensional vector space using a pre-trained model to obtain the corresponding vector set. Taking the BERT (Bidirectional Encoder Representations from Transformers) model (an open source natural language processing model released by Google) as an example, for a given text x i , generate the corresponding vector representation v according to the following formula i :v i =x i =BERT(x i ). BERT(x) represents the embedding function of the model. Then, a security knowledge base is constructed based on the obtained vector set and the original data corresponding to the vector set. For example, a security knowledge base (denoted as DB) can be created using tools such as FAISS, Pinecone, and Milvus. V ), the vector set and its corresponding original data can be stored in the security knowledge base. The relevant formula is as follows:
[0098] DB V ={(v 1 ,d 1 ′),(v 2 ,d 2 ′),…,(v n ,d n ′)}
[0099] Where V = {v 1 ,v 2 ,…,v n}, D = {d′ 1 ,d′ 2 ,…,d′ n}, n is an integer greater than 2, V represents a vector set, and D' represents the corresponding original data.
[0100] In the specific implementation process, an ANN index structure can be constructed, such as IVF-Flat, HNSW, LSH, etc., to accelerate vector retrieval.
[0101] In the embodiments of the present application, Figure 6 As shown, step S102: inputting the candidate instruction set corresponding to the candidate vector set into the large language model, expanding the candidate instruction set by the self-guided generation method, and obtaining the extended instruction set, including:
[0102] S501: Determine the instruction set to be selected corresponding to the candidate vector set;
[0103] S502: Input the instruction set to be selected into the large language model, and expand the instruction set to be selected through multiple rounds of iterations by calling the generation function of the prompt word project to generate the extended instruction set.
[0104] In the specific implementation process, the specific implementation process of step S501 to step S502 is as follows:
[0105] After obtaining the candidate vector set, first, determine the candidate instruction set corresponding to the candidate vector set. Exemplarily, the candidate vector set can be inversely quantized, and the vector elements therein can be converted into natural language to obtain the candidate instruction set corresponding to the candidate vector set. Then, the candidate instruction set is input into the large language model, and the candidate instruction set is expanded through multiple rounds of iterations by calling the generation function of the prompt word project to generate an extended instruction set. In other words, the data set is expanded by self-generation of the model. Exemplarily, the API of the large language model related to the prompt word project can be used to expand the candidate instruction set through multiple rounds of iterations through the corresponding generation function, thereby generating a new instruction set, and the generated new instruction set is used as the extended instruction set. Specifically, the extended instruction set can be generated using the following formula:
[0106] S i+1 =S i ∪Generate(S i )
[0107] Among them, the generation function Generate(S i ) can be generated by multiple rounds of iterations, Generate(S i )=(LLM(s j )|s j ∈S i ,j∈[1,n]), S i+1 Indicates extended instruction set, S i Indicates the instruction set to be selected.
[0108] In this way, by performing multiple rounds of generation of the selected instruction sets, the size of the instruction set is gradually expanded, ensuring the diversity and coverage of the instruction set.
[0109] The following will combine with Figure 7 the flowchart shown below to provide a detailed explanation of the specific implementation process of a method for constructing a fine-tuning dataset for a large language model according to an embodiment of the present application.
[0110] S1: Text preprocessing;
[0111] First, standardize the collected network security text data. The relevant steps are as follows:
[0112] Word segmentation: Split the text data into independent words; for example, split the text data "network security protection" into "network", "security", and "protection".
[0113] Stop word removal: Remove stop words; for example, remove "of", "in", etc.
[0114] Stemming: Restore the words to their basic forms; for example, classify both "detection" and "detected" as "detection".
[0115] S2: Vector representation generation;
[0116] Specifically, map the preprocessed text data to a high-dimensional vector space. Exemplarily, use the BERT model to convert the text x i into a vector representation v i , and the formula is as follows: v i = x i = BERT(x i ).
[0117] For example, if x i is "network security", the v i generated by the BERT model is a 768-dimensional vector.
[0118] S3: Security knowledge base construction;
[0119] Exemplarily, use the FAISS tool to create a security knowledge base DB V ), and store the generated vector set V and its corresponding original data D' together. The HNSW index structure can be adopted to speed up retrieval. In practical applications, the size of the security knowledge base and the vector data depend on the scale of the dataset. For example, the security knowledge base includes 1 million vector data.
[0120] S4: Similarity measurement;
[0121] When retrieving data, the cosine similarity sim(v i , v j) is used as a metric for vector similarity. Through this metric, several records that are most similar to the query vector can be quickly retrieved from the security knowledge base. i and v j Taking these two vectors as an example, a cosine similarity value close to 1 indicates that they are very similar in vector space. The relevant calculation formula is as follows:
[0122]
[0123] S5: Generate an extended instruction set; specifically, including:
[0124] S51: Initial seed instruction set generation: Define key network security categories, such as "intrusion detection" and "vulnerability scanning", and generate initial seed instruction sets S for each category. 0 , for example, "Check firewall settings", "Scan open ports", etc.
[0125] S52: Self-guided generation of extended instruction set: based on the initial seed instruction set S 0 , generate a query vector vq, and call the security knowledge base for retrieval to obtain the corresponding retrieval results (i.e., candidate vector set), and generate a candidate instruction set based on the retrieval results. Exemplarily, through the large language model API, generate a new instruction set (i.e., extended instruction set) S corresponding to the candidate instructions of the retrieval results i+1 :S i+1 =S i ∪Generate(S i ). Then, through multiple rounds of iterative generation, the instruction set is continuously expanded, thus ensuring the diversity and coverage of the instruction set.
[0126] S6: Generate instruction set diversity control;
[0127] After each round of generation, entropy H(S) can be used to evaluate diversity:
[0128] In the specific implementation process, the entropy value can be controlled by adjusting the generation strategy to ensure the diversity of the generated instruction set. For example, the extended instruction set whose entropy value meets the requirements can be used as a distributed instruction set to ensure the diversity of the generated instruction set.
[0129] S7: instruction set screening and evaluation;
[0130] Specifically, it includes:
[0131] S71: Instruction screening algorithm; illustratively, determining the instruction screening algorithm to be Rouge_L score.
[0132] S72: Rouge_L score evaluation: Specifically, the Rouge_L score is used to evaluate the quality of the generated scattered instruction set. The screening conditions are as follows:
[0133] Rouge _ L(X,Y)≥T Rouge_L .
[0134] Among them, T Rouge_L(X,Y) Indicates the preset threshold value. For example, T Rouge_L(X,Y) Set to 0.7. In this way, the high standards of accuracy and coverage of the instruction set are guaranteed through this screening condition.
[0135] Based on the same inventive concept, Figure 8 As shown, the embodiment of the present application provides a device for constructing a large language model fine-tuning dataset, including:
[0136] A retrieval unit 10 is used to call a security knowledge base to retrieve a query vector corresponding to an initial seed instruction set, and select a candidate vector set whose similarity with the query vector is greater than a first preset threshold from a plurality of vector sets stored in the security knowledge base, wherein the security knowledge base is a knowledge base pre-constructed based on local network security text data;
[0137] An expansion unit 20, configured to input a candidate instruction set corresponding to the candidate vector set into a large language model, and expand the candidate instruction set by a self-guided generation method to obtain an expanded instruction set;
[0138] A measuring unit 30, configured to measure the diversity of the extended instruction set, and to select a distributed instruction set having an entropy value less than a second preset threshold from the extended instruction set, wherein the entropy value is inversely proportional to the diversity;
[0139] The quality evaluation unit 40 is used to use the Rouge_L score to perform quality evaluation on the scattered instruction set, filter out instructions that do not meet the quality standards, and obtain a fine-tuning data set, where the fine-tuning data set is used to train the large language model.
[0140] In the embodiment of the present application, the quality assessment unit 40 is specifically used for:
[0141] Obtaining the longest common sequence length between the distributed instruction set and the reference instruction set;
[0142] Obtaining the Rouge_L score of the scattered instruction set according to the longest common sequence length and the reference instruction set;
[0143] From the scattered instruction set, instructions whose Rouge_L scores are greater than or equal to a preset minimum score threshold are filtered out to obtain a fine-tuning data set.
[0144] In the embodiment of the present application, the measurement unit 30 is specifically used for:
[0145] Obtaining a probability distribution of the extended instruction set;
[0146] Obtaining an entropy value of the extended instruction set according to the probability distribution;
[0147] The diversity of the extended instruction set is measured according to the entropy value.
[0148] In the embodiment of the present application, before calling the security knowledge base to retrieve the query vector corresponding to the initial seed instruction set, the device further includes a construction unit 50, and the construction unit 50 is specifically used to:
[0149] Acquire local network security text data, and perform vectorization processing on the network security text data to obtain a corresponding vector set;
[0150] A security knowledge base is constructed based on the vector set and the original data corresponding to the vector set.
[0151] In the embodiment of the present application, the expansion unit 20 is specifically used for:
[0152] Determining the instruction set to be selected corresponding to the candidate vector set;
[0153] The candidate instruction set is input into the large language model, and the candidate instruction set is expanded through multiple rounds of iterations by calling the generation function of the prompt word project to generate the expanded instruction set.
[0154] Based on the same inventive concept, the embodiment of the present application further provides an electronic device, which can implement the functions of the aforementioned device for constructing a large language model fine-tuning dataset, referring to Fig. 9 As shown, the electronic device includes:
[0155] Memory 100, used for storing computer programs;
[0156] The processor 200 is used to implement the steps of the method for constructing a large language model fine-tuning dataset as described in any of the above items when executing the computer program stored in the memory 100.
[0157] The specific connection medium between the processor 200 and the memory 100 is not limited in the embodiment of the present application. Fig. 9 In the example, the processor 200 and the memory 100 are connected via the bus 300. The bus 300 is connected to the memory 100 via the bus 300. Fig. 9 The bus 300 is represented by a bold line, and the connection between other components is only for schematic illustration and is not intended to be limiting. The bus 300 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 200 can also be called a controller, and there is no limitation on the name.
[0158] In the embodiment of the present application, the memory 100 stores instructions that can be executed by the processor 200. When the processor 200 executes the computer program stored in the memory 100, it can execute the method for constructing a large language model fine-tuning dataset discussed above. The processor 200 can implement Figure 8 The functions of each module are shown in the figure.
[0159] Among them, the processor 200 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing the computer program stored in the memory 100 and calling the data stored in the memory 100, the various functions of the device and processing data, the device can be monitored as a whole.
[0160] In one possible design, the processor 200 may include one or more processing units, and the processor 200 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 200. In some embodiments, the processor 200 and the memory 100 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.
[0161] The processor 200 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor 200, etc. The steps of the method for constructing a large language model fine-tuning data set disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor 200.
[0162] The memory 100 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 100 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 100 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 100 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0163] By programming the processor 200, the code corresponding to the method for constructing the large language model fine-tuning dataset described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 2 The steps of a method for constructing a large language model fine-tuning data set in the embodiment shown are as follows: How to design and program the processor 200 is a technique known to those skilled in the art and will not be described in detail here.
[0164] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by the processor 200, it implements the method for constructing a large language model fine-tuning dataset discussed above.
[0165] In some possible implementations, various aspects of the method for constructing a large language model fine-tuning dataset provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of the method for constructing a large language model fine-tuning dataset according to various exemplary embodiments of the present application described above in this specification.
[0166] The embodiment of the present application provides a method, device and electronic device for constructing a large language model fine-tuning data set, wherein, through the construction method, first, the security knowledge base is called to retrieve the query vector corresponding to the initial seed instruction set, and the candidate vector set whose similarity with the query vector is greater than the first preset threshold is screened out from the multiple vector sets stored in the security knowledge base; wherein the security knowledge base is a knowledge base pre-constructed based on local network security text data. In this way, through the localized security knowledge base, the dependence on external data sources is reduced, and the data leakage and privacy risks are reduced. Then, the candidate instruction set corresponding to the candidate vector set is input into the large language model, and the candidate instruction set is expanded by the self-guided generation method to obtain the extended instruction set. In this way, the problem of data scarcity is solved.
[0167] Then, the diversity of the extended instruction set is measured, and the scattered instruction set whose entropy value is less than the second preset threshold is screened out from the extended instruction set, wherein the smaller the entropy value, the better the diversity of the extended instruction set. In this way, the instruction set that meets the diversity requirements can be screened out. Then, the Rouge_L score is used to evaluate the quality of the scattered instruction set, and the instructions that do not meet the quality standards are filtered out to obtain a data set for fine-tuning the large language model. In this way, the high quality and accuracy of the data set are ensured.
[0168] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0169] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for constructing a large language model fine-tuning dataset, characterized in that: include: Calling a security knowledge base to retrieve a query vector corresponding to the initial seed instruction set, and screening out a candidate vector set having a similarity with the query vector greater than a first preset threshold from a plurality of vector sets stored in the security knowledge base, wherein the security knowledge base is a knowledge base pre-built based on local network security text data; Inputting a candidate instruction set corresponding to the candidate vector set into a large language model, and expanding the candidate instruction set by a self-guided generation method to obtain an expanded instruction set; Measuring the diversity of the extended instruction set, and screening out a distributed instruction set having an entropy value less than a second preset threshold from the extended instruction set, wherein the entropy value is inversely proportional to the diversity; The Rouge_L score is used to evaluate the quality of the scattered instruction set, and instructions that do not meet the quality standards are filtered out to obtain a fine-tuning data set, where the fine-tuning data set is used to train the large language model.
2. The method according to claim 1, characterized in that The Rouge_L score is used to evaluate the quality of the scattered instruction set, filter out instructions that do not meet the quality standards, and obtain a fine-tuning data set, including: Obtaining the longest common subsequence length of the scattered instruction set and the reference instruction set; Obtaining the Rouge_L score of the scattered instruction set according to the longest common subsequence length and the reference instruction set; From the scattered instruction set, instructions whose Rouge_L scores are greater than or equal to a preset minimum score threshold are filtered out to obtain a fine-tuning data set.
3. The method according to claim 1, characterized in that The measuring the diversity of the extended instruction set includes: Obtaining a probability distribution of the extended instruction set; Obtaining an entropy value of the extended instruction set according to the probability distribution; The diversity of the extended instruction set is measured according to the entropy value.
4. The method according to any one of claims 1 to 3, characterized in that: Before calling the security knowledge base to retrieve the query vector corresponding to the initial seed instruction set, the method further includes: Acquire local network security text data, and perform vectorization processing on the network security text data to obtain a corresponding vector set; A security knowledge base is constructed based on the vector set and the original data corresponding to the vector set.
5. The method according to any one of claims 1 to 3, characterized in that: The step of inputting the candidate instruction set corresponding to the candidate vector set into the large language model, and expanding the candidate instruction set by a self-guided generation method to obtain an expanded instruction set includes: Determining the instruction set to be selected corresponding to the candidate vector set; The candidate instruction set is input into the large language model, and the candidate instruction set is expanded through multiple rounds of iterations by calling the generation function of the prompt word project to generate the expanded instruction set.
6. A device for constructing a large language model fine-tuning dataset, characterized in that: include: A retrieval unit, configured to call a security knowledge base to retrieve a query vector corresponding to an initial seed instruction set, and select a candidate vector set having a similarity with the query vector greater than a first preset threshold from a plurality of vector sets stored in the security knowledge base, wherein the security knowledge base is a knowledge base pre-constructed based on local network security text data; An expansion unit, configured to input a candidate instruction set corresponding to the candidate vector set into a large language model, and expand the candidate instruction set by a self-guided generation method to obtain an expanded instruction set; A measuring unit, used to measure the diversity of the extended instruction set, and filter out a distributed instruction set having an entropy value less than a second preset threshold from the extended instruction set, wherein the entropy value is inversely proportional to the diversity; The quality assessment unit is used to use the Rouge_L score to perform quality assessment on the scattered instruction set, filter out instructions that do not meet the quality standards, and obtain a fine-tuning data set, where the fine-tuning data set is used to train the large language model.
7. The device according to claim 6, characterized in that The quality assessment unit is specifically used for: Obtaining the longest common sequence length of the distributed instruction set and the reference instruction set; Obtaining the Rouge_L score of the scattered instruction set according to the longest common sequence length and the reference instruction set; From the scattered instruction set, instructions whose Rouge_L scores are greater than or equal to a preset minimum score threshold are filtered out to obtain a fine-tuning data set.
8. The device according to claim 6, characterized in that The measurement unit is specifically used for: Obtaining a probability distribution of the extended instruction set; Obtaining an entropy value of the extended instruction set according to the probability distribution; The diversity of the extended instruction set is measured according to the entropy value.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to implement the method steps of any one of claims 1 to 5 when executing the computer program stored in the memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.