Vulnerability information identification method and device based on large language model

Abstract

The present disclosure belongs to the technical field of nuclear power and specifically relates to a vulnerability information identification method and device based on a large language model. The present disclosure collects PoC information of certain CVEs and manually labels part of the data, thereby enhancing the accuracy and reliability of the training data in a manual labeling manner, so as to improve the learning ability of the subsequent model. The GPT-3.5-turbo model is used to identify and summarize the CVE and PoC data, each piece of information is described based on the TTP content form, the CVE and PoC information are unified into the TTP standard format, the noise interference in the original information is eliminated, the identification and learning ability of the large language model for the key information features can be enhanced, and the instruction fine-tuning technology is used to improve the task completion effect of the GPT model. Based on the Llama-2 model after fine-tuning, the input target CVE and PoC information are identified and processed, and the correlation between the CVE and the PoC is determined based on the output. Thus, the problem of relying on manual analysis in the CVE and PoC correlation identification is solved.

Description

Technical Field

[0001] This invention belongs to the field of nuclear power technology, specifically relating to a method and apparatus for identifying vulnerability information based on a large language model. Background Technology

[0002] With the rapid development of computer and network technologies, network applications have deeply penetrated all walks of life. While internet technology assists industrial production, it has also had a profound impact on network security vulnerabilities. Network security issues are complex and involve diverse scenarios, posing a challenge to vulnerability identification. CVEs (Common Vulnerabilities and Exposures) provide a unique identifier for each vulnerability and offer network security personnel a unified method for identifying and describing publicly disclosed network security vulnerabilities. Meanwhile, to gain a more comprehensive understanding of the causes and harmful processes of security vulnerabilities, security personnel often use PoCs (Proof of Concept) to verify them. CVEs and PoCs describe different stages of the vulnerability lifecycle, enhancing the efficiency of vulnerability management and classification both theoretically and practically. Therefore, identifying and associating CVEs and PoCs originating from the same vulnerability is of great significance. Based on CVEs and their corresponding PoCs, enterprise security personnel can more efficiently identify and handle security vulnerabilities, thereby improving the enterprise's security protection capabilities. However, it's important to note that while CVEs are a standardized method for describing vulnerabilities, in actual security testing and vulnerability identification, different security personnel may produce CVEs with varying content or format due to differences in their professional perspectives or work habits. Similarly, different security personnel may obtain different Proof-of-Concept (PoC) reports for the same vulnerability. Currently, the identification and correlation of PoCs and CVEs still relies on manual identification by security personnel, and the efficiency and accuracy of this identification heavily depend on human experience.

[0003] The emergence of large language models in recent years has shown the possibility of using programs to understand complex text. Large language models possess powerful language understanding capabilities; trained on large-scale corpus datasets, they can automatically learn the grammatical, semantic, and contextual information between words in text. Using large language models, key information related to vulnerability features can be learned from CVE and PoC documents, and based on this information, CVEs and PoCs can be matched to identify and associate vulnerabilities with their corresponding PoCs. However, unlike common textual information, PoC and CVE documents contain numerous specialized representations or code snippets, which increases the difficulty for models to recognize semantic information. Therefore, efficiently identifying whether a pair of PoC and CVE information corresponds remains a significant challenge. Summary of the Invention

[0004] To overcome the problems existing in related technologies, a vulnerability information identification method and device based on a large language model is provided.

[0005] According to one aspect of the embodiments of this disclosure, a vulnerability information identification method based on a large language model is provided, the method comprising:

[0006] Step 1, Training Data Collection: Collect CVE information published on various CVE publishing platforms and existing PoC information for CVEs on the Internet. Manually label the collected information. The labels can be used to describe whether a pair of CVE and PoC data has a correlation.

[0007] Step 2, GPT data processing: The GPT-3.5-turbo model is used to identify and summarize CVE and PoC data. Each CVE and PoC information pair is described based on the TTP content format. The description items of each information include three categories: tactics, technology and procedures, forming a fine-tuning dataset.

[0008] Step 3, GPT data augmentation; use the GPT-3.5-turbo model to augment the fine-tuned dataset;

[0009] Step 4, Llama-2 model tuning; fine-tuning the Llama-2 model using sample pairs of labeled PoC and CVE data;

[0010] Step 5: Based on the fine-tuned Llama-2 model, identify and process the input target CVE and PoC information, and output the correlation index. Based on the correlation index, determine the correlation between CVE and PoC.

[0011] In one possible implementation, step 2 includes:

[0012] Step 21: Based on a question-and-answer interactive format, a template combining instructions, examples, and inputs is used to guide the GPT model to understand the input CVE or PoC information and output content. Here, instructions are used to describe the generated task; examples are used to describe the input and output of the task in the instructions, as well as the expected output structure; and inputs are used to describe the CVE or PoC content to be processed.

[0013] Step 22: Using labeled CVE and PoC datasets as examples, guide the GPT model to identify the datasets and convert CVE and PoC into corresponding TTP data, which will then be used as the fine-tuning dataset for the Llama-2 model.

[0014] In one possible implementation, step 3 includes:

[0015] Step 31: Perform data augmentation in the form of translated languages; for the original CVE or PoC data, translate the original language into another language and then back into the original language, converting the CVE and PoC into the corresponding TTP format data as a supplement to the fine-tuning dataset;

[0016] Step 32: Rewrite the natural language information part of the original CVE or PoC using the GPT model, and require that the GPT data does not change the original semantics, and the rewritten information replaces the original information; then convert the CVE and PoC into the corresponding TTP form data, and use them as a supplementary part of the fine-tuning dataset.

[0017] In one possible implementation, in step 4, the TTP (Tactical, Technical, and Procedural) information of each CVE and PoC is input into the Llama-2 model. The TTP information of each CVE and PoC includes three aspects: tactics, technology, and process. CVE and PoC pairs with corresponding relationships are used as positive sample pairs, and CVE and PoC pairs without corresponding relationships are used as negative sample pairs. The Llama-2 model is trained and fine-tuned based on the positive and negative sample pairs, and finally outputs the discrimination result of whether CVE and PoC have a corresponding relationship.

[0018] In one possible implementation, samples generated through data augmentation form negative sample pairs.

[0019] In one possible implementation, step 4 includes:

[0020] Step 41, input positive sample pairs into the model and negative sample pairs The model learns feature information from the input, transforms the input into a vector form containing feature information, and calculates the similarity between vectors using cosine similarity, including the similarity between positive sample pairs. ;

[0021] Step 42: Calculate the loss function based on the similarity between positive sample pairs and the similarity between negative sample pairs;

[0022] Using the InfoNCE loss as the loss function in the fine-tuning process, as shown in the following equation:

[0023] ;

[0024] Where i is the sample number, Positive sample pairs Positive samples in It is different from negative sample pairs negative samples Other negative samples, where N is the number of other negative samples selected; express and The similarity between them; It is a temperature parameter used to adjust the scale of similarity during the learning process;

[0025] Step 43: Train and fine-tune the model on the fine-tuning dataset until the model loss stabilizes; then fix the model parameters and save the model.

[0026] In one possible implementation, step 5 includes:

[0027] Step 51: The CVE and PoC information to be tested will be input into the GPT model and processed in the manner of step 2, outputting data in TTP format.

[0028] Step 52: Input the CVE and POC data in TTP form into the Llama-2 model and calculate their similarity based on the cosine similarity method. If the similarity exceeds a preset threshold, it is determined that the CVE and POC have a corresponding association relationship. If the similarity does not exceed the preset threshold, it is determined that the CVE and POC do not have a corresponding association relationship.

[0029] According to another aspect of the embodiments of this disclosure, a vulnerability information identification device based on a large language model is provided, the device comprising:

[0030] The training data collection module is used to collect CVE information published on various CVE publishing platforms and existing PoC information for CVE on the Internet. The collected information is manually labeled, and the labels can be used to describe whether a pair of CVE and PoC data has a correlation.

[0031] The GPT data processing module is used to identify and summarize CVE and PoC data using the GPT-3.5-turbo model. It describes each CVE and PoC information pair based on the TTP content format. The description items of each information include three categories: tactics, technology and procedures, forming a fine-tuning dataset.

[0032] The GPT data augmentation module is used to augment fine-tuned datasets using the GPT-3.5-turbo model.

[0033] The Llama-2 model tuning module is used to fine-tune the Llama-2 model using sample pairs of labeled PoC and CVE data.

[0034] The judgment module is used to identify and process the input target CVE and PoC information based on the fine-tuned Llama-2 model, and output the correlation index. The correlation index is used to determine the correlation between CVE and PoC.

[0035] According to another aspect of the embodiments of this disclosure, a vulnerability information identification device based on a large language model is provided, the device comprising:

[0036] processor;

[0037] Memory used to store processor-executable instructions;

[0038] The processor is configured to execute the above-described method.

[0039] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a processor, implement the above-described method.

[0040] The beneficial effects of this disclosure are as follows: The vulnerability information identification method based on a large language model provided in this disclosure collects CVE information published on various CVE release platforms and existing PoC information for certain CVEs on the Internet. Some data is manually labeled, which enhances the accuracy and reliability of the training data, thereby improving the learning ability of the subsequent model. By using the GPT-3.5-turbo model to identify and summarize CVE and PoC data, each piece of information is described based on the TTP (Tactics, Techniques, and Procedures) content format (each piece of information includes three categories: tactics, techniques, and procedures). This unifies CVE and PoC information into the TTP standard format, eliminating noise interference in the original information, enhancing the large language model's ability to identify and learn key information features, and using instruction fine-tuning technology to improve the GPT model's task completion performance. Then, based on the existing descriptive information, the GPT-3.5-turbo model is used to augment the manually labeled data, thereby increasing the quantity and diversity of labeled samples and eliminating the need for complex manual operations. Next, the pre-trained Llama-2 model is fine-tuned using samples composed of labeled PoC and CVE data to enhance its ability to recognize and understand CVE and PoC data. Then, based on the fine-tuned Llama-2 model, the input target CVE and PoC information is identified and processed, and a correlation index is output. The correlation index is then used to determine the association between CVEs and PoCs. Using the Llama-2 model with a small number of parameters in a local environment reduces the risk of data leakage while achieving strong language processing capabilities, thus enabling the model to better adapt to specific tasks. Finally, for target CVEs, after processing them together with data from the existing PoC database, the existence of a corresponding correlation can be determined, thereby solving the problem of relying on manual analysis for CVE and PoC association identification. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating a vulnerability information identification method based on a large language model, as shown in an embodiment of this disclosure.

[0042] Figure 2 This is a schematic diagram illustrating GPT data enhancement according to an embodiment of this disclosure.

[0043] Figure 3 This is a schematic diagram illustrating the Llama-2 model for identifying vulnerability information, as shown in an embodiment of this disclosure.

[0044] Figure 4This is a block diagram of a vulnerability information identification device based on a large language model, as shown in an embodiment of this disclosure. Detailed Implementation

[0045] The present disclosure will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0046] Unless otherwise defined, the technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains; the terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure; the term "comprising" and any variations thereof in this disclosure are intended to cover non-exclusive inclusion. Clearly, the embodiments described in this disclosure are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0047] In this disclosure, the reference to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this disclosure. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0048] Figure 1 This is a flowchart illustrating a vulnerability information identification method based on a large language model, as shown in an embodiment of this disclosure. The method can be executed by a terminal device, which can be a server, desktop computer, laptop computer, etc. This embodiment of the disclosure does not limit the type of terminal device. Figure 1 As shown, the method includes:

[0049] Step 1, Training Data Collection. Collect CVE information published on various CVE publishing platforms and existing PoC information for CVEs on the Internet. Manually label the collected information, and the labels should indicate whether there is a correlation between a pair of CVE and PoC data.

[0050] As an example of this embodiment, in step 1, the terminal device can collect CVE information published on various CVE publishing platforms. It also collects existing PoC information for certain CVEs on the internet, removes noise information unrelated to the vulnerability from the CVE and PoC information, and manually labels some data based on the relationship between CVEs and PoCs.

[0051] Step 2, GPT data processing: The GPT-3.5-turbo model is used to identify and summarize CVE and PoC data. Each CVE and PoC information pair is described based on the TTP (Tactics, Techniques, and Procedures) content format. Each information pair's description includes three categories: tactics, techniques, and procedures, forming a fine-tuned dataset. This adjusts PoC and CVE information of different styles into triplet data in the form of tactics, techniques, and procedures.

[0052] As an example of this embodiment, step 2 may include:

[0053] Step 21: Based on a question-and-answer interactive format, a template combining instructions, examples, and inputs is used to guide the GPT model to understand the input CVE or PoC information and output content. Instructions are used to describe the generated task; examples are used to describe the input and output of the task in the instructions, as well as to give the expected output structure; and inputs are used to describe the CVE or PoC content to be processed.

[0054] Step 22: Using labeled CVE and PoC datasets as examples, guide the GPT model to identify the datasets and convert CVE and PoC into corresponding TTP data, which will then be used as the fine-tuning dataset for the Llama-2 model.

[0055] Step 3, GPT Data Augmentation. Based on the existing descriptive information, the GPT-3.5-turbo model is used to augment the fine-tuned dataset, thereby increasing the quantity and diversity of labeled samples.

[0056] Figure 2 This is a schematic diagram illustrating GPT data enhancement according to an embodiment of this disclosure, as shown below. Figure 2 As shown, step 3 includes:

[0057] Step 31: Perform data augmentation in translated language. For the original CVE or PoC data, translate the original language into another language and then back into the original language, converting the CVE and PoC into the corresponding TTP format data as a supplement to the fine-tuning dataset.

[0058] Step 32: Rewrite the natural language information portion of the original CVE or PoC using the GPT model, ensuring that the GPT data does not alter the original semantics, replacing the original information with the rewritten information. Then, convert the CVE and PoC into their corresponding TTP format data, using them as a supplementary part of the fine-tuning dataset.

[0059] Step 4: Llama-2 model tuning. The Llama-2 model is fine-tuned using sample pairs of labeled PoC and CVE data.

[0060] As an example of this embodiment, when fine-tuning the model, the TTP (Tactical, Technical, and Procedural) information of each CVE and PoC is input into the Llama-2 model. The TTP information for each CVE and PoC includes three aspects: tactics, techniques, and procedures. CVE and PoC pairs with a corresponding relationship are treated as positive sample pairs, while those without a corresponding relationship are treated as negative sample pairs. The Llama-2 model is trained and fine-tuned based on these positive and negative sample pairs, and finally outputs a judgment result indicating whether CVEs and PoCs have a corresponding relationship.

[0061] In one possible implementation, samples generated through data augmentation form negative sample pairs.

[0062] Step 41, input positive sample pairs into the model and negative sample pairs ,like Figure 2 As shown, the model learns feature information from the input, transforms the input into a vector form containing feature information, and calculates the similarity between vectors using the cosine similarity method, including the similarity between positive sample pairs. .

[0063] Step 42: Calculate the loss function based on the similarity between positive sample pairs and the similarity between negative sample pairs.

[0064] Using the InfoNCE loss as the loss function in the fine-tuning process, as shown in the following equation:

[0065] ;

[0066] Where i is the sample number, Positive sample pairs Positive samples in It is different from negative sample pairs negative samples Other negative samples, where N is the number of other negative samples selected; express and The similarity between them; It is a temperature parameter used to adjust the scale of similarity during the learning process.

[0067] Step 43: Train and fine-tune the model on the fine-tuning dataset until the model loss stabilizes. Then fix the model parameters and save the model.

[0068] Step 5: Based on the fine-tuned Llama-2 model, identify and process the input target CVE and PoC information, and output the correlation index. Based on the correlation index, determine the correlation between CVE and PoC.

[0069] Figure 3 This is a schematic diagram illustrating the Llama-2 model for identifying vulnerability information, as shown in the embodiments of this disclosure. Figure 3 As shown, step 5 includes...

[0070] Step 51: The CVE and PoC information to be tested will be input into the GPT model and processed in the manner of step 2, outputting data in TTP format.

[0071] Step 52: Input the CVE and POC data in TTP form into the Llama-2 model and calculate their similarity based on the cosine similarity method. If the similarity exceeds a preset threshold, it is determined that the CVE and POC have a corresponding association relationship. If the similarity does not exceed the preset threshold, it is determined that the CVE and POC do not have a corresponding association relationship.

[0072] For a target CVE, after processing it together with the data in the existing PoC library through this system, it can be determined whether there is a corresponding correlation, thereby solving the problem of relying on manual analysis when identifying the correlation between CVE and PoC.

[0073] This publicly disclosed vulnerability information identification method based on a large language model collects CVE information published on various CVE release platforms and existing PoC information for certain CVEs on the internet. Some data is manually labeled, enhancing the accuracy and reliability of the training data and thus improving the learning ability of the subsequent model. The GPT-3.5-turbo model is used to identify and summarize CVE and PoC data. Each piece of information is described based on TTP (Tactics, Techniques, and Procedures) content (each description includes three categories: tactics, techniques, and procedures), unifying CVE and PoC information into the TTP standard format. This eliminates noise interference in the original information, enhancing the large language model's ability to identify and learn key information features. Instruction fine-tuning technology is used to improve the GPT model's performance. Then, based on the existing descriptive information, the GPT-3.5-turbo model is used to augment the manually labeled data, increasing the quantity and diversity of labeled samples while eliminating complex manual operations. Next, the pre-trained Llama-2 model is fine-tuned using samples composed of labeled PoC and CVE data to enhance its ability to recognize and understand CVE and PoC data. Then, based on the fine-tuned Llama-2 model, the input target CVE and PoC information is identified and processed, and a correlation index is output. The correlation index is then used to determine the association between CVEs and PoCs. Using the Llama-2 model with a small number of parameters in a local environment reduces the risk of data leakage while achieving strong language processing capabilities, thus enabling the model to better adapt to specific tasks. Finally, for target CVEs, after processing them together with data from the existing PoC database, the existence of a corresponding correlation can be determined, thereby solving the problem of relying on manual analysis for CVE and PoC association identification.

[0074] According to another aspect of the embodiments of this disclosure, a vulnerability information identification device based on a large language model is provided, the device comprising:

[0075] The training data collection module is used to collect CVE information published on various CVE publishing platforms and existing PoC information for CVE on the Internet. The collected information is manually labeled, and the labels can be used to describe whether a pair of CVE and PoC data has a correlation.

[0076] The GPT data processing module is used to identify and summarize CVE and PoC data using the GPT-3.5-turbo model. It describes each CVE and PoC information pair based on the TTP content format. The description items of each information include three categories: tactics, technology and procedures, forming a fine-tuning dataset.

[0077] The GPT data augmentation module is used to augment fine-tuned datasets using the GPT-3.5-turbo model.

[0078] The Llama-2 model tuning module is used to fine-tune the Llama-2 model using sample pairs of labeled PoC and CVE data.

[0079] The judgment module is used to identify and process the input target CVE and PoC information based on the fine-tuned Llama-2 model, and output the correlation index. The correlation index is used to determine the correlation between CVE and PoC.

[0080] The description of the above-mentioned apparatus has been elaborated in detail in the description of the above-mentioned method, and will not be repeated here.

[0081] Figure 4 This is a block diagram illustrating a vulnerability information identification device based on a large language model, as shown in an embodiment of this disclosure. For example, device 1900 can be provided as a server. (Refer to...) Figure 4 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0082] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output (I / O) interface 1958. Device 1900 can operate on an operating system stored in memory 1932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0083] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0084] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0085] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0086] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0087] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0088] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0089] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0090] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0091] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0092] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A vulnerability information identification method based on a large language model, characterized in that, The method includes: Step 1, Training Data Collection: Collect CVE information published on various CVE publishing platforms and existing PoC information for CVEs on the Internet. Manually label the collected information. The labels can be used to describe whether a pair of CVE and PoC data has a correlation. Step 2, GPT data processing: The GPT-3.5-turbo model is used to identify and summarize CVE and PoC data. Each CVE and PoC information pair is described based on the TTP content format. The description items of each information include three categories: tactics, technology and procedures, forming a fine-tuning dataset. Step 3, GPT data augmentation; use the GPT-3.5-turbo model to augment the fine-tuned dataset; Step 4, Llama-2 model tuning; fine-tuning the Llama-2 model using sample pairs of labeled PoC and CVE data; Step 5: Based on the fine-tuned Llama-2 model, identify and process the input target CVE and PoC information, and output the correlation index. Based on the correlation index, determine the correlation between CVE and PoC.

2. The method according to claim 1, characterized in that, Step 2 includes: Step 21: Based on a question-and-answer interactive format, a template combining instructions, examples, and inputs is used to guide the GPT model to understand the input CVE or PoC information and output content. Here, instructions are used to describe the generated task; examples are used to describe the input and output of the task in the instructions, as well as the expected output structure; and inputs are used to describe the CVE or PoC content to be processed. Step 22: Using labeled CVE and PoC datasets as examples, guide the GPT model to identify the datasets and convert CVE and PoC into corresponding TTP data, which will then be used as the fine-tuning dataset for the Llama-2 model.

3. The method according to claim 1, characterized in that, Step 3 includes: Step 31: Perform data augmentation in the form of translated languages; for the original CVE or PoC data, translate the original language into another language and then back into the original language, converting the CVE and PoC into the corresponding TTP format data as a supplement to the fine-tuning dataset; Step 32: Rewrite the natural language information part of the original CVE or PoC using the GPT model, and require that the GPT data does not change the original semantics, and the rewritten information replaces the original information; then convert the CVE and PoC into the corresponding TTP form data, and use them as a supplementary part of the fine-tuning dataset.

4. The method according to claim 1, characterized in that, In step 4, the TTP (Tactical, Technical, and Procedural) information of each CVE and PoC is input into the Llama-2 model. The TTP information of each CVE and PoC includes three aspects: tactics, technology, and process. CVE and PoC pairs with corresponding relationships are used as positive sample pairs, and CVE and PoC pairs without corresponding relationships are used as negative sample pairs. The Llama-2 model is trained and fine-tuned based on the positive and negative sample pairs, and finally outputs the discrimination result of whether CVE and PoC have a correlation relationship.

5. The method according to claim 4, characterized in that, The samples generated through data augmentation form negative sample pairs.

6. The method according to claim 1, characterized in that, Step 4 includes: Step 41, input positive sample pairs into the model and negative sample pairs The model learns feature information from the input, transforms the input into a vector form containing feature information, and calculates the similarity between vectors using cosine similarity, including the similarity between positive sample pairs. ; Step 42: Calculate the loss function based on the similarity between positive sample pairs and the similarity between negative sample pairs; Using the InfoNCE loss as the loss function in the fine-tuning process, as shown in the following equation: ； Where i is the sample number, Positive sample pairs Positive samples in It is different from negative sample pairs negative samples Other negative samples, where N is the number of other negative samples selected; express and The similarity between them; It is a temperature parameter used to adjust the scale of similarity during the learning process; Step 43: Train and fine-tune the model on the fine-tuning dataset until the model loss stabilizes; then fix the model parameters and save the model.

7. The method according to claim 1, characterized in that, Step 5 includes: Step 51: The CVE and PoC information to be tested will be input into the GPT model and processed in the manner of step 2, outputting data in TTP format. Step 52: Input the CVE and POC data in TTP form into the Llama-2 model and calculate their similarity based on the cosine similarity method. If the similarity exceeds a preset threshold, it is determined that the CVE and POC have a corresponding association relationship. If the similarity does not exceed the preset threshold, it is determined that the CVE and POC do not have a corresponding association relationship.

8. A vulnerability information identification device based on a large language model, characterized in that, The device includes: The training data collection module is used to collect CVE information published on various CVE publishing platforms and existing PoC information for CVE on the Internet. The collected information is manually labeled, and the labels can be used to describe whether a pair of CVE and PoC data has a correlation. The GPT data processing module is used to identify and summarize CVE and PoC data using the GPT-3.5-turbo model. It describes each CVE and PoC information pair based on the TTP content format. The description items of each information include three categories: tactics, technology and procedures, forming a fine-tuning dataset. The GPT data augmentation module is used to augment fine-tuned datasets using the GPT-3.5-turbo model. The Llama-2 model tuning module is used to fine-tune the Llama-2 model using sample pairs of labeled PoC and CVE data. The judgment module is used to identify and process the input target CVE and PoC information based on the fine-tuned Llama-2 model, and output the correlation index. The correlation index is used to determine the correlation between CVE and PoC.

9. A vulnerability information identification device based on a large language model, characterized in that, The device includes: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images