Vulnerability information identification method and device based on large language model

Abstract

The invention belongs to the technical field of nuclear power, and particularly relates to a vulnerability information identification method and device based on a large language model. According to the method, PoC information of some CVEs is collected, part of data is manually annotated, and the accuracy and reliability of training data are enhanced in a manual annotation mode, so that the learning ability of a subsequent model is improved. CVE and PoC data are recognized and summarized by using a GPT-3. 5-turbo model, each piece of information is described based on a TTP content form, the CVE and PoC information is unified into a TTP standard format, noise interference in original information is eliminated, the recognition and learning ability of a large language model to key information features can be enhanced, and the recognition and learning efficiency of the large language model is improved. And the task completion effect of the GPT model is improved by using an instruction fine tuning technology. And based on the fine-tuned Llama-2 model, carrying out identification processing on input target CVE and PoC information, and based on output, judging an association relationship between the CVE and the PoC. Therefore, the problem that CVE and PoC association identification depends on manual analysis is solved.

Description

Technical Field

[0001] The present invention belongs to the field of nuclear power technology, and in particular relates to a vulnerability information identification method and device based on a large language model. Background Art

[0002] With the rapid development of computer technology and network technology, network applications have been deeply involved in all walks of life. While Internet technology helps industrial production, it also has a profound impact on network security vulnerabilities. Network security issues are complex and involve a variety of scenarios, which poses a challenge to the identification of vulnerabilities. CVE (Common Vulnerabilities and Exposures) provides a unique identifier for each vulnerability and provides network security personnel with a method to uniformly identify and describe publicly disclosed network security vulnerabilities. At the same time, in order to have a more comprehensive understanding of the causes of security vulnerabilities and their harmful processes, security personnel often use PoC (Proof of Concept) to verify security vulnerabilities. CVE and PoC describe different stages of the vulnerability life cycle, which enhances the management and classification efficiency of vulnerabilities in theory and practice. Therefore, how to identify and associate CVE and PoC from the same vulnerability is of great significance. Based on CVE and its corresponding PoC, enterprise security personnel can more efficiently identify and handle security vulnerabilities, thereby improving the security protection capabilities of enterprises. However, it should be noted that although CVE is a unified way to describe vulnerabilities, in the actual security detection and vulnerability identification process, different security personnel may obtain CVEs with different contents or forms due to their different professional perspectives or working habits. Similarly, different security personnel may obtain PoCs with different contents for the same vulnerability. At present, the work of identifying and associating PoCs and CVEs still relies on manual identification by security personnel, and the identification efficiency and accuracy are heavily dependent on manual experience.

[0003] The emergence of large language models in recent years has made it possible to use programs to understand complex texts. Large language models have powerful language understanding capabilities. By training in large-scale corpus data sets, the model can automatically learn the grammatical, semantic, and contextual information between words in the text. Using large language models, key information related to vulnerability features can be learned from CVE and PoC documents, and based on this information feature, CVE and PoC can be matched, thereby identifying and associating the vulnerability CVE and its corresponding PoC. However, unlike common text information, there are a large number of dedicated representations or code snippets in PoC and CVE documents, which will increase the difficulty of the model to identify text semantic information. Therefore, how to efficiently identify whether a pair of PoC and CVE information has a corresponding relationship still faces huge challenges. Summary of the invention

[0004] In order to overcome the problems existing in the related art, a vulnerability information identification method and device based on a large language model are provided.

[0005] According to one aspect of an embodiment of the present disclosure, a vulnerability information identification method based on a large language model is provided, the method comprising:

[0006] Step 1: Training data collection: collect CVE information published on various CVE publishing platforms, collect existing PoC information for CVE on the Internet, and manually label the collected information. The label can be used to describe whether a pair of CVE and PoC data has a correlation relationship;

[0007] Step 2: GPT data processing. The GPT-3.5-turbo model is used to identify and summarize CVE and PoC data. Each CVE and PoC information pair is described based on the TTP content form. The description items of each information include tactics, techniques, and procedures to form a fine-tuning data set.

[0008] Step 3, GPT data enhancement: Use the GPT-3.5-turbo model to perform data enhancement on the fine-tuning dataset;

[0009] Step 4: Llama model tuning: Use the sample data consisting of labeled PoC and CVE data to fine-tune the Llama-2 model;

[0010] Step 5: Based on the fine-tuned Llama-2 model, the input target CVE and PoC information is identified and processed, and the correlation index is output. The correlation relationship between CVE and PoC is judged based on the correlation index.

[0011] In a possible implementation, step 2 includes:

[0012] Step 21, based on the form of question-answer interaction, use a template that combines instructions, examples, and input to guide the GPT model to understand the input CVE or PoC information and output content; where the instructions are used to describe the generated tasks; the examples are used to describe the input, output, and expected output structure of the tasks in the instructions; the input is used to describe the CVE or PoC content to be processed;

[0013] Step 22, taking the labeled CVE and PoC datasets as examples, guiding the GPT model to identify the datasets, converting CVE and PoC into corresponding TTP form data as the fine-tuning dataset for the subsequent Llama model.

[0014] In a possible implementation, step 3 includes:

[0015] Step 31, perform data enhancement in the form of translated language; for the original CVE or PoC data, translate the original language into another language and then translate it back to the original language, convert CVE and PoC into corresponding TTP form data as a supplementary part of the fine-tuning dataset;

[0016] Step 32: Use the GPT model to rewrite the natural language information part of the original CVE or PoC, and require that the GPT data does not change the original semantics, and the rewritten information replaces the original information; then convert the CVE and PoC into corresponding TTP form data and use it as a supplementary part of the fine-tuning data set.

[0017] In a possible implementation, in step 4, the TTP form information of each CVE and PoC is input into the Llama-2 model, wherein the TTP form information of each CVE and PoC includes three aspects: tactics, technology, and process; the CVE and PoC pairs with corresponding relationships are taken as positive sample pairs, and the CVE and PoC pairs without corresponding relationships are taken as negative sample pairs; the Llama-2 model is trained and fine-tuned based on the positive and negative sample pairs, and finally outputs the judgment result of whether the CVE and PoC have an associated corresponding relationship.

[0018] In one possible implementation, samples generated by data augmentation constitute negative sample pairs.

[0019] In a possible implementation, step 4 includes:

[0020] Step 41: Input positive sample pairs to the model And negative sample pairs like Figure 2 As shown in Figure 1, the model learns feature information from the input, converts the input into a vector containing feature information, and calculates the similarity between vectors using the cosine similarity method, including the similarity between positive sample pairs. The similarity between the negative sample pairs

[0021] Step 42, calculating a loss function based on the similarity between the positive sample pairs and the similarity between the negative sample pairs;

[0022] InfoNCE loss is used as the loss function in the fine-tuning process, as shown in the following formula:

[0023]

[0024] in, is with x i Similar positive samples, is with x i Dissimilar negative samples, sim(x i,x j ) represents x i With x j The similarity between them, here we use cosine similarity; τ is the temperature parameter, which is used to adjust the scale of similarity during the learning process;

[0025] Step 43, perform model training and fine-tuning on the fine-tuning dataset until the model loss tends to be stable; then fix the model parameters and save the model.

[0026] In a possible implementation, step 5 includes:

[0027] Step 51, the CVE and PoC information to be tested will be input into the GPT model, and processed in the form of step 2, and the data in the form of TTP will be output;

[0028] Step 52, input the CVE and POC data in TTP form into the Llama model, and calculate their similarity based on the cosine similarity method. When the similarity exceeds the preset threshold, it is determined that there is a corresponding association relationship between the CVE and the POC. When the similarity does not exceed the preset threshold, it is determined that there is no corresponding association relationship between the CVE and the POC.

[0029] According to another aspect of an embodiment of the present disclosure, a vulnerability information identification device based on a large language model is provided, the device comprising:

[0030] The training data collection module is used to collect CVE information published on various CVE publishing platforms, collect existing PoC information on CVE on the Internet, and manually label the collected information. The label can be used to describe whether a pair of CVE and PoC data has a correlation relationship;

[0031] The GPT data processing module is used to identify and summarize CVE and PoC data using the GPT-3.5-turbo model, and describe each CVE and PoC information pair based on the TTP content form. The description items of each information include tactics, techniques, and procedures to form a fine-tuning data set;

[0032] GPT data enhancement module, used to perform data enhancement on the fine-tuning dataset using the GPT-3.5-turbo model;

[0033] Llama model tuning module, used to fine-tune the Llama-2 model using sample pairs consisting of labeled PoC and CVE data;

[0034] The judgment module is used to identify and process the input target CVE and PoC information based on the fine-tuned Llama-2 model, output the correlation index, and judge the correlation relationship between CVE and PoC based on the correlation index.

[0035] According to another aspect of an embodiment of the present disclosure, a vulnerability information identification device based on a large language model is provided, the device comprising:

[0036] processor;

[0037] a memory for storing processor-executable instructions;

[0038] Wherein, the processor is configured to execute the above method.

[0039] According to another aspect of an embodiment of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0040] The beneficial effect of the present disclosure is that the vulnerability information identification method based on the large language model provided by the present disclosure collects the CVE information published in each CVE publishing platform, collects the existing PoC information for certain CVEs on the Internet, and manually annotates some data, thereby enhancing the accuracy and reliability of the training data in the form of manual annotation, thereby improving the learning ability of the subsequent model. By using the GPT-3.5-turbo model to identify and summarize CVE and PoC data, each piece of information is described based on the TTP (Tactics, Techniques, and Procedures) content form (the description items of each piece of information include tactics, techniques and procedures), the CVE and PoC information are unified into the TTP standard format, the noise interference in the original information is eliminated, the recognition and learning ability of the large language model for key information features can be enhanced, and the instruction fine-tuning technology is used to improve the GPT model to complete the task effect. Then, based on the existing description information, the GPT-3.5-turbo model is used to enhance the data that has been manually labeled, thereby increasing the number, scale and diversity of label samples, and eliminating the complex operations of manual participation. Then, fine-tune the pre-trained Llama-2 model, and use the sample data consisting of labeled PoC and CVE data to fine-tune the Llama-2 model to enhance the model's ability to recognize and understand CVE and PoC data. Then, based on the fine-tuned Llama-2 model, identify and process the input target CVE and PoC information, output the correlation index, and judge the correlation relationship between CVE and PoC based on the correlation index. Using the Llama-2 model with a smaller number of parameters in the local environment can reduce the risk of data leakage while obtaining stronger language processing capabilities, so that the model can better adapt to specific tasks. Finally, for the target CVE, after it is processed by the system together with the data in the existing PoC library, it can be obtained whether there is a corresponding correlation relationship, thereby solving the problem of relying on manual analysis when identifying the association between CVE and PoC. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flowchart of a vulnerability information identification method based on a large language model shown in an embodiment of the present disclosure.

[0042] Figure 2 It is a schematic diagram of GPT data enhancement shown in an embodiment of the present disclosure.

[0043] Figure 3 It is a schematic diagram of the Llama-2 model identifying vulnerability information shown in an embodiment of the present disclosure.

[0044] Figure 4It is a block diagram of a vulnerability information identification device based on a large language model shown in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0045] The present disclosure is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0046] Unless otherwise defined, the technical and scientific terms used in the present disclosure have the same meanings as those generally understood by those skilled in the art to which the present disclosure belongs; the terms used in the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure; the term "including" and any variations thereof in the present disclosure are intended to cover non-exclusive inclusions. Obviously, the embodiments described in the present disclosure are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative efforts are within the scope of protection of the present disclosure.

[0047] Reference to "embodiments" in this disclosure means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the disclosure. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0048] Figure 1 is a flow chart of a vulnerability information identification method based on a large language model shown in an embodiment of the present disclosure. The method can be executed by a terminal device, wherein the terminal device can be a server, a desktop computer, a laptop computer, etc. The embodiment of the present disclosure does not limit the type of terminal device. Figure 1 As shown, the method includes:

[0049] Step 1: Training data collection. Collect CVE information published on various CVE publishing platforms, collect existing PoC information for CVE on the Internet, and manually label the collected information. The label can indicate whether a pair of CVE and PoC data has a correlation.

[0050] As an example of this embodiment, in step 1, the terminal device can collect CVE information published in various CVE publishing platforms. The existing PoC information for certain CVEs on the Internet is collected, and noise information irrelevant to vulnerability information in CVE and PoC information is removed, and some data is manually labeled according to the relationship between CVE and PoC.

[0051] Step 2, GPT data processing, using the GPT-3.5-turbo model to identify and summarize CVE and PoC data, and describe each CVE and PoC information pair based on the TTP (Tactics, Techniques, and Procedures) content form. The description items of each information include tactics, techniques, and procedures to form a fine-tuning data set. In this way, PoC and CVE information of different styles are adjusted to triple data in the form of tactics, techniques, and procedures.

[0052] As an example of this embodiment, step 2 may include:

[0053] Step 21, based on the form of question-answer interaction, use a template that combines instructions, examples, and input to guide the GPT model to understand the input CVE or PoC information and output content; where instructions are used to describe the generated tasks; examples are used to describe the input and output of the tasks in the instructions, and to give the expected output structure; input is used to describe the CVE or PoC content to be processed.

[0054] Step 22, taking the labeled CVE and PoC datasets as examples, guiding the GPT model to identify the datasets, converting CVE and PoC into corresponding TTP form data as the fine-tuning dataset for the subsequent Llama model.

[0055] Step 3: GPT data enhancement: Based on the existing description information, the GPT-3.5-turbo model is used to perform data enhancement on the fine-tuning dataset to increase the number and diversity of labeled samples.

[0056] Figure 2 is a schematic diagram of GPT data enhancement shown in an embodiment of the present disclosure, such as Figure 2 As shown, step 3 includes:

[0057] Step 31: Perform data enhancement in the form of translated language. For the original CVE or PoC data, translate the original language into another language and then translate it back to the original language, converting CVE and PoC into corresponding TTP form data as a supplement to the fine-tuning dataset.

[0058] Step 32: Use the GPT model to rewrite the natural language information in the original CVE or PoC, and require that the GPT data does not change the original semantics, and the rewritten information replaces the original information. Then convert the CVE and PoC into corresponding TTP format data and use it as a supplement to the fine-tuning dataset.

[0059] Step 4: Llama model tuning, use the sample data consisting of labeled PoC and CVE data to fine-tune the Llama-2 model.

[0060] As an example of this embodiment, when fine-tuning the model, the TTP form information of each CVE and PoC is input into the Llama-2 model, wherein the TTP form information of each CVE and PoC includes three aspects: tactics, technology, and process. The CVE and PoC pairs with corresponding relationships are used as positive sample pairs, and the CVE and PoC pairs without corresponding relationships are used as negative sample pairs; the Llama-2 model is trained and fine-tuned based on the positive and negative sample pairs, and finally outputs the judgment result of whether the CVE and PoC have an associated corresponding relationship.

[0061] In one possible implementation, samples generated by data augmentation constitute negative sample pairs.

[0062] Step 41: Input positive sample pairs to the model And negative sample pairs like Figure 2 As shown in Figure 1, the model learns feature information from the input, converts the input into a vector containing feature information, and calculates the similarity between vectors using the cosine similarity method, including the similarity between positive sample pairs. The similarity between the negative sample pairs

[0063] Step 42, calculating a loss function based on the similarity between the positive sample pairs and the similarity between the negative sample pairs.

[0064] InfoNCE loss is used as the loss function in the fine-tuning process, as shown in the following formula:

[0065]

[0066] in, is with x i Similar positive samples, is with x i Dissimilar negative samples, sim(x i ,x j ) represents x i With x j The similarity between them is measured by cosine similarity. τ is the temperature parameter used to adjust the scale of similarity during the learning process.

[0067] Step 43: Perform model training and fine-tuning on the fine-tuning dataset until the model loss tends to be stable. Then fix the model parameters and save the model.

[0068] Step 5: Based on the fine-tuned Llama-2 model, the input target CVE and PoC information is identified and processed, and the correlation index is output. The correlation relationship between CVE and PoC is judged based on the correlation index.

[0069] Figure 3 is a schematic diagram of the Llama-2 model identifying vulnerability information shown in an embodiment of the present disclosure, such as Figure 3 As shown, step 5 includes.

[0070] In step 51, the CVE and PoC information to be tested will be input into the GPT model and processed in the form of step 2 to output data in the form of TTP.

[0071] Step 52, input the CVE and POC data in TTP form into the Llama model, and calculate their similarity based on the cosine similarity method. When the similarity exceeds the preset threshold, it is determined that there is a corresponding association relationship between the CVE and the POC. When the similarity does not exceed the preset threshold, it is determined that there is no corresponding association relationship between the CVE and the POC.

[0072] For the target CVE, after it is processed together with the data in the existing PoC library by the system, it can be obtained whether there is a corresponding association relationship, thereby solving the problem of relying on manual analysis when identifying the association between CVE and PoC.

[0073] The vulnerability information identification method based on the large language model provided by the present disclosure collects the CVE information published in each CVE publishing platform, collects the existing PoC information for certain CVEs on the Internet, and manually annotates some data, thereby enhancing the accuracy and reliability of the training data in the form of manual annotation, thereby improving the learning ability of the subsequent model. By using the GPT-3.5-turbo model to identify and summarize CVE and PoC data, each piece of information is described based on the TTP (Tactics, Techniques, and Procedures) content form (the description items of each piece of information include tactics, techniques and procedures), and the CVE and PoC information are unified into the TTP standard format, eliminating the noise interference in the original information, which can enhance the recognition and learning ability of the large language model for key information features, and use instruction fine-tuning technology to improve the GPT model to complete the task effect. Then, based on the existing description information, the GPT-3.5-turbo model is used to enhance the data that has been manually labeled, thereby increasing the number, scale and diversity of label samples, and eliminating the complex operations of manual participation. Then, fine-tune the pre-trained Llama-2 model, and use the sample data consisting of labeled PoC and CVE data to fine-tune the Llama-2 model to enhance the model's ability to recognize and understand CVE and PoC data. Then, based on the fine-tuned Llama-2 model, identify and process the input target CVE and PoC information, output the correlation index, and judge the correlation relationship between CVE and PoC based on the correlation index. Using the Llama-2 model with a smaller number of parameters in the local environment can reduce the risk of data leakage while obtaining stronger language processing capabilities, so that the model can better adapt to specific tasks. Finally, for the target CVE, after it is processed by the system together with the data in the existing PoC library, it can be obtained whether there is a corresponding correlation relationship, thereby solving the problem of relying on manual analysis when identifying the association between CVE and PoC.

[0074] According to another aspect of an embodiment of the present disclosure, a vulnerability information identification device based on a large language model is provided, the device comprising:

[0075] The training data collection module is used to collect CVE information published on various CVE publishing platforms, collect existing PoC information on CVE on the Internet, and manually label the collected information. The label can be used to describe whether a pair of CVE and PoC data has a correlation relationship;

[0076] The GPT data processing module is used to identify and summarize CVE and PoC data using the GPT-3.5-turbo model, and describe each CVE and PoC information pair based on the TTP content form. The description items of each information include tactics, techniques, and procedures to form a fine-tuning data set;

[0077] GPT data enhancement module, used to perform data enhancement on the fine-tuning dataset using the GPT-3.5-turbo model;

[0078] Llama model tuning module, used to fine-tune the Llama-2 model using sample pairs consisting of labeled PoC and CVE data;

[0079] The judgment module is used to identify and process the input target CVE and PoC information based on the fine-tuned Llama-2 model, output the correlation index, and judge the correlation relationship between CVE and PoC based on the correlation index.

[0080] The description of the above-mentioned device has been explained in detail in the description of the above-mentioned method, and will not be repeated here.

[0081] Figure 4 1900 is a block diagram of a vulnerability information identification device based on a large language model according to an embodiment of the present disclosure. For example, the device 1900 may be provided as a server. Figure 4 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0082] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output (I / O) interface 1958. The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0083] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0084] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0085] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0086] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0087] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0088] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0089] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0090] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0091] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0092] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A vulnerability information identification method based on a large language model, characterized in that: The method comprises: Step 1: Training data collection: collect CVE information published on various CVE publishing platforms, collect existing PoC information for CVE on the Internet, and manually label the collected information. The label can be used to describe whether a pair of CVE and PoC data has a correlation relationship; Step 2: GPT data processing. The GPT-3.5-turbo model is used to identify and summarize CVE and PoC data. Each CVE and PoC information pair is described based on the TTP content form. The description items of each information include tactics, techniques, and procedures to form a fine-tuning data set. Step 3, GPT data enhancement: Use the GPT-3.5-turbo model to perform data enhancement on the fine-tuning dataset; Step 4: Llama model tuning: Use the sample data consisting of labeled PoC and CVE data to fine-tune the Llama-2 model; Step 5: Based on the fine-tuned Llama-2 model, the input target CVE and PoC information is identified and processed, and the correlation index is output. The correlation relationship between CVE and PoC is judged based on the correlation index.

2. The method according to claim 1, characterized in that Step 2 includes: Step 21, based on the form of question-answer interaction, use a template that combines instructions, examples, and input to guide the GPT model to understand the input CVE or PoC information and output content; where the instructions are used to describe the generated tasks; the examples are used to describe the input, output, and expected output structure of the tasks in the instructions; the input is used to describe the CVE or PoC content to be processed; Step 22, taking the labeled CVE and PoC datasets as examples, guiding the GPT model to identify the datasets, converting CVE and PoC into corresponding TTP form data as the fine-tuning dataset for the subsequent Llama model.

3. The method according to claim 1, characterized in that Step 3 includes: Step 31, perform data enhancement in the form of translated language; for the original CVE or PoC data, translate the original language into another language and then translate it back to the original language, convert CVE and PoC into corresponding TTP form data as a supplementary part of the fine-tuning dataset; Step 32: Use the GPT model to rewrite the natural language information part of the original CVE or PoC, and require that the GPT data does not change the original semantics, and the rewritten information replaces the original information; then convert the CVE and PoC into corresponding TTP form data and use it as a supplementary part of the fine-tuning data set.

4. The method according to claim 1, characterized in that: In step 4, the TTP form information of each CVE and PoC is input into the Llama-2 model, wherein the TTP form information of each CVE and PoC includes three aspects: tactics, technology, and process; the CVE and PoC pairs with corresponding relationships are taken as positive sample pairs, and the CVE and PoC pairs without corresponding relationships are taken as negative sample pairs; the Llama-2 model is trained and fine-tuned based on the positive and negative sample pairs, and finally outputs the judgment result of whether the CVE and PoC have an associated corresponding relationship.

5. The method according to claim 4, characterized in that The samples generated by data augmentation constitute negative sample pairs.

6. The method according to claim 1, characterized in that Step 4 includes: Step 41: Input positive sample pairs to the model And negative sample pairs As shown in Figure 2, the model learns feature information from the input, converts the input into a vector containing feature information, and calculates the similarity between vectors using the cosine similarity method, including the similarity between positive sample pairs. The similarity between the negative sample pairs Step 42, calculating a loss function based on the similarity between the positive sample pairs and the similarity between the negative sample pairs; InfoNCE loss is used as the loss function in the fine-tuning process, as shown in the following formula: in, is with x i Similar positive samples, is with x i Dissimilar negative samples, sim(x i ,x j )express x i With x j The similarity between them, here we use cosine similarity; τ is the temperature parameter, which is used to adjust the scale of similarity during the learning process; Step 43, perform model training and fine-tuning on the fine-tuning dataset until the model loss tends to be stable; then fix the model parameters and save the model.

7. The method according to claim 1, characterized in that Step 5 includes: Step 51, the CVE and PoC information to be tested will be input into the GPT model, and processed in the form of step 2, and the data in the form of TTP will be output; Step 52, input the CVE and POC data in TTP form into the Llama model, and calculate their similarity based on the cosine similarity method. When the similarity exceeds the preset threshold, it is determined that there is a corresponding association relationship between the CVE and the POC. When the similarity does not exceed the preset threshold, it is determined that there is no corresponding association relationship between the CVE and the POC.

8. A vulnerability information identification device based on a large language model, characterized in that: The device comprises: The training data collection module is used to collect CVE information published on various CVE publishing platforms, collect existing PoC information on CVE on the Internet, and manually label the collected information. The label can be used to describe whether a pair of CVE and PoC data has a correlation relationship; The GPT data processing module is used to identify and summarize CVE and PoC data using the GPT-3.5-turbo model, and describe each CVE and PoC information pair based on the TTP content form. The description items of each information include tactics, techniques, and procedures to form a fine-tuning data set; GPT data enhancement module, used to perform data enhancement on the fine-tuning dataset using the GPT-3.5-turbo model; Llama model tuning module, used to fine-tune the Llama-2 model using sample pairs consisting of labeled PoC and CVE data; The judgment module is used to identify and process the input target CVE and PoC information based on the fine-tuned Llama-2 model, output the correlation index, and judge the correlation relationship between CVE and PoC based on the correlation index.

9. A vulnerability information identification device based on a large language model, characterized in that: The device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images