Vulnerability load classification method and system based on codes and semantic features

Abstract

The invention discloses a vulnerability load classification method and system based on codes and semantic features, and belongs to the technical field of intrusion detection. The method comprises the following steps: encoding a vulnerability load, masking an encoding result, and obtaining a first representation of the vulnerability load through predicting a mask value; acquiring a second representation of the vulnerability load based on a TF-IDF encoder; synthesizing the first representation and the second representation to obtain a comprehensive representation of the vulnerability load; and classifying the comprehensive representation to obtain a classification result of the vulnerability load. According to the method and the device, the vulnerability detection accuracy is improved by enhancing the characterization capability of the vulnerability load.

Description

Technical Field

[0001] The present invention belongs to the technical field of intrusion detection, and in particular relates to a vulnerability load classification method and system based on code and semantic features. Background Art

[0002] At present, intrusion detection systems play a vital role in the field of network security. As network attack methods continue to evolve and become more complex, traditional intrusion detection methods mainly rely on rule matching and classic machine learning techniques. However, in the face of increasingly diverse and complex new attack scenarios, these methods often show high false positive and false negative rates, especially in the detection of vulnerability payloads. Traditional methods are difficult to provide fine-grained identification and classification of vulnerability attack payloads, thus limiting the ability to accurately assess security threats and respond quickly.

[0003] With the continuous advancement of deep learning technology, new development opportunities have been brought to vulnerability payload classification. Convolutional neural networks (CNN), recurrent neural networks (RNN), and Transformer-based models can automatically extract features from large-scale data and achieve accurate detection and classification. However, the adaptability of general pre-trained models in the field of vulnerability payloads is relatively limited, so models specifically pre-trained for vulnerability payloads are particularly important.

[0004] Through pre-training, the model can deeply learn and understand the semantic and structural characteristics of vulnerability payloads, thereby significantly improving the representation ability in various vulnerability payload classification tasks. This refined representation enables the model to achieve higher accuracy and efficiency in downstream tasks of various vulnerability payload classification, thereby achieving more accurate and efficient vulnerability payload detection and classification. Summary of the invention

[0005] In view of the shortcomings and defects of the prior art, the present invention provides a vulnerability load classification method and system based on code and semantic features, aiming to improve the accuracy of vulnerability detection by enhancing the characterization capability of vulnerability loads.

[0006] To achieve the above-mentioned object of the invention, the technical solution of the present invention includes the following contents.

[0007] A vulnerability load classification method based on code and semantic features, the method comprising:

[0008] Encoding the vulnerable payload, and after masking the encoding result, obtaining a first representation of the vulnerable payload by predicting the mask value;

[0009] Obtain a second representation of the vulnerability payload based on a TF-IDF encoder;

[0010] Combining the first representation and the second representation to obtain a comprehensive representation of the vulnerability load;

[0011] The comprehensive representation is classified to obtain a classification result of the vulnerability load.

[0012] Further, encoding the vulnerable payload and masking the encoding result includes:

[0013] Collect and build a pre-trained corpus including vulnerability information and load information;

[0014] Based on the pre-trained corpus, a fine-grained tokenizer in the field of vulnerability load classification is constructed;

[0015] Using the fine-grained word segmenter to segment the vulnerability payload;

[0016] Use one symbol [CLS] and multiple symbols [SEP] to specially mark the word segmentation to obtain the encoding result; among them, the symbol [CLS] is the beginning mark of the sentence, and the symbol [SEP] is the middle and end mark of the sentence;

[0017] Use the symbol [MASK] to randomly mask some words in the encoded result.

[0018] Furthermore, the collecting and constructing of a pre-trained corpus including vulnerability information and load information includes:

[0019] Collect and parse the system rule information of the intrusion detection system to obtain vulnerability information and load information in the intrusion detection system rules, wherein the vulnerability information in the intrusion detection system rules includes: port, category and reference link, and the load information in the intrusion detection system rules includes: content and regular expression;

[0020] Using a crawler to collect vulnerability information and payload information of a vulnerability library; wherein the vulnerability information of the vulnerability library includes: vulnerability number, vulnerability description, vulnerability category, vulnerability category description and vulnerability rating, and the payload information of the vulnerability library includes: payload;

[0021] Capturing malicious traffic, and using the tshark tool in the Wireshark toolkit to extract load information in the malicious traffic, wherein the load information in the malicious traffic includes: port and application layer load content;

[0022] Collect vulnerability information and load information of the penetration testing tool, wherein the vulnerability information of the penetration testing tool includes: vulnerability number, vulnerability description, vulnerability category and vulnerability category description, and the load information of the penetration testing tool includes: request, http and tag;

[0023] The text data of the pre-trained corpus is obtained by synthesizing the vulnerability information and load information in the intrusion detection system rules, the vulnerability information and load information in the vulnerability library, the load information in the malicious traffic, and the vulnerability information and load information of the penetration testing tool;

[0024] The Prompt prompt information is written using the vulnerability number as a connection, and the text data of the pre-trained corpus is formatted into six-tuple data in combination with the language model, so as to obtain a pre-trained corpus C1 including vulnerability information and load information; wherein, the pre-trained corpus C1 = {id, description, payload, cwe_id, cwe_description, level}, id represents the vulnerability number, description is the vulnerability description, payload is the vulnerability load, cwe_id is the vulnerability category number, cwe_description is the vulnerability category description, and level is the vulnerability rating.

[0025] Furthermore, based on the pre-trained corpus, a fine-grained word segmenter in the field of vulnerability load classification is constructed, including:

[0026] Create a byte pair encoding word segmenter, and use the byte pair encoding word segmenter to obtain high-frequency words and subwords in the text data of the pre-training corpus to obtain a vocabulary bpe_vocab;

[0027] Extracting special coded words related to vulnerability description and attack mode from the text data of the pre-training corpus to obtain a vocabulary special_words;

[0028] Using the vocabulary special_words to perform word-level segmentation on the pre-trained corpus, obtaining a vocabulary special_vocab;

[0029] Performing operations of case transformation and random character repeating on the words in the vocabulary special_vocab to obtain a vocabulary vul_words;

[0030] According to the load information of the pre-training corpus, concatenate the words in the vocabulary special_words and the vocabulary vul_words to obtain the vocabulary concatenate_vocab;

[0031] The vocabulary bpe_vocab, the vocabulary special_words, the vocabulary special_vocab, the vocabulary vul_words and the vocabulary concatenate_vocab are integrated to obtain a combined vocabulary combined_vocab;

[0032] A word segmenter is constructed, and the word segmenter is trained based on the combined vocabulary combined_vocab to obtain a fine-grained word segmenter in the field of vulnerability load classification.

[0033] Further, obtaining a first representation of the vulnerability load by predicting the mask value includes:

[0034] Build a model based on the RoBERTa pre-trained model;

[0035] On the pre-trained corpus, the model is pre-trained using an adversarial perturbation training strategy and a mask task strategy to obtain a VulBERT model; wherein the adversarial perturbation training strategy is to use a random noise matrix to perform adversarial perturbations on an original embedding matrix generated based on the pre-trained corpus encoding, and the mask task strategy is to predict a mask value of the pre-trained corpus encoding, and the pre-trained corpus encoding is generated based on the pre-trained corpus and the fine-grained word segmenter;

[0036] The masked encoding result is input into the VulBERT model, and the state value h of each layer of the VulBERT model is extracted. j After that, the state value h j The mean of is taken as the first representation of the vulnerability load.

[0037] Furthermore, when the model is pre-trained, the loss function L_total = α·Loss(f(E1), y)+β·Loss(f(E1+espsilon*P), y); wherein Loss is the loss function, f is the output function of the model, y is the original word before being masked by [MASK], E1 represents the original embedding matrix, espsilon represents the adversarial perturbation parameter, P represents the random noise matrix, α represents the first weight parameter, and β represents the second weight parameter.

[0038] Furthermore, the comprehensive representation is classified to obtain the classification result of the vulnerability load, including:

[0039] Building a machine learning model on the pre-trained corpus;

[0040] The comprehensive representation is input into the machine learning model to obtain a classification result of the vulnerability load.

[0041] A vulnerability load classification system based on code and semantic features, the system comprising:

[0042] A first characterization module, used for encoding the vulnerability payload, and after masking the encoding result, obtaining a first characterization of the vulnerability payload by predicting the mask value;

[0043] A second representation module, configured to obtain a second representation of the vulnerability payload based on a TF-IDF encoder;

[0044] A classification module is used to integrate the first representation and the second representation to obtain a comprehensive representation of the vulnerability load, and classify the comprehensive representation to obtain a classification result of the vulnerability load.

[0045] An electronic device, characterized in that the electronic device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements any of the above-mentioned vulnerability load classification methods based on code and semantic features.

[0046] A computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, any of the above-mentioned vulnerability load classification methods based on code and semantic features is implemented.

[0047] Compared with the prior art, the present invention has at least the following beneficial effects.

[0048] 1. In order to solve the problem of low accuracy in vulnerability load classification, the present invention designs an efficient vulnerability load classification method based on code and semantic features. Compared with the method of characterizing and training on limited training data, the present invention performs pre-training on the vulnerability load corpus to obtain the vulnerability load VulBERT model, which incorporates vulnerability load prior knowledge and improves the characterization capability of vulnerability load data.

[0049] 2. The present invention designs a pre-training strategy based on the Transformer Encoder RoBERTa pre-trained mask model combined with anti-perturbation. This strategy is more suitable for the problem of high noise information in the vulnerability load in the real network environment, and the anti-perturbation strategy can be flexibly configured in different application scenarios, making the present invention more widely applicable;

[0050] 3. The present invention obtains a vulnerability load pre-training model VulBERT model, which has wider applicability and more accurate effects in various classification tasks such as vulnerability load category classification and severity classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The present invention provides a flowchart of a vulnerability load classification method based on code and semantic features.

[0052] Figure 2 A schematic diagram of a feature enhancement model based on adversarial disturbance proposed in an embodiment of the present invention.

[0053] Figure 3A flowchart of collecting and constructing a pre-trained corpus including vulnerability information and load information proposed in an embodiment of the present invention.

[0054] Figure 4 This is a workflow diagram of the vulnerability load classification module proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will combine the drawings in the embodiments of the present invention to conduct a more comprehensive and complete analysis and expression of the technical solutions in the examples of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, in order to further explain the present invention so that those skilled in the art can clearly and thoroughly understand the present invention, and are not used to limit the present invention.

[0056] like Figure 1 As shown, the vulnerability load classification method based on code and semantic features of the present invention includes the following steps.

[0057] Step 1: Encode the vulnerability payload, mask the encoding result, and obtain the first representation of the vulnerability payload by predicting the mask value.

[0058] This step is to first collect a large number of corpora related to vulnerabilities and loads, build a fine-grained word segmenter suitable for the vulnerability load field, and then use the RoBERTa pre-training model and combine it with the adversarial perturbation training strategy for pre-training. Figure 2 The feature enhancement model (VulBERT model) based on adversarial perturbations is shown, and finally the first representation of the vulnerability load is obtained based on the VulBERT model.

[0059] Step 1.1: Collect and build a pre-training corpus including vulnerability information and payload information.

[0060] The present invention constructs a pre-trained corpus C1 including vulnerability information and load information, such as Figure 3 As shown, including:

[0061] (1) Collecting intrusion detection system rule information, parsing intrusion detection system Snort rules and Suricata rules, and extracting vulnerability information and load information in the rules, wherein the vulnerability information in the intrusion detection system rules includes: port, category (classtype), reference link (reference), and the load information in the intrusion detection system rules includes: content (content), regular expression (pcre);

[0062] (2) Using crawlers to collect vulnerability information and payload information of the vulnerability library, wherein the vulnerability information of the vulnerability library includes: vulnerability number (id), vulnerability description (description), vulnerability category (cwe_id), vulnerability category description (cwe_description), vulnerability rating (level), and the payload information of the vulnerability library includes: payload;

[0063] (3) Capturing malicious traffic and using the tshark tool in the Wireshark toolkit to extract the load information in the malicious traffic, wherein the load information in the malicious traffic includes: port and application layer load content;

[0064] (4) Collect vulnerability information and load information of the public penetration testing tools Pocsuite and Nuclei. The vulnerability information of the penetration testing tools includes: vulnerability number (cve_id), vulnerability description (description), vulnerability category (cwe_id), and vulnerability category description (cwe_description). The load information of the penetration testing tools includes: request (request), http, and tags (tags);

[0065] (5) Based on the language model GPT-3.5, all the above information is connected with the vulnerability number to form text data, and prompt information is written. The text data is formatted into six-tuple data, thereby constructing a pre-trained corpus C1 including vulnerability information and payload information. The format is a YAML file containing six-tuple data, which can be expressed as: C1 = {id, description, payload, cwe_id, cwe_description, level}

[0066] Among them, C1 is the pre-trained corpus, id represents the vulnerability number, description is the vulnerability description, payload is the vulnerability payload, cwe_id is the vulnerability category number, cwe_description is the vulnerability category description, and level is the vulnerability rating.

[0067] Step 1.2: Based on the pre-trained corpus, build a fine-grained tokenizer in the field of vulnerability payload classification.

[0068] The present invention is based on the pre-trained corpus C1 of vulnerability loads, combines multiple word segmentation strategies, and constructs a fine-grained word segmenter T1 in the field of vulnerability load classification, including:

[0069] (1) Create a byte pair encoding (BPE) tokenizer. First, collect all the text data in the pre-training corpus C1. Then, input this data into the BPE tokenizer for training to achieve dynamic vocabulary expansion. During the training process, the tokenizer will identify and learn common subword units and byte pairs in the text, thereby generating a vocabulary containing high-frequency words and subword combinations. After the training is completed, the resulting BPE vocabulary is named bpe_vocab, which will serve as the basis for subsequent token segmentation and feature extraction.

[0070] (2) In-depth analysis of the corpus features of specialized fields. By performing statistics and feature extraction on the text in the corpus, special encoding rules applicable to the vulnerability payload field are determined. Special attention is paid to the payload field in the corpus, and all occurrences of special encoding words are extracted. These special words are usually related to vulnerability descriptions, attack patterns, etc., and the extracted words are finally named special_words.

[0071] (3) Use the obtained special coded vocabulary special_words to perform word-level segmentation on the pre-trained corpus C1. In this process, the word segmenter will segment the text according to the vocabulary rules defined in special_words to ensure that the specific vocabulary related to the vulnerability payload can be correctly identified and retained. The final segmentation result is named special_vocab.

[0072] (4) Create a special vocabulary vul_vocab for vulnerability payloads. First, select relevant words from special_vocab and perform case transformation to ensure that the same words in different formats can be captured. In addition, the operation of randomly repeating characters will also be applied to increase the robustness of the model and its adaptability to diverse inputs. The generated new vocabulary is named vul_words.

[0073] (5) For the payload field in the corpus, concatenate the special coded words special_words and the special word vul_vocab. The concatenation depth is set to 5, that is, a maximum of 5 words are allowed to be combined when combining words. This step ensures that more context information can be included in the vocabulary, thereby improving the performance of the word segmenter. The final concatenated vocabulary is named concatenate_vocab.

[0074] (6) Combine all the above vocabularies (bpe_vocab, special_words, special_vocab, vul_vocab and concatenate_vocab) to form a final combined vocabulary, named combined_vocab.

[0075] (7) Construct a word segmenter and train the word segmenter based on the combined vocabulary combined_vocab to obtain a fine-grained word segmenter in the field of vulnerability load classification.

[0076] (8) The fine-grained tokenizer T1 in the vulnerability load classification domain built based on the pre-trained corpus C1 is saved. The saving process ensures that the tokenizer can be easily loaded and applied in the future to support subsequent vulnerability load analysis and classification tasks.

[0077] Step 1.3: Based on the pre-trained corpus C1 and the fine-grained tokenizer T1, configure the special tagging strategy post-processor mechanism, configure the masking mechanism, and generate the pre-trained corpus encoding B1.

[0078] The present invention is based on the pre-trained corpus C1 and the fine-grained word segmenter T1, configures a special tagging strategy post-processor mechanism, configures a mask mechanism, and generates a pre-trained corpus encoding B1, including:

[0079] (1) Input and tag configuration: Input the pre-trained corpus C1 into the fine-grained tokenizer and load a RoBERTa post-processor. Set a special tagging strategy, set to use 1 symbol [CLS] and multiple symbols [SEP] for special tagging, where [CLS] is the sentence start marker, [SEP] is the sentence middle and sentence end marker, which can be represented in the C1 six-tuple as [cls]id, [sep]description, [sep]payload, [sep]cwe_id, [sep]cwe_descriptions, [sep]severity[sep];

[0080] (2) Masking mechanism configuration: Configure the masking mechanism and use the symbol [MASK] to randomly mask some words in the corpus C1. The [MASK] symbol represents the masked words, which aims to enhance the pre-training effect of the model.

[0081] (3) Generating results: After the above processing, the final result data generated is the pre-training corpus encoding B1.

[0082] Step 1.4: Get the first characterization of the vulnerability load by predicting the mask value.

[0083] The present invention builds a RoBERTa model, inputs a pre-training corpus code B1, adopts an adversarial perturbation training strategy and a mask task strategy, performs pre-training, obtains a VulBERT model, and then calculates a first representation of the vulnerability load to be classified based on the VulBERT model, including:

[0084] (1) Build a model based on the RoBERTa pre-trained model and adjust the size of the model to the size of the combined vocabulary combined_vocab;

[0085] (2) The model uses the mask task strategy, increases the mask task probability Pmask parameter, and sets the half-precision floating point training strategy to make the pre-trained mask model guess the value masked by [MASK], defined as y;

[0086] (3) Define the adversarial perturbation strategy, divide the pre-trained corpus encoding B1 into multiple original embedding matrices, defined as E1, and for each original embedding matrix E1, calculate the model's loss L1 on the original input embedding; randomly generate a random noise matrix of the same dimension, defined as P, set the adversarial perturbation parameter espsilon, form the adversarial embedding matrix E2 = E1 + espsilon * P, and calculate the loss L2 of the perturbed input embedding; calculate the total loss as the weighted sum of the original loss and the perturbed loss, assign different weights to the loss L1 and the loss L2, and perform loss back propagation. The formula of the adversarial perturbation loss function is as follows:

[0087] L_total=α·Loss(f(E1),y)+β·Loss(f(E1+espsilon*P),y)

[0088] Among them, Loss is the loss function (such as cross entropy loss), which is obtained through normal forward propagation and back propagation, f is the output function of the model, and y is the original word before being masked by [MASK].

[0089] (4) Repeat step (3) to complete multiple training of the pre-training corpus encoding B1;

[0090] (5) Save the trained model as a VulBERT model.

[0091] (6) For the vulnerability payload to be classified, load the fine-grained tokenizer T1 for encoding, use the post-processor for masking and special marking, input it into the VulBERT model, extract the last layer state value of the model, denoted as h, calculate the mean of the state value as the first representation, denoted as r, L is the number of layers of the VulBERT model, and the formula is expressed as:

[0092] Step 2: Obtain the second representation of the vulnerability payload based on the TF-IDF encoder.

[0093] In order to enhance the ability to express vulnerability load characteristics, the present invention also introduces a TF-IDF encoder to obtain a second representation of the vulnerability load to be classified, expressed as tfidf.

[0094] Step 3: The first representation and the second representation are combined to obtain a comprehensive representation of the vulnerability load, and the comprehensive representation is classified to obtain a classification result of the vulnerability load.

[0095] Figure 4 The flowchart of the vulnerability load classification module is shown in Figure 1. First, the present invention assigns different weights to the first representation r generated by the pre-trained model and the second representation tfidf generated by the TF-IDF encoder, stacks them horizontally (i.e., by column), and forms a comprehensive representation, which is represented by A1. The formula is

[0096]

[0097] Among them, α is the weight of the representation r output by the pre-trained model, h is the state value of the last layer of the pre-trained model, β is the weight of the TF-IDF feature vector, and tfidf is the TF-IDF feature vector.

[0098] Finally, the load data representation A1 is put into the machine learning model for classification to obtain the classification result. The machine learning model is trained based on the pre-training corpus.

[0099] Based on the same concept, the present invention also discloses a vulnerability load classification system based on code and semantic features, including:

[0100] A first characterization module, used for encoding the vulnerability payload, and after masking the encoding result, obtaining a first characterization of the vulnerability payload by predicting the mask value;

[0101] A second representation module, configured to obtain a second representation of the vulnerability payload based on a TF-IDF encoder;

[0102] A classification module is used to integrate the first representation and the second representation to obtain a comprehensive representation of the vulnerability load, and classify the comprehensive representation to obtain a classification result of the vulnerability load.

[0103] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions of the technical solutions of the present invention made by ordinary technicians in the field should all be included in the protection scope of the present invention. The protection scope of the present invention shall be based on what is defined in the claims.

Claims

1. A vulnerability load classification method based on code and semantic features, characterized in that: The method comprises: Encoding the vulnerable payload, and after masking the encoding result, obtaining a first representation of the vulnerable payload by predicting the mask value; Obtain a second representation of the vulnerability payload based on a TF-IDF encoder; Combining the first representation and the second representation to obtain a comprehensive representation of the vulnerability load; The comprehensive representation is classified to obtain a classification result of the vulnerability load.

2. The method according to claim 1, characterized in that The encoding of the vulnerable payload and masking of the encoding result includes: Collect and build a pre-trained corpus including vulnerability information and load information; Based on the pre-trained corpus, a fine-grained tokenizer in the field of vulnerability load classification is constructed; Using the fine-grained word segmenter to segment the vulnerability payload; Use one symbol [CLS] and multiple symbols [SEP] to specially mark the word segmentation to obtain the encoding result; among them, the symbol [CLS] is the beginning mark of the sentence, and the symbol [SEP] is the middle and end mark of the sentence; Use the symbol [MASK] to randomly mask some words in the encoded result.

3. The method according to claim 2, characterized in that The collecting and constructing of a pre-trained corpus including vulnerability information and load information includes: Collect and parse the system rule information of the intrusion detection system to obtain vulnerability information and load information in the intrusion detection system rules, wherein the vulnerability information in the intrusion detection system rules includes: port, category and reference link, and the load information in the intrusion detection system rules includes: content and regular expression; Using a crawler to collect vulnerability information and payload information of a vulnerability library; wherein the vulnerability information of the vulnerability library includes: vulnerability number, vulnerability description, vulnerability category, vulnerability category description and vulnerability rating, and the payload information of the vulnerability library includes: payload; Capturing malicious traffic, and using the tshark tool in the Wireshark toolkit to extract load information in the malicious traffic, wherein the load information in the malicious traffic includes: port and application layer load content; Collect vulnerability information and load information of the penetration testing tool, wherein the vulnerability information of the penetration testing tool includes: vulnerability number, vulnerability description, vulnerability category and vulnerability category description, and the load information of the penetration testing tool includes: request, http and tag; The text data of the pre-trained corpus is obtained by synthesizing the vulnerability information and load information in the intrusion detection system rules, the vulnerability information and load information in the vulnerability library, the load information in the malicious traffic, and the vulnerability information and load information of the penetration testing tool; The Prompt prompt information is written using the vulnerability number as a connection, and the text data of the pre-trained corpus is formatted into six-tuple data in combination with the language model, so as to obtain a pre-trained corpus C1 including vulnerability information and load information; wherein, the pre-trained corpus C1 = {id, description, payload, cwe_id, cwe_description, level}, id represents the vulnerability number, description is the vulnerability description, payload is the vulnerability load, cwe_id is the vulnerability category number, cwe_description is the vulnerability category description, and level is the vulnerability rating.

4. The method according to claim 3, characterized in that Based on the pre-trained corpus, a fine-grained tokenizer in the field of vulnerability load classification is constructed, including: Create a byte pair encoding word segmenter, and use the byte pair encoding word segmenter to obtain high-frequency words and subwords in the text data of the pre-training corpus to obtain a vocabulary bpe_vocab; Extracting special coded words related to vulnerability description and attack mode from the text data of the pre-training corpus to obtain a vocabulary special_words; Using the vocabulary special_words to perform word-level segmentation on the pre-trained corpus, obtaining a vocabulary special_vocab; Performing operations of case transformation and random character repeating on the words in the vocabulary special_vocab to obtain a vocabulary vul_words; According to the load information of the pre-training corpus, concatenate the words in the vocabulary special_words and the vocabulary vul_words to obtain the vocabulary concatenate_vocab; The vocabulary bpe_vocab, the vocabulary special_words, the vocabulary special_vocab, the vocabulary vul_words and the vocabulary concatenate_vocab are integrated to obtain a combined vocabulary combined_vocab; A word segmenter is constructed, and the word segmenter is trained based on the combined vocabulary combined_vocab to obtain a fine-grained word segmenter in the field of vulnerability load classification.

5. The method according to claim 2, characterized in that: A first representation of the vulnerability load is obtained by predicting the mask value, including: Build a model based on the RoBERTa pre-trained model; On the pre-trained corpus, the model is pre-trained using an adversarial perturbation training strategy and a mask task strategy to obtain a VulBERT model; wherein the adversarial perturbation training strategy is to use a random noise matrix to perform adversarial perturbations on an original embedding matrix generated based on the pre-trained corpus encoding, and the mask task strategy is to predict a mask value of the pre-trained corpus encoding, and the pre-trained corpus encoding is generated based on the pre-trained corpus and the fine-grained word segmenter; The masked encoding result is input into the VulBERT model, and the state value h of each layer of the VulBERT model is extracted. j After that, the state value h j The mean of is taken as the first representation of the vulnerability load.

6. The method according to claim 5, characterized in that When pre-training the model, the loss function L_total = α·Loss(f(E1), y)+β·Loss(f(E1+espsilon*P), y); where Loss is the loss function, f is the output function of the model, y is the original word before being masked by [MASK], E1 represents the original embedding matrix, espsilon represents the adversarial perturbation parameter, P represents the random noise matrix, α represents the first weight parameter, and β represents the second weight parameter.

7. The method according to claim 2, characterized in that The comprehensive representation is classified to obtain a classification result of the vulnerability load, including: Building a machine learning model on the pre-trained corpus; The comprehensive representation is input into the machine learning model to obtain a classification result of the vulnerability load.

8. A vulnerability load classification system based on code and semantic features, characterized in that: The system comprises: A first characterization module, used for encoding the vulnerability payload, and after masking the encoding result, obtaining a first characterization of the vulnerability payload by predicting the mask value; A second representation module, configured to obtain a second representation of the vulnerability payload based on a TF-IDF encoder; A classification module is used to integrate the first representation and the second representation to obtain a comprehensive representation of the vulnerability load, and classify the comprehensive representation to obtain a classification result of the vulnerability load.

9. An electronic device, characterized in that: The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the vulnerability load classification method based on code and semantic features as described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the vulnerability load classification method based on code and semantic features as described in any one of claims 1 to 7 is implemented.

Images