A semi-supervised submission-level vulnerability classification method based on reinforcement learning enhancement

Abstract

This application belongs to the field of software vulnerability classification technology, specifically disclosing a semi-supervised submission-level vulnerability classification method based on reinforcement learning. This application first identifies vulnerability fix code changes from patch data; then, it inputs unlabeled vulnerability fix code changes into the teacher model to generate pseudo-labeled sample points, interpolating labeled data points and pseudo-labeled sample points to obtain virtual sample points; next, it uses the mean squared error of the student model on the virtual sample points as a reinforcement learning reward signal, and constructs a confidence-weighted reinforcement learning loss by using the student model's prediction confidence of unlabeled vulnerability fix code changes as the weight of the reward signal; it iteratively trains the teacher model and student model by fusing the confidence-weighted reinforcement learning loss, consistency loss, and supervision loss, until the iteration terminates and the student model is output for vulnerability classification. This vulnerability classification method shows better performance than traditional semi-supervised learning vulnerability classification methods.

Description

Technical Field

[0001] This application belongs to the field of software vulnerability classification technology, and more specifically, relates to a semi-supervised submission-level vulnerability classification method based on reinforcement learning. Background Technology

[0002] With the widespread adoption of open-source software, software supply chain security risks are becoming increasingly severe. Timely identification and remediation of vulnerabilities, i.e., completing a Vulnerability Fix Commit (VFC), is crucial for maintaining open-source software security. However, existing vulnerability identification methods are mostly limited to binary classification problems—only determining whether a commit involves vulnerability remediation, while ignoring fine-grained classification of vulnerability tags.

[0003] To address the classification of submission-level vulnerabilities, the most accurate and effective method is manual annotation. However, manual annotation is costly and time-consuming, making it unsuitable for large-scale vulnerability classification. To tackle large-scale vulnerability classification, researchers have proposed a semi-supervised learning-based method. This method trains a vulnerability classification model using a small amount of manually annotated data and a large amount of unannotated data. However, this method suffers from several drawbacks: pseudo-label generation strategies rely on manual design (e.g., threshold-based, loss-based, or retrieval-based methods), requiring manual optimization for different specific classification tasks, which is time-consuming, labor-intensive, and often fails to meet performance requirements; patch code often contains changes unrelated to security (e.g., comment adjustments, test file updates, or performance optimizations), and directly introducing all patch code as training samples introduces significant training noise, severely interfering with the accuracy of target feature extraction. Due to these shortcomings, existing semi-supervised learning-based vulnerability classification methods still exhibit insufficient performance. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this application is to provide a semi-supervised submission-level vulnerability classification method based on reinforcement learning, which aims to solve the technical problem of insufficient classification performance of existing vulnerability classification methods based on semi-supervised learning.

[0005] The first aspect of this application relates to a semi-supervised submission-level vulnerability classification method based on reinforcement learning, comprising: Identify vulnerability fix code changes from the patch data to be classified; input the vulnerability fix code changes into the trained vulnerability classification model to obtain the vulnerability labels corresponding to the vulnerability fix code changes; The vulnerability classification model is trained through the following steps: Identify vulnerability fix code changes from the patch data submitted for vulnerability fixes; Based on a semi-supervised learning framework, the unlabeled vulnerability repair code changes in the vulnerability repair code changes are input into the teacher model to generate pseudo-labeled sample points. The MixUp technology is used to interpolate the labeled data points in the vulnerability repair code changes and the pseudo-labeled sample points to obtain virtual sample points. The mean squared error of the student model on the virtual sample points is used as the reinforcement learning reward signal, and the prediction confidence of the student model for the changes in the unlabeled vulnerability repair code is used as the weight of the reinforcement learning reward signal to construct a confidence-weighted reinforcement learning loss. The student model is iteratively trained by fusing the confidence-weighted reinforcement learning loss, consistency loss and supervision loss. After each round of training, the teacher model is iteratively updated based on the student model until the iteration termination target is reached. At this time, the student model is the vulnerability classification model that has completed training.

[0006] Preferably, vulnerability fix code changes are identified from patch data, specifically: Based on the patch data, the function nodes involved in the changes are located from the project code property graph, and the function call information and / or variable declaration information of the modified nodes are found by traversing within the function nodes. The patch data is analyzed using an intelligent agent to identify modified nodes. The function call information and / or variable declaration information of the modified nodes are called as needed to determine the relevance between the modified nodes and the vulnerability fix. If the relevance is greater than the relevant threshold, the modified node is identified as a vulnerability fix code change.

[0007] Preferably, the intelligent agent is a large language model with tool invocation capabilities. By designing a prompt template that includes role definition, task description, and output format constraints, the large language model can identify modified nodes from the patch data and determine whether it can determine the relevance of the modified node to the vulnerability fix based on the modified node. If not, it will autonomously decide to invoke a tool to obtain the function call information and / or variable declaration information of the modified node to assist in determining the relevance of the modified node to the vulnerability fix.

[0008] Preferably, the unlabeled vulnerability repair code changes in the vulnerability repair code changes are input into the teacher model to generate pseudo-label sample points. Specifically, the unlabeled vulnerability repair code changes are input into the teacher model to obtain the corresponding vulnerability labels, and the unlabeled vulnerability repair code changes and the corresponding vulnerability labels together constitute the pseudo-label sample points.

[0009] Preferably, the MixUp technique is used to interpolate the tagged data points in the vulnerability fix code changes and the pseudo-tagged sample points to obtain virtual sample points, specifically: Upsample all labeled data points to obtain a set of labeled data points, and make the size of the set of labeled data points equal to the size of the set of pseudo-labeled sample points; The MixUp technique is used to interpolate the vulnerability remediation code changes and vulnerability labels in the labeled data points and pseudo-labeled sample points, and the interpolated vulnerability remediation code changes and vulnerability labels constitute virtual sample points. In the MixUp technology, the sampling parameters of the labeled data points are always greater than the sampling parameters of the pseudo-labeled sample points; and the vulnerability labels are encoded using one-hot encoding.

[0010] Preferred, the first The virtual sample points are specifically: ,in For the first Vulnerability fix code changes in a virtual sample point For the first The vulnerability labels in a set of virtual sample points satisfy: ; ; ; in, For the first Vulnerability fix code changes in tagged data points For the first Vulnerability tags in tagged data points; For the first Vulnerability fix code changes in individual pseudo-label sample points For the first Vulnerability labels in a sample of pseudo-labeled points; The sampling parameters for the label data points, obey distributed, These are the sampling parameters for pseudo-label sample points; all vulnerability labels use one-hot encoding.

[0011] Preferably, the mean squared error of the student model on the virtual sample points is used as the reinforcement learning reward signal. Specifically, the vulnerability repair code changes in the virtual sample points are input into the student model to obtain the predicted vulnerability label. Let the mean squared error between the predicted vulnerability label and the corresponding vulnerability label in the virtual sample point be the single-point mean squared error. The reinforcement learning reward signal is obtained by accumulating all the single-point mean squared errors and taking the negative value.

[0012] Preferably, the confidence-weighted reinforcement learning loss is specifically: ; in, The reinforcement learning loss is weighted by confidence level. For student models Reinforcement learning reward signals, Represents the set of virtual sample points. For student models For the first Code changes to fix an unlabeled vulnerability. Predicted vulnerability label distribution and uniform distribution Between Divergence, used to measure the student model The prediction confidence level; This represents a collection of code changes to fix unlabeled vulnerabilities. The expected data in the data.

[0013] Preferably, the teacher model is iteratively updated based on the student model, specifically by using an exponential moving average algorithm to iteratively update the teacher model based on the student model.

[0014] In a second aspect, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0015] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) Traditional semi-supervised learning relies on a fixed confidence threshold to filter pseudo-labels. Setting the threshold too high can easily lead to missed classifications, while setting it too low can easily introduce noise. It lacks a self-feedback mechanism to autonomously correct the classification model and requires manual adjustment of the confidence threshold. This application integrates MixUp (hybrid reinforcement) technology and a closed-loop self-feedback mechanism of reinforcement learning loss. In this application, virtual sample points between labeled data points and pseudo-label sample points are generated through MixUp technology to evaluate the generalization ability of the classification model as reinforcement learning loss. Through training, the prediction performance of the classification model is guided to evolve towards real missing labels (labeled data points). This realizes an adaptive and optimizable pseudo-label filtering strategy, freeing it from the dependence of traditional semi-supervised learning on manually designed complex filtering strategies.

[0016] (2) The loss function of traditional semi-supervised learning often only includes supervision loss and consistency loss. For vulnerability classification tasks with extremely imbalanced data distribution and high noise, this can easily lead to the model blindly trusting false pseudo-labels with high confidence. In this application, the reinforcement learning loss part is integrated into the loss function, and confidence based on KL divergence is introduced to weight the reinforcement learning loss, so that the classification model automatically pays attention to pseudo-labeled sample points with high confidence during training, thereby further improving the performance of the classification model.

[0017] (3) Screening code changes related to vulnerability fixes from patch code: Traditional methods rely on hard rules for filtering, which cannot identify irrelevant vulnerability fix code at the semantic level. In this application, by combining code parsing with a large language model to build an intelligent agent, it is possible to automatically identify and remove irrelevant vulnerability fix code changes in patch data, thereby improving the quality of training data and further improving the performance of the classification model.

[0018] (4) This application also maintains the stability of the teacher model by using the exponential moving average algorithm, which significantly improves the generalization performance and robustness of the classification model on unseen data. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a semi-supervised submission-level vulnerability classification method based on reinforcement learning, as provided in an embodiment of this application.

[0020] Figure 2 This is a schematic diagram of the training process of the vulnerability classification model provided in the embodiments of this application.

[0021] Figure 3 This is a schematic diagram of the vulnerability repair code modification and filtering output process provided in the embodiments of this application.

[0022] Figure 4 This is a schematic diagram illustrating the process of constructing a confidence-weighted reinforcement learning loss according to an embodiment of this application.

[0023] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0026] In this application, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0027] In this application, the term "electrical connection" can refer to a direct circuit connection or a signal transmission via a communication protocol.

[0028] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0029] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0030] The embodiments of this application are described below with reference to the accompanying drawings.

[0031] like Figure 1 As shown in the figure, this invention discloses a method for semi-supervised submission-level vulnerability classification based on reinforcement learning, which specifically includes the following steps: Identify vulnerability fix code changes from patch data to be categorized.

[0032] Input the vulnerability fix code changes into the trained vulnerability classification model to obtain the vulnerability labels corresponding to the vulnerability fix code changes.

[0033] like Figure 2 As shown, the vulnerability classification model in this application embodiment is trained through the following steps: (1) Identify vulnerability fix code changes from the patch data submitted for vulnerability remediation; such as Figure 3 As shown, the specific steps include the following: (1.1) Based on the patch data, locate the function node involved in the change from the project code property graph, and find the function call information and / or variable declaration information of the modified node by traversing the function node.

[0034] In some embodiments, the specific process is as follows: (1.1.1) First, parse the patch data and extract the file names before and after the modification, the starting line number, the ending line number, and the specific code changes from the patch data.

[0035] (1.1.2) Then, the Joern tool is used to perform static analysis on the project code to generate a project code property graph (CPG). In the CPG, the corresponding function nodes are located based on the start and end line numbers of the modification.

[0036] (1.1.3) Then, the depth-first traversal algorithm is used to explore the abstract syntax tree (AST) of the function node and filter out the modification nodes whose line numbers are within the modification range.

[0037] (1.1.4) Then extract the function call information and / or variable declaration information of the modified node: If the modified node is an identifier node, its fully qualified type name (typeFullName) is extracted. If the modified node is a non-basic type (such as a custom class or struct), the type declaration is further queried to extract the complete definition. If the modified node is a function call node (CALL type), the definition node of the called function is queried to extract the function signature, parameter list, and return type.

[0038] (1.1.5) Finally, all modified node information is stored in structured JSON format for easy subsequent intelligent proxy query.

[0039] (1.2) Analyze the patch data using an intelligent agent, identify the modified nodes, and call the function call information and / or variable declaration information of the modified nodes as needed to determine the correlation between the modified nodes and the vulnerability fix. If the correlation is greater than the relevant threshold, the modified node is identified as a vulnerability fix code change.

[0040] Specifically, the intelligent agent is a large language model with tool invocation capabilities. By designing prompt templates that include role definitions, task descriptions, and output format constraints, the large language model can identify modified nodes from the patch data and determine whether it can determine the relevance of the modified node to the vulnerability fix based on the modified node. If not, it will autonomously decide to invoke tools to obtain the function call information and / or variable declaration information of the modified node to assist in determining the relevance of the modified node to the vulnerability fix.

[0041] In some embodiments, the intelligent agent specifically refers to: Large-scale language models such as QwQ-32B are deployed as the core of the intelligent agent. The design of the prompt template includes: system prompts (defining expert roles, providing tool call interfaces, specifying task flows and JSON output format) and user prompts (including submission hash, filename, submission message, and specific code change blocks). During analysis, if the intelligent agent determines that supplementary function call information and / or variable declaration information is needed, it automatically calls the corresponding tools (get_identifiers_info or get_callee_info) to obtain them, and then performs code modification and relevance assessments to vulnerability fixes.

[0042] In some embodiments, through the method disclosed in this application, the intelligent agent can effectively identify changes in comments and test files in the embodiments as code changes unrelated to vulnerability fixation, and retain only code changes related to vulnerability fixation as vulnerability fixation code changes.

[0043] (2) Based on the teacher-student semi-supervised learning framework, the unlabeled vulnerability repair code changes are input into the teacher model to form pseudo-labeled sample points. The MixUp technique is used to interpolate the labeled data points in the vulnerability repair code changes and the pseudo-labeled sample points to obtain virtual sample points. This includes the following sub-steps: In this embodiment, most of the vulnerability patching code changes are untagged vulnerability patching code changes, while a small portion are tagged vulnerability patching code changes. The tagged vulnerability patching code changes and their corresponding vulnerability tags constitute tagged data points.

[0044] In some embodiments, Common Weakness Enumeration (CWE) is used as the classification standard for vulnerability labels. This standard can systematically describe the characteristics of various software vulnerabilities, helping downstream users to quickly understand the root causes, potential impacts, and remediation strategies of vulnerabilities.

[0045] (2.1) Input the changes to the unlabeled vulnerability repair code into the teacher model to obtain the corresponding vulnerability label. The changes to the unlabeled vulnerability repair code and the corresponding vulnerability label together constitute the pseudo-label sample point.

[0046] In some embodiments, a teacher model is used. Code changes to fix the unlabeled vulnerability Generate prediction probability distribution Modified from the code that fixes the unlabeled vulnerability. Generate prediction probability distribution Constituting the first One pseudo-label sample point The set of pseudo-label sample points consists of all the pseudo-label sample points. .

[0047] (2.2) For all labeled data points Upsampling is performed to obtain a set of labeled data points, and the size of the set of labeled data points is equal to the size of the set of pseudo-labeled sample points.

[0048] The number of labeled data points is much smaller than the number of pseudo-labeled sample points. Therefore, in some embodiments, to solve the problem of quantity mismatch, all labeled data points are first upsampled, thereby expanding the number of labeled data points so that the size of the set of labeled data points is equal to the size of the set of pseudo-labeled sample points.

[0049] Then, the labeled data points in the set of labeled data points are randomly shuffled to obtain the set of labeled data points. .

[0050] (2.3) The MixUp technique is used to interpolate the vulnerability repair code changes and vulnerability labels in the labeled data points and pseudo-labeled sample points respectively, and the resulting vulnerability repair code changes and vulnerability labels constitute virtual sample points.

[0051] MixUp is a data augmentation technique. Its core idea is to randomly select two pairs of sample points from the training set and perform linear interpolation on the input and label of the sample points according to weights to generate virtual training samples.

[0052] In some embodiments, there is a set of tagged data points. The set of pseudo-label sample points ,from Distributed sampling And perform the following operations to ensure that the virtual sample points are biased towards the labeled data points: ; Therefore, there are sampling parameters for labeled data points. The sampling parameters of the pseudo-label sample points are always greater than or equal to those of the pseudo-label sample points. .

[0053] Then, interpolate the labeled data points and pseudo-labeled sample points using the following formula: ; ; in, For the first Vulnerability fix code changes in tagged data points For the first Vulnerability tags in tagged data points; For the first Vulnerability fix code changes in individual pseudo-label sample points For the first Vulnerability labels in a sample of pseudo-labeled points; The sampling parameters for the label data points, obey distributed, These are the sampling parameters for pseudo-label sample points; all vulnerability labels use one-hot encoding.

[0054] The final interpolation value is obtained at the [number]. The virtual sample points .

[0055] (3) Model training methods (3.1) The mean squared error of the student model on the virtual sample points is used as the reinforcement learning reward signal, and the prediction confidence of the student model on the change of the unlabeled vulnerability repair code is used as the weight of the reinforcement learning reward signal to construct a confidence-weighted reinforcement learning loss; like Figure 4 As shown, in some embodiments, the vulnerability fix code in the virtual sample points is modified. Input student model In the process of calculating the predicted value Vulnerability labels in virtual sample points The mean squared error is negatively summed to obtain the reinforcement learning loss (reward signal). : ; in, The total number of sample points. Virtual sample points The set form.

[0056] This reward signal is used to evaluate the quality of the pseudo-labels generated by the current teacher model; the higher the quality, the larger the reward. When calculating the reward, gradient propagation is stopped to ensure that this signal is used only for evaluation and not directly for updating the teacher model.

[0057] For each unlabeled vulnerability fix code change Calculate the student model Predicted distribution With uniform distribution The KL divergence is used to measure the prediction confidence of the student model, and then this prediction confidence is used as a weight to construct a confidence-weighted reinforcement learning loss: ; in, The reinforcement learning loss is weighted by confidence level. For student models Reinforcement learning reward signals, Represents the set of virtual sample points. For student models For the first Code changes to fix an unlabeled vulnerability. Predicted vulnerability label distribution and uniform distribution Between Divergence is used to measure the confidence of a student model's predictions. This represents a collection of code changes used to fix unlabeled vulnerabilities. The expected data in the data.

[0058] (3.3) Monitoring losses In some embodiments, the student model Label category-weighted cross-entropy loss for labeled data points As a result of monitoring losses: ; in, Code changes to fix vulnerabilities in student model predictions of labeled data points belong The probability of a vulnerability tag. For the tag One-hot encoding tags for vulnerability types, For the tag Weight of vulnerability class This represents the total number of vulnerability tag categories; For a set of labeled data points The expected data in the data.

[0059] (3.2) Consistency loss In some embodiments, KL divergence is used to measure the teacher model. With student model Code changes to fix the unlabeled vulnerability The difference in the predicted distribution is used as the consistency loss: ; in, Vulnerability labels predicted by the teacher model Vulnerability labels predicted by student models Between Divergence. This represents a collection of code changes used to fix unlabeled vulnerabilities. The expected data in the data.

[0060] (3.4) The student model is iteratively trained by integrating the confidence-weighted reinforcement learning loss, consistency loss and supervision loss. After each round of training, the teacher model is iteratively updated based on the student model until the iteration termination target is reached. At this time, the student model is the completed vulnerability classification model.

[0061] Final total loss for: ; in and Let be the weighting coefficient, satisfying .

[0062] In some embodiments, and All values ​​were set to 0.1. Gradient descent was performed using the AdamW optimizer with a learning rate of 2e-5, and the student model was trained using a cosine annealing scheduling strategy.

[0063] After each round of training, the teacher model is iteratively updated based on the student model. Specifically, the teacher model is iteratively updated based on the student model using an exponential moving average algorithm. ; in, In some embodiments, to update parameters, Set to 0.999. This smooth update strategy avoids parameter mutations caused by hard updates, maintaining the stability of the reward function evaluation.

[0064] (3.5) Until the iteration termination target is reached, the student model is the completed vulnerability classification model.

[0065] Through iterative training using the three steps described above, the model can fully utilize unlabeled data, automatically optimize pseudo-label quality, and effectively remove patch noise, ultimately achieving performance superior to existing technologies in the submission-level vulnerability classification task. The performance metrics of the vulnerability classification method in this application and existing vulnerability classification methods are validated on the TREEVUL-C / C++ dataset; specific data are shown in Table 1.

[0066] Table 1. Performance metrics of the TREEVUL-C / C++ dataset;

[0067] The performance metrics of the vulnerability classification method in this application and existing vulnerability classification methods were verified on the MegaVul-C / C++ dataset. Specific data are shown in Table 2.

[0068] Table 2. Performance metrics of the MegaVul-C / C++ dataset;

[0069] As can be seen, the vulnerability classification method in this application improves the performance of existing vulnerability classification methods by 9%-16% on metrics such as MCC, Kappa, and F1.

[0070] In the above verification data: In this application, {sent} refers to the MixUp interpolation performed at the sentence-level (sentence vector level) in the method of this application.

[0071] In this application, {word} refers to the method in this application, in Word... MixUp interpolation is performed at the level (lexical vector level).

[0072] MCC (Matthews Correlation Coefficient) takes into account the four fundamental quantities of the confusion matrix and is a correlation coefficient that can stably evaluate binary classification performance even when the classes are highly imbalanced.

[0073] Kappa (Cohen's Kappa coefficient) measures the consistency between the classifier's predictions and the true labels, eliminating accidental consistency caused by random guessing.

[0074] Weighted F1 is a weighted average of the F1 scores for each category based on the number of actual samples in each category, focusing on reflecting the performance of the overall sample.

[0075] Macro F1 (Macro-average F1) directly calculates the arithmetic mean of F1 for all classes, treating each class equally regardless of its sample size.

[0076] Micro F1 (micro average F1) first summarizes the TP, FP, and FN of all classes and then calculates the global F1, which is equivalent to a sample-weighted average and is biased towards the performance of the majority class.

[0077] CodeBERT (Bidirectional Code Encoder Representation) is a bimodal pre-trained model based on Transformer. It learns a general semantic representation of code by modeling a hybrid mask language of natural language and programming language, which can be used for downstream tasks such as code search and generation.

[0078] TREEVUL (Tree Structure Vulnerability Type Prediction Network) formalizes vulnerability type prediction as a hierarchical multi-label classification problem. It uses CWE tree structure information for modeling and hierarchical prediction to achieve vulnerability type path prediction.

[0079] Teacher-Student The student model (or teacher model) generates soft labels for unlabeled data as a supervision signal, and the student model learns this signal and backpropagates to update it. This is usually used for model compression or semi-supervised learning.

[0080] Mean-Teacher model, in Teacher Based on the Student model, the teacher network parameters are not directly copied by the student, but are updated through exponential moving averages to provide the student with a more stable and smoother prediction target.

[0081] FixMatch combines consistency regularization and pseudo-labels to generate high-confidence pseudo-labels only for weakly augmented samples, and forces the model output to be consistent with the pseudo-labels for strongly augmented samples, thus achieving efficient semi-supervised learning.

[0082] HINT (a semi-supervised vulnerability detection method) combines the training loss of the teacher model with the removal of low-quality pseudo-label data based on retrieval. It utilizes the symmetric cross-entropy loss function and model prediction consistency regularization for noise-resistant training, thereby improving the pre-trained code model using pseudo-label data.

[0083] Zero-Shot Learning refers to the ability of a large language model to infer and predict new categories or tasks directly using only the natural language description of the task (without providing any examples).

[0084] The Predefined CWE List refers to a set of common software security vulnerability type numbers (such as CWE-89, CWE-79) from a pre-organized general vulnerability enumeration standard, used for static analysis or vulnerability classification of code or system.

[0085] One-Shot Learning refers to a large language model's ability to understand task patterns and generalize them to new test samples after seeing only one input-output example.

[0086] Chain of Thought Prompt refers to adding prompts such as "Let's think step by step" to encourage large language models to output intermediate reasoning steps, thereby improving the accuracy of complex logical reasoning tasks.

[0087] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0088] Based on the methods in the above embodiments, this application provides an electronic device, such as... Figure 5 As shown, the electronic device may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions stored in the memory to execute the methods described in the above embodiments.

[0089] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0090] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0091] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0092] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0093] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0094] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0095] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0096] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A semi-supervised submission-level vulnerability classification method based on reinforcement learning enhancement, characterized in that, include: Identify vulnerability fix code changes from patch data to be categorized; The vulnerability fix code changes are input into the trained vulnerability classification model to obtain the vulnerability label corresponding to the vulnerability fix code changes; The vulnerability classification model is trained through the following steps: Identify vulnerability fix code changes from the patch data submitted for vulnerability fixes; Based on a semi-supervised learning framework, the unlabeled vulnerability repair code changes in the vulnerability repair code changes are input into the teacher model to generate pseudo-labeled sample points. The MixUp technology is used to interpolate the labeled data points in the vulnerability repair code changes and the pseudo-labeled sample points to obtain virtual sample points. The mean squared error of the student model on the virtual sample points is used as the reinforcement learning reward signal, and the prediction confidence of the student model for the changes in the unlabeled vulnerability repair code is used as the weight of the reinforcement learning reward signal to construct a confidence-weighted reinforcement learning loss. The student model is iteratively trained by fusing the confidence-weighted reinforcement learning loss, consistency loss and supervision loss. After each round of training, the teacher model is iteratively updated based on the student model until the iteration termination target is reached. At this time, the student model is the vulnerability classification model that has completed training.

2. The semi-supervised commit-level vulnerability classification method of claim 1, wherein, The vulnerability fix code changes were identified from the patch data, specifically: Based on the patch data, the function nodes involved in the changes are located from the project code property graph, and the function call information and / or variable declaration information of the modified nodes are found by traversing within the function nodes. The patch data is analyzed using an intelligent agent to identify modified nodes. The function call information and / or variable declaration information of the modified nodes are called as needed to determine the relevance between the modified nodes and the vulnerability fix. If the relevance is greater than the relevant threshold, the modified node is identified as a vulnerability fix code change.

3. The semi-supervised commit-level bug classification method of claim 2, wherein, The intelligent agent is specifically a large language model with tool invocation capabilities. By designing prompt templates that include role definitions, task descriptions, and output format constraints, the large language model can identify modified nodes from the patch data and determine whether it can determine the relevance of the modified node to the vulnerability fix based on the modified node. If not, it will autonomously decide to invoke tools to obtain the function call information and / or variable declaration information of the modified node to assist in determining the relevance of the modified node to the vulnerability fix.

4. The semi-supervised commit-level bug classification method of claim 1, wherein, The unlabeled vulnerability repair code changes in the vulnerability repair code changes are input into the teacher model to generate pseudo-labeled sample points. Specifically, the unlabeled vulnerability repair code changes are input into the teacher model to obtain the corresponding vulnerability labels. The unlabeled vulnerability repair code changes and the corresponding vulnerability labels together constitute the pseudo-labeled sample points.

5. The semi-supervised commit-level bug classification method of claim 1, wherein, The MixUp technique is used to interpolate the labeled data points in the vulnerability fix code changes and the pseudo-label sample points to obtain virtual sample points. Specifically: Upsample all labeled data points to obtain a set of labeled data points, and make the size of the set of labeled data points equal to the size of the set of pseudo-labeled sample points; The MixUp technique is used to interpolate the vulnerability remediation code changes and vulnerability labels in the labeled data points and pseudo-labeled sample points, and the interpolated vulnerability remediation code changes and vulnerability labels constitute virtual sample points. In the MixUp technology, the sampling parameters of the labeled data points are always greater than the sampling parameters of the pseudo-labeled sample points; and the vulnerability labels are encoded using one-hot encoding.

6. The semi-supervised submission level vulnerability classification method according to claim 1 or 5, characterized in that, No. The virtual sample points are specifically: ,in For the first Vulnerability fix code changes in a virtual sample point For the first The vulnerability labels in a set of virtual sample points satisfy: ； ； ； in, For the first Vulnerability fix code changes in tagged data points For the first Vulnerability tags in tagged data points; For the first Vulnerability fix code changes in individual pseudo-label sample points For the first Vulnerability labels in a sample of pseudo-labeled points; The sampling parameters for the label data points, obey distributed, These are the sampling parameters for pseudo-label sample points; all vulnerability labels use one-hot encoding.

7. The semi-supervised submission level vulnerability classification method according to claim 1, characterized in that, The mean squared error of the student model on the virtual sample points is used as the reinforcement learning reward signal. Specifically, the vulnerability repair code changes in the virtual sample points are input into the student model to obtain the predicted vulnerability label. Let the mean squared error between the predicted vulnerability label and the corresponding vulnerability label in the virtual sample points be the single-point mean squared error. The reinforcement learning reward signal is obtained by accumulating all the single-point mean squared errors and taking the negative value.

8. The semi-supervised submission level vulnerability classification method according to claim 1, characterized in that, The confidence-weighted reinforcement learning loss is specifically as follows: ； in, The reinforcement learning loss is weighted by confidence level. For student models Reinforcement learning reward signals, Represents the set of virtual sample points. For student models For the first Code changes to fix an unlabeled vulnerability. Predicted vulnerability label distribution and uniform distribution Between Divergence, used to measure the student model The prediction confidence level; This represents a collection of code changes to fix unlabeled vulnerabilities. The expected data in the data.

9. The semi-supervised submission level vulnerability classification method according to claim 1, characterized in that, The teacher model is iteratively updated based on the student model, specifically by using an exponential moving average algorithm to iteratively update the teacher model based on the student model.

Images