Robustness verification method and system based on source code pre-training model and storage medium
By performing token sequence slicing, recombination, and edit distance calculation on a large pre-trained model, the robustness and accuracy of the model in source code scenarios are improved, solving the problem of insufficient robustness of existing methods in source code scenarios and achieving effective detection of adversarial perturbations.
Patent Information
- Application Number
- CN202511196522.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-26
- Publication Date
- 2025-11-18
AI Technical Summary
Existing stochastic smoothing methods struggle to finely process different features when dealing with source code scenarios that are structurally complex and highly heterogeneous, resulting in insufficient robustness of large pre-trained language models in adversarial environments.
By acquiring a large pre-trained model of the target, extracting code data features and performing token sequence slicing, sampling and recombining random token sequences, designing a classification loss function for fine-tuning, calculating classification confidence and robustness radius, and constructing a new token sequence based on edit distance to improve the robustness of the model.
It effectively reduces the impact of complex perturbations on code syntax and logic, improves the robustness and accuracy of the model in the source code environment, provides mathematically provable robustness boundaries, and enhances the model's ability to resist perturbations.
Smart Images
Figure CN120974346A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of information security, and particularly relates to a robustness verification method and system based on a source code pre-training model and a storage medium. BACKGROUND
[0002] In recent years, deep learning (DL) technology has been widely applied to security-related tasks such as malware detection, code auditing, and vulnerability mining. With the breakthrough of large pre-trained language models (LLMs) in natural language processing, their application in source code analysis tasks has also increased, such as pre-trained models (such as CodeBERT, GraphCodeBERT, CodeGPT, CodeT5, etc.) for deep semantic recognition of source code. Large pre-trained language models can learn rich semantic information from large-scale source code and exhibit good performance in multiple downstream tasks. However, deep learning models are generally highly sensitive to small perturbations, and even minor modifications at the source code level, such as replacing variable names or adjusting code structure without changing semantics, can cause significant changes in model output, which can be exploited by attackers to bypass detection systems and cause security risks.
[0003] To improve the robustness of large pre-trained language models in adversarial environments, a randomized smoothing method that can provide theoretical robustness guarantees has been proposed. This method adds random noise to the input and makes predictions based on the majority vote of the model under multiple noise perturbations, thereby obtaining mathematical guarantees of the invariance of the model output under a specific threat model. However, this method is mainly applied to text classification, graph neural networks, and other specific structures, and cannot effectively handle complex and highly heterogeneous source code scenarios.
[0004] In the source code scenario, input data has discrete and diverse structural forms, such as abstract syntax trees, control flow graphs, call graphs, and source code sequences. When these heterogeneous data are subjected to adversarial modifications, attackers can design perturbations for different feature types, such as deleting function call nodes. Such complex perturbations are difficult to model with a unified noise model, and existing randomized smoothing methods cannot handle each type of feature finely, resulting in ineffective authentication range and unsatisfactory defense effect.
[0005] To solve the problem that complex perturbations in the source code scenario make it difficult to model uniformly, leading to large pre-trained language models being unable to handle each type of feature finely and thus being unable to guarantee robustness, a robustness verification method and system based on a source code pre-training model and a storage medium are proposed. SUMMARY
[0006] The application provides a robustness verification method and system based on a source code pre-training model and a storage medium to at least solve the problem that complex perturbations in a source code scene make it difficult for an existing random smoothing method to be uniformly modeled, so that a large pre-training language model cannot be finely processed for various features, thereby making it difficult to ensure robustness.
[0007] According to an embodiment of the application, a robustness verification method based on a source code pre-training model is provided, comprising the steps of:
[0008] obtaining a target large pre-training model and extracting code data features to obtain a token sequence;
[0009] adopting an N-gram model to slice the token sequence;
[0010] sampling the token sequence slices and recombining them to obtain a random token sequence;
[0011] designing a classification loss function and fine-tuning the large pre-training model;
[0012] taking the random token sequence as the input of the large pre-training model to obtain the classification label predicted by the model;
[0013] calculating the confidence of each classification according to the frequency of the classification label predicted by the model multiple times, and obtaining the output prediction class of the large pre-training model according to the confidence;
[0014] constructing a new token sequence based on the edit distance and calculating the sequence similarity;
[0015] calculating the robust radius according to the limit of the prediction label confidence value and the bisection method.
[0016] In a preferred embodiment, the obtaining of the target large pre-training model and the extraction of the code data features to obtain the token sequence comprises:
[0017] obtaining the specified source code large pre-training model parameters from an open source website or its mirror website to obtain the target large pre-training model;
[0018] selecting code-related detection as a downstream task; the code-related detection includes any one or a combination of multiple code duplication detection, code author classification, and code vulnerability detection;
[0019] extracting sequence features from the input code according to the selected large pre-training model;
[0020] adopting a tokenizer of a language model to process the sequence features to obtain a token sequence.
[0021] In a preferred embodiment, the slicing of the token sequence by the N-gram model comprises the steps of:
[0022] cutting each token sequence into token slices of length N;
[0023] forming a pair of the slice element and an index, where the first element is the slice and the second element is the position index of the first element in the sequence.
[0024] In a preferred embodiment, the sampling, slicing and reordering of the token sequence to obtain a random token sequence comprises the steps of:
[0025] randomly sampling K token slices from the token slice set according to a uniform distribution;
[0026] continuously placing the K token slices in order according to the index in ascending or descending order;
[0027] reordering the placed token slices to obtain a random token sequence.
[0028] In a preferred embodiment, the designing of the classification loss function and the fine-tuning of the large pre-trained model comprises the steps of:
[0029] designing a cross-entropy loss as the classification loss function;
[0030] fine-tuning the large pre-trained model using backpropagation to update the model parameters and an optimization method.
[0031] In a preferred embodiment, the calculating of the confidence of each class from the frequency of the class in the classification labels obtained from multiple model predictions and obtaining the output prediction class of the large pre-trained model comprises the steps of:
[0032] repeating T times the step of inputting the random token sequence into the large pre-trained model to obtain a model-predicted classification label, and obtaining T model-predicted classification labels; T is a preset number of training times;
[0033] counting the frequency of each class in the T model-predicted classification labels;
[0034] calculating the confidence of each class according to the frequency of the class and the number of training times;
[0035] selecting the class with the highest confidence as the output prediction class of the large pre-trained model.
[0036] In a preferred embodiment, the constructing of a new token sequence based on the edit distance and calculating of the similarity of the sequences comprises the steps of:
[0037] constructing a new token sequence by performing the minimum number of element replacement, addition, and deletion operations on the token sequence and calculating the edit distance;
[0038] calculating the similarity of two token sequences according to the edit distance.
[0039] In a preferred embodiment, the robust radius is calculated according to the threshold of the predicted label confidence value and dichotomy, including:
[0040] The lower bound of the predicted label with the highest confidence value of the classification according to the pre-set confidence is recorded as the lower bound of the predicted label confidence value.
[0041] The upper bound of the predicted label ranked second in the confidence value of the classification according to the pre-set confidence is recorded as the upper bound of the second predicted label confidence value.
[0042] The difference inequality is constructed according to the relationship between the difference between the lower bound of the predicted label confidence value and the upper bound of the second predicted label confidence value and the sequence similarity.
[0043] The robust radius is calculated according to the constructed difference inequality and dichotomy.
[0044] According to another embodiment of the present application, a computer readable storage medium is provided, which stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the above method.
[0045] According to another embodiment of the present application, a robustness verification system based on a source code pre-trained model is provided, including:
[0046] A processor;
[0047] A memory;
[0048] And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the signal processing unit, and the programs cause the computer to execute the above method.
[0049] The robustness verification method, system and storage medium based on the source code pre-trained model of the present application have the following advantages:
[0050] (1) According to the model, the code data token sequence is extracted, the features are transformed based on the N-gram model, the token slices obtained by transformation are randomly sampled, and the sampled slices are recombined into new token sequences. Under the premise of preserving the code function, diversified perturbation samples are generated. Compared with traditional adversarial sample generation, it may damage the code syntax or semantics or the perturbation mode is single, which can effectively reduce the influence of complex perturbation on code syntax and logic, and facilitate to improve the robustness of the pre-trained model in the source code environment.
[0051] (2) Designing a classification loss function based on a random token sequence and fine-tuning a large pre-trained model, compared with traditional static fine-tuning methods, can effectively reduce the subsequent adjustment difficulty of large pre-trained models, improve the model training speed and improve the model training accuracy.
[0052] (3) The present application takes the random token sequence as the input of the large pre-training model to obtain the classification label predicted by the model, and calculates the confidence of each classification according to the frequency of the classification label obtained by multiple model predictions, and obtains the output prediction category of the large pre-training model, compared with the traditional random smoothing method, which can effectively improve the adaptability of the model to uncertain disturbance, thereby improving the accuracy and precision of the pre-training model.
[0053] (4) Based on the least element replacement, addition and deletion operation to construct a new token sequence and calculate the edit distance, the similarity of two token sequences is calculated according to the edit distance, and the robust radius is calculated, which provides a mathematically provable robustness boundary for model adversarial disturbance, compared with the traditional random smoothing method, which not only considers the sequence relationship between token sequences, thereby using fewer token elements, but also increases the robust radius while achieving higher detection effect, and effectively ensures that the code transformation within a certain edit distance will not cause the prediction result to deviate, thereby enhancing the accuracy and robustness of the large pre-training model in the code environment. BRIEF DESCRIPTION OF DRAWINGS
[0054] Figure 1 It is a robustness verification method flow chart of the source code pre-training model of the embodiment of the present application.
[0055] Figure 2 It is a flow chart of step S01 of the robustness verification method of the source code pre-training model of the embodiment of the present application.
[0056] Figure 3 It is a flow chart of step S02 of the robustness verification method of the source code pre-training model of the embodiment of the present application.
[0057] Figure 4 It is a flow chart of step S03 of the robustness verification method of the source code pre-training model of the embodiment of the present application.
[0058] Figure 5 It is a flow chart of sub-step S04 of the robustness verification method of the source code pre-training model of the embodiment of the present application.
[0059] Figure 6 It is a flow chart of sub-step S06 of the robustness verification method of the source code pre-training model of the embodiment of the present application.
[0060] Figure 7 It is a flow chart of sub-step S07 of the robustness verification method of the source code pre-training model of the embodiment of the present application.
[0061] Figure 8 It is a flow chart of sub-step S08 of the robustness verification method of the source code pre-training model of the embodiment of the present application.
[0062] Figure 9 is a structural schematic diagram of a robustness verification system based on a source code pre-training model according to an embodiment of the present application. DETAILED DESCRIPTION
[0063] The present application will be described in detail below with specific embodiments. The following embodiments will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These are within the scope of the present application.
[0064] According to one embodiment of the present application, a robustness verification method based on a source code pre-training model is provided, and a flowchart is shown as Figure 1 , which includes:
[0065] Step S01, obtaining a target large pre-training model and extracting code data features to obtain a token sequence;
[0066] Step S02, slicing the token sequence using an N-gram model;
[0067] Step S03, sampling the token sequence slices and recombining to obtain a random token sequence;
[0068] Step S04, designing a classification loss function and fine-tuning the large pre-training model;
[0069] Step S05, taking the random token sequence as input of the large pre-training model to obtain the classification label predicted by the model;
[0070] Step S06, calculating the confidence of each classification according to the frequency of the classification labels obtained by multiple model predictions, and obtaining the output prediction class of the large pre-training model;
[0071] Step S07, constructing a new token sequence based on the edit distance and calculating the sequence similarity;
[0072] Step S08, calculating the robust radius according to the limit of the prediction label confidence value and the bisection method.
[0073] In a preferred embodiment, the step S01, obtaining a target large pre-training model and extracting code data features to obtain a token sequence, a flowchart is shown as Figure 2 , which includes:
[0074] Step S011, obtaining specified source code large pre-training model parameters from an open source website or its mirror website to obtain a target large pre-training model;
[0075] Step S012, selecting code-related detection as the downstream task; the code-related detection includes any one or a combination of multiple items of code duplication detection, code author classification, and code vulnerability detection;
[0076] Step S013, extracting sequence features from the input code according to the selected large pre-training model;
[0077] Step S014, processing the sequence features by using a tokenizer of a language model to obtain a token sequence.
[0078] In this embodiment, the CodeBERT source code large pre-training model parameters are obtained from an open source website HuggingFace or a mirror website thereof, JAVA code plagiarism detection is selected as the downstream task, sequence features are extracted from the input code according to the selected model, and a token-level sequence is obtained by processing the sequence features by using a tokenizer of a large language model.
[0079] In a preferred embodiment, the step S02 includes slicing processing of the token sequence by using an N-gram model, and a flowchart is shown in Figure 3 , which includes the following steps:
[0080] Step S021, cutting each token sequence into a token slice with a length of N;
[0081] Step S022, constructing a two-tuple by using a slice element and an index, where the first element is the slice, and the second element is the position index of the first element of the slice in the sequence.
[0082] In this embodiment, the token sequence features of the input code are denoted as , where represents a token element, the token sequence is sliced by using an N-gram model, that is, each token sequence X is sliced into a token slice with a length of N , , …, ; a two-tuple is constructed by using a slice element and an index, where the first element is the slice, and the second element is the index of the first element of the slice in the sequence X, for example , and the corresponding index is 2.
[0083] In a preferred embodiment, the step S03 includes sampling and recombining the token sequence slices to obtain a random token sequence, and a flowchart is shown in Figure 4 , which includes the following steps:
[0084] Step S031, randomly sampling K token slices from the token slice set according to a uniform distribution;
[0085] Step S032, placing K token slices in order from small to large or from large to small according to the subscripts;
[0086] Step S033, recombining the re-placed token slices to obtain a random token sequence.
[0087] In this embodiment, the corresponding training set is collected according to the task CodeBERT supports JAVA source code analysis tasks, uses an open source dataset (website: https: / / arxiv.org / pdf / 2002.08653) containing 90102 training JAVA samples, and uses a Transformer tokenizer to process the code to obtain a token sequence The token sequence of each code sample in the training set is processed according to the operation in step S02 to obtain N-gram token slices and corresponding slice subscripts. For example, set N=2, randomly sample K=20 (preset) token slices from the token set according to uniform distribution and place them in order from small to large according to the subscripts to obtain a recombined token sequence, denoted as .
[0088] In a preferred embodiment, the step S04, designing a classification loss function and fine-tuning a large pre-training model, the flow chart is as shown in Figure 5 , including the steps of:
[0089] Step S041, designing cross-entropy loss as a classification loss function;
[0090] Step S042, fine-tuning the large pre-training model using backpropagation to fine-tune the model parameters and optimization method.
[0091] In this embodiment, cross-entropy loss is designed as a classification loss function, and backpropagation is used to fine-tune the model parameters to reduce the classification loss. For example, set the AdamW optimization method to fine-tune CodeBERT, with a batch size of 16 and a learning rate of 0.00001. In each mini-batch training, the random token sequence obtained by the method described in step S03 is used. In the training token dataset Repeat step S04 for 5 rounds to end the training and obtain the fine-tuned large pre-training model.
[0092] In step S05, the recombined random token sequence is used as input to the fine-tuned large pre-training model to obtain the model's predicted label.
[0093] In a preferred embodiment, the step S06, according to the frequency of each classification in the classification labels obtained by multiple model predictions, the confidence of the classification is calculated and the output prediction class of the large pre-training model is obtained, the flow chart is as shown inFigure 6 as shown, comprising steps of:
[0094] Step S061, repeat step S05 for T times to obtain T times of model prediction classification labels;
[0095] Step S062, count the frequency of each classification in T times of model prediction classification labels;
[0096] Step S063, calculate the confidence of each classification according to the frequency of each classification and the number of training times;
[0097] Step S064, select the class with the highest confidence as the output prediction class of the large pre-trained model.
[0098] In this embodiment, step S05 is repeated for T times, and T is a positive integer and needs to be large enough, for example, greater than 10000. The frequency of each classification in T times of prediction results is counted , wherein represents the number of categories, and satisfies .
[0099] The confidence of each classification is calculated according to the frequency of each classification and the number of training times, as shown in formula (1).
[0100] (1)
[0101] The class with the highest confidence is selected as the prediction class of the model for the sample, and the confidence of the confidence value is evaluated by using the binomial distribution statistical hypothesis test. If the confidence is greater than 0.95, the prediction class is output, that is, the output prediction class of the large pre-trained model; otherwise, the prediction is cancelled.
[0102] In a preferred embodiment, the step S07, a new token sequence based on edit distance is constructed and the similarity of the sequence is calculated, and the flow chart is as shown in Figure 7 , comprising steps of:
[0103] Step S071, the token sequence is subjected to the minimum element replacement, addition and deletion operation to construct a new token sequence and calculate the edit distance;
[0104] Step S072, the similarity of the two token sequences is calculated according to the edit distance.
[0105] In this embodiment, the edit distance is defined to transform the token sequence into another token sequence by the minimum element replacement, addition and deletion operation ; the edit distance is calculated as shown in formula (2).
[0106] (2)
[0107] , wherein denotes the sequence left by removing the first element in the token sequence, denotes keeping only the first element in the input token sequence, and minimum denotes taking the minimum value.
[0108] The similarity between two token sequences is calculated according to the edit distance, as shown in equation (3).
[0109] (3)
[0110] where N is the length of the slice.
[0111] In a preferred embodiment, the step S08 of calculating the robust radius according to the bounds of the predicted label confidence value and the bisection method includes the following steps, as shown in the flow chart of Figure 8 .
[0112] Step S081: calculating the lower bound of the predicted label with the highest confidence value of classification according to the pre-set confidence, denoted as the lower bound of the predicted label confidence value;
[0113] Step S082: calculating the upper bound of the predicted label ranked second in the confidence value of classification according to the pre-set confidence, denoted as the upper bound of the second predicted label confidence value;
[0114] Step S083: constructing a difference inequality according to the relationship between the difference between the lower bound of the predicted label confidence value and the upper bound of the second predicted label confidence value and the sequence similarity;
[0115] Step S084: calculating the robust radius according to the constructed difference inequality and the bisection method.
[0116] In this embodiment, the upper and lower bounds of the confidence value of classification are calculated according to the method described in step S06 by empirical assumption with a confidence of 0.95 (pre-set confidence), the predicted label with the highest confidence value of classification is denoted as , the lower bound of which is denoted as , and the predicted label ranked second in the confidence value of classification is denoted as , the upper bound of which is denoted as .
[0117] The difference inequality is constructed according to the relationship between the difference between the lower bound of the predicted label confidence value and the upper bound of the second predicted label confidence value and the sequence similarity, as shown in equation (4).
[0118] (4)
[0119] where K=20 is the number of token slices, and p is obtained according to equation (3).
[0120] The robust radius is calculated according to the constructed difference inequality and the bisection method by searching the robust radius through the bisection method so that it satisfies The final robust radius value obtained is shown as formula (5).
[0121] (5)
[0122] This embodiment reports the effectiveness on the test set by standard indicators false negative rate (FNR), false positive rate (FPR), F1 score, accuracy (abbreviated as Acc, which is the proportion of correctly predicted examples in the test data set) and balanced accuracy (bAcc).
[0123] The test data set also comes from the above open source data set and does not coincide with the training data, and the test data set contains 4000 JAVA code files.
[0124] The comparative method is CodeBERT, that is, the original model and the fine-tuning, but without robustness verification capability; CodeBERT-RA, which adopts a random method (Random) based on binomial distribution to mask part of the token elements; CodeBERT-RD, which adopts a random deletion method based on binomial distribution to remove part of the token elements.
[0125] Table 1 includes the detection effect of the method of the present application on the JAVA test data set. The method of the present application achieves a detection accuracy of 95.33%, a false negative rate (misjudgment of plagiarism samples as non-plagiarism samples) of 6.6%, a false positive rate (misjudgment of non-plagiarism samples as plagiarism samples) of 2.75, and an F1 score of 95.23% when K=20, while the random smoothing method achieves a detection rate of 94.90% and an F1 score of 94.80%, which is lower than the effect of the method of the present application. At the same time, the random deletion method achieves a detection rate of 94.53% and an F1 score of 94.37%. This shows that the method of the present application is significantly better than the random masking and random deletion methods.
[0126]
[0127] Table 2 shows the model robustness radius obtained when the method of the present application is compared with the random masking method and the random deletion method.
[0128]
[0129] According to the average value and the median of the radius, it can be seen that the method of the present application achieves a higher average authentication radius (≥3.84) and median (≥4.5) than the two comparative methods. Therefore, the method of the present application considers the sequence relationship between token sequences and uses fewer token elements, and compared with the random masking and random deletion methods which randomly operate on token elements, it achieves higher detection effect while the obtained robust radius is also significantly higher than other methods.
[0130] According to another embodiment of the present application, there is provided a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to perform the method for verifying robustness of a pre-trained model based on source code according to any one of the above embodiments.
[0131] According to another embodiment of the present application, there is provided a system for verifying robustness of a pre-trained model based on source code, as shown in the structural schematic diagram Figure 9
[0132] a processor;
[0133] a memory;
[0134] and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the signal processing unit, and the programs cause the computer to perform the method for verifying robustness of a pre-trained model based on source code according to any one of the above embodiments.
[0135] The above method according to the present application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk or a magneto-optical disk, or be downloaded through a network originally stored in a remote recording medium or a non-transitory machine readable medium and stored in a local recording medium, so that the method described herein can be processed by such software using a general purpose computer, a special purpose processor, or programmable or special purpose hardware (such as AuIC or FPGA). It can be understood that the computer, processor, microprocessor controller or programmable hardware includes a storage component (for example, RAM, ROM, flash memory, etc.) that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, processor or hardware, the method described herein is implemented. In addition, when a general purpose computer accesses the code for implementing the processing shown herein, the execution of the code will convert the general purpose computer into a special purpose computer for executing the processing shown herein.
[0136] Of course, those skilled in the art should recognize that the above embodiments are only used to illustrate the present application, and are not intended to limit the present application, as long as the changes and modifications of the above embodiments fall within the scope of the present application.
Claims
1. A robustness verification method based on a source code pre-trained model, characterized in that, include: Obtain the target large pre-trained model and extract code data features to obtain the token sequence; The token sequence is sliced using an N-gram model; Slice the sampled token sequence and recombine it to obtain a random token sequence; Design a classification loss function and fine-tune it using a large pre-trained model; The random token sequence is used as input to a large pre-trained model to obtain the predicted classification labels. The confidence level of the classification is calculated based on the frequency of each category in the classification labels obtained from multiple model predictions, and the output predicted category of the large pre-trained model is obtained accordingly. Construct a new token sequence based on edit distance and calculate sequence similarity; The robust radius is calculated based on the bounds of the predicted label confidence value and the dichotomy method.
2. The robustness verification method based on a source code pre-trained model according to claim 1, characterized in that, The process of obtaining a target large-scale pre-trained model and extracting code data features to obtain a token sequence includes: Obtain the parameters of a large pre-trained model with specified source code from an open-source website or its mirror site to obtain the target large pre-trained model. Code-related detection is selected as a downstream task; the code-related detection includes any one or a combination of code copy detection, code author classification, and code vulnerability detection. Extract sequence features from the input code based on the selection of a large pre-trained model; The token sequence is obtained by processing the sequence features using a tokenizer based on a language model.
3. The robustness verification method based on a source code pre-trained model according to claim 1, characterized in that, The step of slicing the token sequence using an N-gram model includes the following steps: Cut each token sequence into token slices of length N; Form a tuple with the slice element and its index, where the first element is the slice and the second element is the index of the first element of the slice in the sequence.
4. The robustness verification method based on a source code pre-trained model according to claim 1, characterized in that, The process of slicing and recombining the sampled token sequence to obtain a random token sequence includes the following steps: K token slices are obtained by randomly sampling the token slice set according to a uniform distribution; K is the preset sampling value; Place K token slices consecutively in ascending or descending order of their indices; The repositioned token slices are reassembled to obtain a random token sequence.
5. The robustness verification method based on a source code pre-trained model according to claim 1, characterized in that, The design of the classification loss function and fine-tuning of the large pre-trained model includes the following steps: Design cross-entropy loss as a classification loss function; Backpropagation is used to fine-tune model parameters and optimization methods to fine-tune large pre-trained models.
6. The robustness verification method based on a source code pre-trained model according to claim 1, characterized in that, The step of calculating the confidence level of a classification based on the frequency of each category in the classification labels obtained from multiple model predictions, and using this confidence level to obtain the output predicted category of the large pre-trained model, includes the following steps: Repeat the steps T times: use the random token sequence as input to a large pre-trained model to obtain the model's predicted classification labels, and obtain the model's predicted classification labels T times. T is the preset number of training iterations; Statistical analysis of the frequency of each category in the T-time model prediction; The confidence level of a category is calculated based on the frequency of occurrence of each category and the number of training iterations. The category with the highest confidence level is selected as the output prediction category of the large pre-trained model.
7. The robustness verification method based on a source code pre-trained model according to claim 1, characterized in that, The construction of a new token sequence based on edit distance and the calculation of sequence similarity include the following steps: Construct a new token sequence by performing the fewest element replacement, addition, and deletion operations on the token sequence and calculate the edit distance; The similarity between two token sequences is calculated based on the edit distance.
8. The robustness verification method based on a source code pre-trained model according to claim 1, characterized in that, The calculation of the robust radius based on the bounds of the predicted label confidence value and the bisection method includes: The lower bound of the predicted label with the highest confidence score is calculated according to the preset confidence score and is denoted as the lower bound of the predicted label confidence score. The upper bound of the second-ranked predicted label confidence value, calculated according to the pre-set confidence level, is denoted as the upper bound of the second-ranked predicted label confidence value. Construct a difference inequality based on the relationship between the difference between the lower bound of the predicted label confidence value and the upper bound of the next predicted label confidence value and the sequence similarity; The robust radius is calculated based on the constructed difference inequality and the bisection method.
9. A computer-readable storage medium storing a computer program for electronic data interchange, wherein, The computer program causes the computer to perform the method as described in any one of claims 1-8.
10. A robustness verification system based on a source code pre-trained model, characterized in that, include: processor; Memory; as well as One or more programs, wherein the one or more programs are stored in memory and configured to be executed by the processor, the programs causing the computer to perform the method as described in any one of claims 1-8.