Code model robustness evaluation method and system based on variable importance analysis

Abstract

The invention provides a code model robustness evaluation method and system based on variable importance analysis. The method comprises the steps of obtaining a to-be-evaluated code model and a source code sample, and extracting variables from a real source code library to construct a first alternative variable set; calculating the importance score of each initial variable in the source code sample; screening alternative variables of the important variables in the first alternative variable set to construct a second alternative variable set, and applying length constraint and K-means clustering in the screening process to ensure semantic diversity; an improved beam search method is used for executing a variable replacement test, and the search space is expanded by combining iteration results before and after so as to avoid falling into local optimum. According to the method, the sensitivity of the code model to the variable naming change can be systematically and efficiently evaluated, the robustness weakness of the model can be accurately identified even under the limited query condition, and technical support is provided for improving the stability and safety of the code model.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and particularly to a method and system for evaluating the robustness of a code model based on variable importance analysis. Background Art

[0002] With the development of deep learning technology, various code models such as CodeBERT, GraphCodeBERT, and CodeT5 have been widely applied to tasks such as code understanding, classification, and generation. However, research shows that these code models are often very sensitive to minor perturbations in the input, especially lacking sufficient robustness to code transformations that preserve semantics but change in form, such as variable renaming. For example, by simply renaming some variables in the source code, it may lead to completely different model outputs, which seriously affects the reliability and security of code models in practical applications.

[0003] Currently, the main methods for evaluating the robustness of code models are as follows: 1. The method based on random selection (such as MHM), which randomly selects 1500 words from a fixed dictionary as alternative variables. However, this method may result in overly long variable names and limited diversity of alternative variables; 2. The method based on pre-trained model generation (such as ALERT), which generates 60 alternative variables for each original variable through a pre-trained masked model. However, these alternative variables often lack diversity; 3. The method based on historical attack data (such as RNNS), which uses historical attack data as search seeds to identify potential adversarial variable substitutions. However, each variable name is limited to 60 variable names.

[0004] The above methods usually adopt a greedy strategy or a combination of greedy and heuristic algorithms to search for adversarial samples. Although these algorithms are effective, they are prone to falling into local optimal solutions, resulting in excessive perturbations and low attack success rates under a limited query budget. In addition, the existing methods fail to fully utilize the diversity of the alternative variable set, do not consider the importance differences of variables on the model output, and also fail to effectively solve the problem of search space expansion.

[0005] Therefore, there is an urgent need for a method that can comprehensively evaluate the robustness of code models. This method should be able to identify the importance of variables, provide diverse alternative variable selections, and improve the search strategy to avoid local optimal solutions, so as to achieve efficient robustness evaluation under a limited query budget. Summary of the Invention

[0006] The present invention provides a method for evaluating the robustness of a code model based on variable importance analysis, which solves the technical problems in the prior art such as insufficient diversity of alternative variables, the search algorithm being prone to falling into local optimal solutions, and low query efficiency. The present invention significantly improves the efficiency and accuracy of evaluating the robustness of the code model by calculating variable importance, using the constrained K-means clustering algorithm to identify diverse alternative variables in the embedding space, and adopting an improved beam search algorithm to construct test samples.

[0007] To achieve the above-mentioned invention purpose, the technical solutions provided by the present invention include: A method for evaluating the robustness of a code model based on variable importance analysis, comprising the steps of: S1. Obtain the code model to be evaluated and the source code samples; extract variables from the real source code library to construct the first set of alternative variables; S2. Calculate the importance scores of each initial variable in the source code samples for the output result of the code model to be evaluated, and the importance scores are calculated based on the soft label method; S3. Screen out the alternative variables of the initial variables with importance scores greater than the first preset threshold in the first set of alternative variables to construct the second set of alternative variables; S4. Use the second set of alternative variables to iteratively perform variable substitution on the source code samples, generate and evaluate test samples in each iteration, and continuously optimize the test samples based on the prediction result differences between each round of test samples and the source code samples until the robustness evaluation is completed.

[0008] Preferably, the construction method of the second set of alternative variables includes: S301. In the first set of alternative variables, extract the variables whose variable lengths have a difference within the range of the second preset threshold from the variable lengths of the initial variables with importance scores greater than the first preset threshold to construct the third set of alternative variables; S302. Obtain the embedding vector representations of each variable in the third set of alternative variables, and apply the K-means clustering algorithm to cluster the variables with similar semantics in the third set of alternative variables into K candidate clusters; S303. Select at least one variable from each of the candidate clusters to form the second set of alternative variables.

[0009] Preferably, step S4 includes: S401. Initialize the test sample set and add the source code samples to the test sample set; S402. Traverse each sample in the test sample set. For the current initial variable being processed, replace the initial variable with the candidate alternative variables in the second set of alternative variables to generate new test samples and add them to the temporary sample set; S403. Test each sample in the temporary sample set using the code model to be evaluated, and calculate the difference value of the predicted probability distribution between each sample and the source code sample; S404. Combine the test sample set retained in the previous iteration with the currently generated temporary sample set, and select the top B samples with the largest difference values as the new test sample set; S405. Repeat steps S402 to S404 until the stop condition is met.

[0010] Preferably, the method for selecting the initial variables to be currently processed in step S402 includes: Extract the initial variables with importance scores greater than the first preset threshold, arrange them in descending order of importance scores, and sequentially select them in the sorting order in each iteration as the initial variables to be currently processed.

[0011] Preferably, the stop condition includes: Discover test samples that can change the prediction results of the code model to be evaluated; All initial variables with importance scores greater than the first preset threshold have been processed; The number of iterations reaches the third preset threshold.

[0012] Preferably, the method for calculating the importance scores of each initial variable in the source code sample for the output result of the code model to be evaluated in step S2 includes: replacing the initial variable to be calculated in all instances of the source code sample with a special marker to ensure the compilability of the source code sample and comprehensively calculate the importance of the initial variable to be calculated.

[0013] Preferably, when applying the K-means clustering algorithm to cluster the third set of alternative variables in step S302, the Euclidean distance is used as the similarity measurement standard between the embedding vectors of each variable.

[0014] The present invention also provides a code model robustness evaluation system based on variable importance analysis, and the system is used to implement the above-mentioned code model robustness evaluation method based on variable importance analysis.

[0015] Beneficial effects 1. By introducing a variable importance calculation method based on soft labels, it is possible to accurately identify the variables that have the greatest impact on the model output, effectively reduce the number of variables to be processed, and improve the evaluation efficiency. Specifically, the present invention quantifies the importance of variables by calculating the change in the model's predicted probability distribution before and after replacing the variables, ensuring that the most influential variables are processed first.

[0016] 2. By processing the alternative variable set through the constrained K-means clustering algorithm, effective grouping of semantically similar variables is achieved, significantly improving the diversity of alternative variables. The present invention first applies length constraints to screen alternative variables, then measures the similarity between variables through the Euclidean distance of embedding vectors, selects representative variables from each cluster to form a more diverse set of alternative variables, providing richer candidate variables for subsequent searches.

[0017] 3. An improved beam search algorithm is used to generate test samples, effectively avoiding the problem of falling into local optimal solutions. Different from traditional greedy algorithms, the beam search method of the present invention combines the samples retained in the previous iteration with the samples generated in the current iteration in each iteration and selects the top B samples with the largest difference value as the new test sample set, significantly expanding the search space and increasing the probability of finding the optimal test samples.

[0018] 4. Under a limited query budget (such as 100 queries), the method of the present invention can achieve a higher attack success rate and a lower perturbation rate. Experimental results show that under the same query budget, the method of the present invention is significantly superior to existing soft label attack methods, with the average attack success rate increased by 2.45% and the average perturbation rate decreased by 2.33%.

[0019] 5. The method of the present invention has wide applicability and is applicable to multiple programming languages (such as Java, Python, and C) and multiple code-related tasks (such as author prediction, clone detection, vulnerability detection, and problem-solving classification), and shows good evaluation effects on different code models (such as CodeBERT, GraphCodeBERT, and CodeT5). BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic flowchart of a method for evaluating the robustness of a code model based on variable importance analysis in a preferred embodiment disclosed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings. In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0022] Embodiment As Figure 1As shown in the figure, the present invention discloses a method for evaluating the robustness of a code model based on variable importance analysis, including the steps: S1. Obtain the code model to be evaluated and source code samples; extract variables from the real source code library to construct a first set of alternative variables.

[0023] Specifically, obtaining suitable evaluation objects and data is the premise of the evaluation process, while constructing a high-quality set of alternative variables directly affects the effectiveness of variable substitution. The present invention first determines the code model to be evaluated, which can be a pre-trained code understanding model or a dedicated code task model for performing tasks such as code clone detection, vulnerability identification, code summary generation, etc. Source code samples are collected from multiple channels, and these samples need to cover mainstream programming languages and should have functional integrity and syntactic correctness. The method of the present invention pays particular attention to the task relevance of the source code samples, that is, the selected samples should match the specific task type of the model to be evaluated. For example, code fragments containing known security vulnerabilities are selected for a vulnerability detection model.

[0024] For the collected source code samples, the method of the present invention uses a professional code parser to perform syntactic analysis and extract various variables therein, including local variables, global variables, function parameters, class attributes, etc. The extracted variable information is processed in a structured manner, and attributes such as the name, type, scope, and occurrence frequency of each variable are recorded. To ensure the quality of the first set of alternative variables, the method of the present invention systematically cleans the variable set, including removing duplicate variable names, filtering variable names with inappropriate lengths, handling special characters, and filtering variable names that coincide with programming language reserved words, etc.

[0025] By means of optimization measures such as balancing the distribution of different categories of variables, introducing domain-specific variable names, and weighting according to usage frequency, the method of the present invention finally constructs a structured and semantically rich first set of alternative variables. This set of alternative variables not only contains the original variable names extracted from real code, but also undergoes systematic cleaning and optimization, which can provide a high-quality candidate library for subsequent variable substitution strategies, ensuring that the generated code variants not only maintain syntactic correctness, but also have sufficient semantic rationality and diversity, thus effectively supporting the robustness evaluation of the code model.

[0026] S2. Calculate the importance scores of each initial variable in the source code samples for the output results of the code model to be evaluated, and the importance scores are calculated based on the soft label method.

[0027] Variable importance calculation aims to quantify the impact degree of each variable on the model decision-making and provide a scientific basis for subsequent variable substitution strategies. The present invention calculates variable importance in a soft label-based manner, that is, not only considering the final predicted category (hard label) of the model, but also fully utilizing the complete probability distribution information (soft label) output by the model, so as to achieve an accurate measurement of the variable impact.

[0028] In some preferred embodiments, a specific importance score calculation process is given, specifically including: Assume that the source code sample is X, which contains n initial variables {var_1, var_2, ..., var_n}, the code model to be evaluated is F, and the predicted probability distribution for the input X is P(Y|X), where Y = {y_1, y_2, ..., y_m} represents m possible output classes. For each initial variable var_i, first generate a modified code sample X / {var_i}, where the variable var_i is replaced with a specific special token (such as "UNK" or " <mask>”), this step is to ensure the compilability of the source code sample and comprehensively calculate the importance of the initial variable to be calculated. Then, the original sample X and the modified sample X / {var_i} are respectively input into the code model F to obtain the corresponding predicted probability distributions P(Y|X) and P(Y|X / {var_i}).

[0029] The importance calculation method of the present invention distinguishes two cases: when the predicted label of the model remains unchanged and the predicted label changes after replacing the variable. Let the true label of the original sample X be y_true, and the predicted label of the model F for X is also y_true, that is, F(X) = y_true. When the variable var_i is replaced and the predicted label of the model is still y_true, that is, F(X / {var_i}) = y_true, the importance score I(var_i) of the variable var_i is calculated as follows: I(var_i) = P(y_true|X) - P(y_true|X / {var_i}) This formula quantifies the degree of decrease in the predicted probability of the correct class caused by replacing the variable var_i. The greater the decrease in probability, the more significant the impact of the variable on the model prediction result, that is, the higher the importance.

[0030] When the variable var_i is replaced and the predicted label of the model becomes y_other, that is, F(X / {var_i}) = y_other ≠ y_true, the importance score I(var_i) of the variable var_i is calculated by the following formula: I(var_i) = [P(y_true|X) - P(y_true|X / {var_i})] - [P(y_other|X / {var_i}) - P(y_other|X)] This formula not only considers the decrease in the probability of the correct class but also the increase in the probability of the wrong class, thus more comprehensively reflecting the impact of variable replacement on the model decision boundary. The first term represents the decrease in the probability of the correct class, and the second term represents the increase in the probability of the wrong class. The difference between the two comprehensively reflects the importance of the variable.

[0031] It should be understood that the soft label method of the present invention has significant advantages compared with the traditional hard label method (only considering whether the predicted class changes). The soft label method can capture those variables that do not change the final predicted class but significantly affect the prediction confidence, providing a more fine-grained importance assessment. Especially in the case of high-confidence predictions, the hard label method may be difficult to distinguish the relative importance of variables, while the soft label method can more accurately identify key variables by considering the probability distribution changes.

[0032] Through the above importance calculation, the method of the present invention can sort all the initial variables in the source code in descending order of importance scores to form a variable importance sorted list. This sorted list will be used to guide the variable replacement strategy in subsequent steps, giving priority to processing variables with higher importance scores, thereby improving the evaluation efficiency and attack success rate. At the same time, the variable importance analysis also provides valuable insights for the interpretability research of the code model, helping to understand the decision-making mechanism of the model and potential robustness defects.

[0033] S3. Screen the alternative variables of the initial variables with importance scores greater than the first preset threshold in the first alternative variable set to construct a second alternative variable set.

[0034] This step improves the pertinence and efficiency of subsequent variable replacement by focusing on the variables that have a significant impact on the model output. The present invention first sets a first preset threshold θ for determining the significance level of variable importance. This threshold can be determined by means of empirical setting, data distribution analysis, or adaptive calculation, etc. For example, sort all the variable importance scores calculated in step S2 in descending order, and select the variable importance values in the top 10%-30% as the threshold, or use the mean plus n times the standard deviation as the threshold.

[0035] In the actual implementation process, for each initial variable var_i with an importance score I(var_i) greater than the threshold θ, the method of the present invention selects a suitable alternative variable from the first alternative variable set. The selection process considers multi-dimensional matching characteristics, including variable type compatibility, naming style consistency, semantic relevance, and context adaptability, etc. For type compatibility, ensure that the alternative variable is compatible with the original variable at the language syntax level. For example, an integer variable should be replaced with an integer variable name; for the naming style, keep the naming specifications consistent before and after replacement, such as camel case naming or snake case naming; for semantic relevance, prefer to select alternative terms that are semantically close to the original variable but not exactly equivalent, maintaining the rationality of the code semantics while introducing sufficient perturbations; for context adaptability, consider the specific usage environment of the variable in the code to avoid generating replacement terms that are semantically contradictory in a specific context.

[0036] In some preferred embodiments, the following method for constructing a second alternative variable set using a multi-layer screening strategy is given, including: S301. In the first alternative variable set, extract the variables whose variable length and importance score are greater than the first preset threshold and the difference in the initial variable length is within the range of the second preset threshold, and construct a third alternative variable set. This step ensures the similarity in length between the alternative variables and the original variables, avoiding code structure changes caused by excessive differences in variable length. The second preset threshold can be set to a reasonable range value, such as ±2 or ±3 characters, to ensure that the replaced code remains basically the same in visual form, so as to more accurately evaluate the sensitivity of the model to semantic changes in variable names.

[0037] In another preferred embodiment, the method of the present invention determines the first preset threshold by using an adaptive calculation mechanism, that is, dynamically adjusts the threshold according to the characteristics of different code samples and the model prediction confidence. Specifically, the Bootstrap sampling technique can be combined to calculate the variable importance distribution through multiple random samplings, and then set the threshold based on the distribution characteristics. For samples with high prediction confidence, the threshold can be appropriately increased to focus on the most critical variables; for samples with low prediction confidence, the threshold can be reduced to consider more potential influencing factors. This dynamic threshold mechanism significantly improves the accuracy of variable screening, making the evaluation process more adaptable to code samples of different complexities.

[0038] S302. Obtain the embedded vector representations of the variables in the third alternative variable set, and apply the K-means clustering algorithm to cluster the variables with similar semantics in the third alternative variable set into K candidate clusters. This step uses modern natural language processing techniques to convert variable names into points in a high-dimensional vector space, and then clusters and groups them based on semantic similarity. The embedded vectors of variables can be obtained through a pre-trained word embedding model or an embedding model specifically designed for code, so that variables with similar semantics are mapped to close positions in the vector space. In a further optimization of this preferred embodiment, when applying the K-means clustering algorithm to cluster the third alternative variable set, the Euclidean distance is used as the similarity metric between the embedded vectors of each variable. As a classic distance metric method, the Euclidean distance can effectively measure the straight-line distance between points in the vector space and is suitable for quantifying the semantic similarity degree between variable embedded vectors. This measurement method more intuitively expresses the semantic distance relationship between variables compared with other measurement methods such as cosine similarity, facilitating the selection of appropriate alternative variables in subsequent steps.

[0039] S303. Select at least one variable from each of the candidate clusters to form a second set of alternative variables. This step ensures that the final set of alternative variables has sufficient semantic diversity because variables from different clusters have distinct semantic differences. By selecting representative variables from each cluster, the method of the present invention constructs a comprehensive set of alternatives that includes both semantically similar and different variables, providing a rich selection space for subsequent variable substitution.

[0040] S4. Iteratively perform variable substitution on the source code sample using the second set of alternative variables, generate and evaluate test samples in each iteration, and continuously optimize the test samples based on the difference between the prediction results of each round of test samples and the source code sample with respect to the code model to be evaluated until the robustness evaluation is completed.

[0041] Step S4 is the core execution phase of the method of the present invention. By generating a test sample set through iterative variable substitution, it systematically evaluates the robustness of the code model. Based on the previously constructed second set of alternative variables, it searches for adversarial samples in the source code sample space that may cause changes in the model's prediction results, thereby comprehensively evaluating the sensitivity of the model to variable naming changes.

[0042] In some preferred embodiments, it specifically includes the following steps: S401. Initialize the test sample set and add the source code sample to the test sample set. Adding the original source code sample as the initial test sample to the test sample set serves as the basis for subsequent variable substitution operations. The initial test sample inherits all the syntactic structures, semantic information, and execution logic of the original source code, ensuring that the subsequent generated variants only differ in variable naming while the functional logic remains completely consistent.

[0043] S402. Traverse each sample in the test sample set. For the current initial variable being processed, replace the initial variable with a candidate alternative variable from the second set of alternative variables to generate a new test sample and add it to the temporary sample set. In this step, the initial variable to be processed can be selected through various strategies. In some preferred embodiments, the initial variables with importance scores greater than the first preset threshold θ are extracted, sorted in descending order of importance scores, and then sequentially selected in each iteration as the current initial variable to be processed in the sorted order. This strategy preferentially processes the variables that have the most significant impact on the model prediction, which can effectively improve the evaluation efficiency.

[0044] S403. Use the code model to be evaluated to test each sample in the temporary sample set, and calculate the difference value of the predicted probability distribution between each sample and the source code sample. The difference value of the predicted probability distribution between each sample and the source code sample refers to the statistical difference measure between the predicted results of the code model to be evaluated for the test sample after variable substitution and the original source code sample in the method of the present invention. Specifically, this difference value quantifies the influence degree of variable substitution on the model output distribution and is the core index for evaluating the sensitivity of the code model to variable naming. When the code model to be evaluated processes the source code sample and the test sample, it will output the probability distribution of each class label. In some preferred embodiments, there are two cases for calculating the difference value: when the predicted label of the test sample is the same as the predicted label of the source code sample, the difference value is calculated as the difference of the original label probabilities, that is, I(var_i) = P(y_true|X) - P(y_true|X'); when the predicted label of the test sample changes, the calculation of the difference value is more complex, and it is necessary to consider both the decrease of the original label probability and the increase of the new label probability, that is, I(var_i) = [P(y_true|X) - P(y_true|X')] - [P(y_other|X') - P(y_other|X)]. This method of calculating the difference value not only considers the change of probability values, but also reflects whether this change is sufficient to cause a change in the prediction result, so as to comprehensively evaluate the stability of the model's decision boundary. In the improved beam search process, the sample with a larger difference value is considered more likely to reveal the robustness weakness of the model, so it will be preferentially retained as the basis for the next round of iteration to efficiently discover the sensitive area of the model to variable naming changes.

[0045] S404. Merge the test sample set retained in the previous round of iteration with the currently generated temporary sample set, and select the top B samples with the largest difference values as the new test sample set. This step realizes the dynamic evolution of the test sample set. The test sample set retained in the previous round of iteration is merged with the currently generated temporary sample set, and the top B samples with the largest difference values are selected as the new test sample set. The parameter B is the number of samples to be retained and is the key hyperparameter for controlling the search breadth. A larger B value is beneficial to comprehensively explore the variable substitution space, while a smaller B value provides a more focused search to accelerate the discovery of adversarial samples.

[0046] In the traditional beam search algorithm, only the samples generated in the current iteration are considered in each round of iteration. However, in the improved method of the present invention, during the sample selection stage of each round of iteration, two parts of sample sets are merged: one part is the B most promising test samples (test sample set) retained after evaluation in the previous round of iteration, and the other part is a series of samples newly generated by performing variable substitution on these B samples in the current round (temporary sample set). This merging operation significantly expands the scale and diversity of the candidate sample pool, thus forming a larger search space. In some preferred embodiments, when the beam width parameter B is set to a relatively large value (such as B = 10), the merged candidate pool may contain hundreds of candidate samples. Subsequently, the system selects the top B samples with the largest difference value from this expanded candidate pool as the basis for the next round of iteration. This innovative merging mechanism effectively solves the problem that traditional beam search and greedy algorithms are prone to falling into local optima, because it allows the temporarily ignored search paths to "resurrect" in subsequent iterations. When some samples discarded earlier may produce more ideal evaluation results after being combined with new variable substitutions, they have the opportunity to re-enter the consideration range. This dynamic integration strategy not only maintains the breadth and diversity of the search, but also controls the computational complexity through the beam width parameter, providing a more comprehensive and efficient technical solution for the robustness evaluation of the code model.

[0047] S405. Repeat steps S402 to S404 until the stop condition is met. The stop condition refers to the judgment criterion for terminating the current search process during the iterative execution of variable substitution tests in the improved beam search algorithm. Specifically, the stop condition is a set of preset judgment rules for determining when to end the evaluation iteration of the code model robustness. During the execution of the improved beam search, the system continuously replaces important variables and evaluates the model response, and the stop condition determines when this process terminates, ensuring both the sufficiency of the evaluation and avoiding unnecessary waste of computing resources. In some preferred embodiments, the stop condition includes three situations: First, when a test sample that can change the prediction result of the code model to be evaluated is found, it indicates that the robustness weakness of the model has been successfully identified, and the search can be stopped immediately at this time; Second, when all initial variables with importance scores greater than the first preset threshold have completed the substitution test, it indicates that all possible substitutions of important variables have been exhausted, and it is meaningless to continue the test; Finally, when the number of iterations reaches the preset upper limit, to avoid excessive consumption of computing resources, the system will also terminate the search process. The setting of these three stop conditions ensures that the evaluation process can comprehensively detect the model robustness and efficiently complete within a reasonable computational complexity range.

[0048] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.< / mask>

Claims

1. A code model robustness evaluation method based on variable importance analysis, characterized in that: Includes steps: S1. Obtain the code model to be evaluated and the source code sample; extract variables from the real source code library to construct the first substitute variable set; S2. Calculate the importance score of each initial variable in the source code sample to the output result of the code model to be evaluated, wherein the importance score is calculated based on the soft label method; S3. Selecting replacement variables of the initial variables whose importance scores are greater than a first preset threshold from the first replacement variable set to construct a second replacement variable set; S4. Iteratively perform variable replacement on the source code sample using the second replacement variable set, generate and evaluate test samples in each round of iteration, and continuously optimize the test samples based on the difference in prediction results between each round of test samples and source code samples by the code model to be evaluated until the robustness evaluation is completed.

2. The method for evaluating the robustness of a code model based on variable importance analysis according to claim 1, characterized in that: The method for constructing the second set of substitution variables includes: S301. From the first replacement variable set, extract variables whose variable lengths differ from the initial variable lengths whose importance scores are greater than the first preset threshold and are within the range of the second preset threshold, and construct a third replacement variable set; S302. Obtain an embedded vector representation of each variable in the third replacement variable set, and apply a K-means clustering algorithm to cluster semantically similar variables in the third replacement variable set into K candidate clusters; S303. Select at least one variable in each candidate cluster to form a second replacement variable set.

3. The code model robustness evaluation method based on variable importance analysis according to claim 1, characterized in that: Step S4 includes: S401. Initialize a test sample set and add the source code sample to the test sample set; S402. Traverse each sample in the test sample set, replace the initial variable currently being processed with a candidate replacement variable in the second replacement variable set, generate a new test sample and add it to the temporary sample set; S403. Use the code model to be evaluated to test each sample in the temporary sample set, and calculate the difference value of the predicted probability distribution between each sample and the source code sample; S404. Combine the test sample set retained in the previous iteration with the currently generated temporary sample set, and select the first B samples with the largest difference values ​​as the new test sample set; S405. Repeat steps S402 to S404 until the stop condition is met.

4. The method for evaluating the robustness of a code model based on variable importance analysis according to claim 3, characterized in that: The method for selecting the initial variable of the current processing in step S402 includes: Initial variables with importance scores greater than a first preset threshold are extracted and sorted in descending order of importance scores, and then selected in order of sorting as initial variables for current processing in each iteration.

5. The method for evaluating the robustness of a code model based on variable importance analysis according to claim 3, characterized in that: The stop conditions include: Find test samples that can change the prediction results of the code model to be evaluated; All initial variables with importance scores greater than the first preset threshold have been processed; The number of iterations reaches a third preset threshold.

6. The method for evaluating the robustness of a code model based on variable importance analysis according to claim 1, characterized in that: The method for calculating the importance score of each initial variable in the source code sample to the output result of the code model to be evaluated in step S2 includes: replacing the initial variables to be calculated in all instances of the source code sample with special tags to ensure the compilability of the source code sample and comprehensively calculate the importance of the initial variables to be calculated.

7. The method for evaluating the robustness of a code model based on variable importance analysis according to claim 2, characterized in that: When the K-means clustering algorithm is applied to cluster the third set of alternative variables in step S302, the Euclidean distance is used as the similarity metric between the embedding vectors of each variable.

8. A code model robustness evaluation system based on variable importance analysis, characterized by: The system is used to implement the code model robustness assessment method based on variable importance analysis of any one of claims 1 to 7 above.

Images