An iterative method for generating adversarial examples for large models

Abstract

The present invention discloses an iterative generation method for adversarial samples for large models, including constructing a search space based on candidate identifier index vectors; generating adversarial examples using a Bayesian optimization framework, and dynamically selecting the optimal candidate points through a Gaussian process proxy objective function; performing diversity sampling on candidate samples in combination with a determinant point process to ensure that the generated adversarial examples have a wide coverage range and are highly aggressive; iteratively optimizing the generation process, dynamically adjusting the direction according to attack feedback, and gradually optimizing the efficiency and success rate of adversarial example generation. The present invention effectively solves the local optimality problem and efficiency dilemma in existing adversarial example generation methods by combining feedback optimization and diversity sampling technology. Experiments show that the adversarial examples generated by the present invention exhibit significant attack effects in tasks such as vulnerability prediction, clone detection, and code summarization, can fully test the robustness of deep code models, and provide technical support for model improvement and optimization.

Description

Technical Field

[0001] The present invention relates to the field of security of large code models, and in particular to a method for generating iterative adversarial examples for deep code models. Background Art

[0002] Deep code models have made significant progress in recent years and have become an important tool in the field of software engineering, widely used in tasks such as code comprehension and code generation. For example, in code comprehension, deep code models are used for tasks such as vulnerability detection and code clone detection; in code generation, they are used for tasks such as code summarization and code completion. Typical deep code models include CodeBERT, CodeGPT, GraphCodeBERT, CodeT5, and PLBART. These models, trained through self-supervised learning on large-scale code corpora, are able to capture syntactic and semantic relationships in code, providing efficient support for downstream tasks.

[0003] Despite the impressive performance of deep code models, their security and robustness have raised widespread concern, particularly in safety-critical applications such as vulnerability detection systems. If a model incorrectly classifies vulnerable code as safe, serious security consequences can result. This potential threat emphasizes the importance of thorough testing and evaluation of deep code models to identify their vulnerabilities and improve their robustness.

[0004] Adversarial attacks are an important means of testing the robustness of deep code models. Adversarial attacks generate adversarial examples that can deceive the model by making subtle, semantically preserving modifications to the input code (such as identifier replacement). These adversarial examples cause the model to produce outputs that differ from the original code, revealing model vulnerabilities. For example, by modifying identifier names while preserving code semantics, adversarial examples can be generated that mislead the model, exposing the model's sensitivity to input variations.

[0005] However, existing adversarial attack methods face the following challenges in deep code models:

[0006] 1. Insufficient Utilization of Feedback Information: Current attack methods typically focus only on whether an adversarial example is successful, while underutilizing feedback from failed attacks. Failed attacks may contain important clues about the model's decision boundary, which is crucial for guiding subsequent attacks, but is often overlooked.

[0007] 2. Local Optima: Many methods rely on greedy search strategies to gradually generate adversarial examples, such as replacing identifiers one by one. However, this approach can get stuck in local optima, preventing the exploration of better solutions. For example, if the initial choice of identifier replacement fails to achieve optimal results, subsequent generation may be limited by it.

[0008] 3. Efficiency Dilemma: Currently, the most successful adversarial attack methods often require a large number of model queries, which is both time-consuming and may trigger defense mechanisms. For example, methods based on exhaustive search can improve the success rate but significantly increase computational costs and the number of queries.

[0009] To address these issues, various improvements have been proposed. For example, black-box attack techniques generate adversarial examples through identifier replacement. Representative methods such as ALERT and BeamAttack employ greedy search and beam search strategies, respectively, to optimize attack efficiency and success rate. However, these methods still have significant shortcomings in fully utilizing feedback information, achieving global optimization, and reducing the number of queries. More efficient solutions are urgently needed. Summary of the Invention

[0010] In response to the above-mentioned problems existing in the prior art, the technical problem to be solved by the present invention is: the technical problems of local optimality and efficiency dilemma in the existing adversarial example generation method.

[0011] To solve the above technical problems, the present invention adopts the following technical solution: an iterative generation method of adversarial samples for large models, comprising the following steps:

[0012] S100: Construct candidate identifier index vector search space T v , obtain the original code, extract the identifier set I from the original code, generate a candidate replacement set for each identifier in the identifier set I, and use the candidate replacement set to construct the identifier search space T c , T c Each identifier combination in is mapped to a candidate identifier index vector CIIV, and CIIV is used to construct T v .

[0013] S200: Constructing a historical failed attack set through random sampling and historical records and use Train the Gaussian process GP as a proxy model and use the expected improvement acquisition function to select the optimal sampling point v′ opt , and according to v′ opt The optimal code variants sampled correspond to Update the evaluation results And continue training the proxy model until preliminary generative adversarial examples are obtained.

[0014] S300: The current optimal sampling point v′ obtained according to S200 opt, select all candidate sampling points whose Hamming distance does not exceed the set threshold to build a distance sphere H, and select e candidate sampling points from H to form the initial candidate sampling point set S; v The candidate sampling points that can maximize the diversity gain in S are selected until S reaches the set maximum capacity; the candidate sampling points in S are input into the target model for evaluation, and the posterior distribution of the proxy model is updated according to the objective function value until the set budget limit is reached. The proxy model is no longer updated and the adversarial examples of successful attacks selected in each iteration are output.

[0015] Furthermore, the candidate identifier index vector search space T is constructed in S100 v The steps are as follows:

[0016] S101: Map a piece of original code to an identifier set, extract the identifier set I={x1,x2,…,x l}.

[0017] S102: Generate a candidate replacement set for each identifier in the identifier set I, ensuring that the replacement items comply with the grammatical rules and maintain the semantic consistency of the code; the candidate replacement set is a set of candidate identifiers derived from an open source test dataset, which is used to replace the identifiers in the original code.

[0018] S103: Use the candidate replacement set to construct the identifier search space T c , a set of identifiers corresponds to a T c , T c The elements in are called identifier combinations; a piece of code obtained by replacing the identifiers with the same meaning in the original code with an identifier combination is called a code variant. An identifier combination represents a code variant:

[0019]

[0020] Where l represents the number of identifiers, C i represents the set of candidate replacements for the i-th identifier.

[0021] S104: T c Each identifier combination in is mapped to a candidate identifier index vector CIIV, where each element in CIIV represents the index of an identifier in the identifier combination in the candidate replacement set. The CIIV of each identifier combination is expressed as:

[0022] v′=[v1,v2,…v i …,v l ] (2)

[0023] where v iRepresents the index of the i-th identifier in its candidate replacement set.

[0024] Then use the CIIV of each identifier combination to construct the candidate identifier index vector search space T v :

[0025]

[0026] in, A set of CIIVs representing all combinations of identifiers.

[0027] Furthermore, the process of obtaining the preliminary generated adversarial examples is as follows:

[0028] S201: First, use the original code to attack the deep code model. If the attack is successful, a valid adversarial example is obtained and defined as the initial generated adversarial example output. If the attack fails, the Bayesian optimization framework is used to mine the evaluation information of the failed attack to find the code variant x′ that is mapped to the optimal attack vector and has the potential to successfully carry out the adversarial attack. Code understanding task: Define the function that maximizes the negative margin to measure the difference in the prediction confidence of the attacked deep code model for the category, where the attacked deep code model is called the target model:

[0029]

[0030] Where Y represents the set of all possible categories of the target model, and Represents classes respectively and real class The predicted probability of .

[0031] Code generation task: Define the code variants that are subjected to adversarial attacks as adversarial examples, and define the maximum negative BLEU score to measure the semantic similarity between the adversarial example and the original code:

[0032]

[0033] Among them, L BLEU (x′) is the BLEU score of x′.

[0034] S202: Construct a historical failed attack set through random sampling and historical records

[0035] From T c Randomly sample n identifier combinations, replace the identifiers in each identifier combination with the identifiers with corresponding meanings in the original code, and obtain code variants. The formula for determining the number of samples n is:

[0036]

[0037] Where l represents the total number of identifiers in the original code, and N represents the total number of identifiers and their candidate sets.

[0038] The code variants obtained by random sampling and replacement are input into the target model for evaluation, and the objective function value output after each code variant is input into the target model is recorded. mar gin (x′ i ) and -L BLEU (x′ i );

[0039] For the comprehension task, if -L margun (x′ i )>0, it means that the code variant is a successful attack, otherwise it means that the code variant is a failed attack;

[0040] For build tasks, if -L BLEU (x′ i )=0, it means that the code variant is a successful attack, otherwise it means that the code variant is a failed attack;

[0041] Use the CIIV corresponding to the code variants that failed the attack and their corresponding objective function values ​​to construct the historical failed attack set

[0042]

[0043] Among them, v′ i CIIV,x′ corresponding to the code variant that indicates the failed attack i Code variant indicating a failed attack, where n represents the number of samples.

[0044] S203: Use Train the Gaussian process GP as the proxy model, and the posterior distribution of the proxy model is expressed as:

[0045]

[0046] μ=k(v1′,X)(k(X,X)+τ 2 I′) -1 Y (9)

[0047] ∑=k(v′1,v′2)-k(v′1,X)(k(X,X)+τ 2 I′) -1 k(X,v′2) (10)

[0048] in, is a normal distribution, v′1 and v′2 are two different CIIVs, and X is The matrix composed of CIIV, μ is the mean function, ∑ is the covariance matrix, τ 2 is the noise variance, I′ is the identity matrix used for regularization, k(v′1,v′2) is the kernel function, k(v′1,X) is the kernel function matrix, and k(X,X) is the kernel matrix.

[0049] The kernel function k(v′1,v′2) is:

[0050]

[0051] where δ(·) is the Kronecker delta function, v′ i is the code variant x′ i Corresponding to CIIV, l is the total number of identifiers in the code, indicating the dimension of CIIV, w 1i and w 2i Represents the i-th element in the two CIIV vectors, which is the identifier replacement corresponding to the two different code variants. is the signal variance, β i is the length scale parameter corresponding to the correlation of the ith element.

[0052] S204: Based on the trained proxy model, use the expected improved acquisition function to select the optimal sampling point v′ opt ,The calculation formula of expected improved acquisition function is:

[0053]

[0054] Among them, v′ opt represents the optimal sampling point, v′∈T v Indicates T v The sampling points v′,y in * Represents the current optimal objective function value; in the classification task, y * is max-L margin (x′ i ); in the generation task, y * is max-L BLEU (x i ′), Represents the objective function value calculated by the surrogate model M The predicted distribution of max(y * -y,0) is the measure of improvement, which indicates the improvement value that the current sampling point v′ may bring.

[0055] S205: According to v′ opt The optimal code variants sampled correspond to The evaluation results of execution:

[0056] For the comprehension task, when -Lmargin (x i ′)>0, then stop the iteration and return the adversarial example x′ opt , and the adversarial example x′ opt Defined as the initial generation of adversarial example output; otherwise, the current sampling result and L margin (x i ')join in and update And return to S203.

[0057] For build tasks, when -L BLEU (x i ′)=0, then stop the iteration and return the adversarial example x′ opt , and the adversarial example x′ opt Defined as the initial generation of adversarial example output; otherwise, the current sampling result and L BLEU (x i ')join in and update And return to S203.

[0058] Furthermore, in S300, the process of outputting the adversarial examples selected for successful attacks in each round of iteration is as follows:

[0059] S301: Constructing a set of candidate sampling points: Calculating the current optimal sampling point v′ opt The Hamming distance with other candidate sampling points is based on the calculated Hamming distance at the current optimal sampling point v′ opt Construct a distance ball H around it, which contains the current optimal sampling point v′ opt The Hamming of all candidate sampling points does not exceed the set threshold, and the first e candidate sampling points with the largest expected improvement EI are selected from H to form the initial candidate sampling point set S:

[0060] S=[v′ i1 ,v′ i2 ,…,v′ ie ] (13)

[0061] Among them, v′ i1 Indicates that the position index of the i-th candidate acquisition point in S is 1.

[0062] The calculation method of Hamming distance is:

[0063]

[0064] Among them, v1 and v2 represent the optimal sampling point v′ respectively. opt The corresponding CIIV and CIIV corresponding to other candidate sampling points, v1,i and v 2,i Indicates the element at the corresponding position.

[0065] Maximum expected improvement (EI):

[0066]

[0067] Where v′∈T v Indicates T v The sampling points v′,y in * Represents the current optimal objective function value, y * is max-L margin (x′ i ); in the generation task, y * is max-L BLEU (x′ i ), represents the predicted distribution of the objective function value y calculated by the surrogate model M, max(y * -y,0) is the measure of improvement, which indicates the improvement value that the current sampling point v′ may bring.

[0068] S302: For S, use formula (11) to calculate the similarity matrix V ij =sim(S,S), which is used to quantify the similarity between candidate sampling points, that is, to measure the proximity of two candidate sampling points in the search space. ij Represents the candidate sampling point v i ′ and v j The similarity matrix is ​​calculated based on the CIIV of the candidate sampling points.

[0069] S303: Calculate the similarity matrix between a candidate sampling point in S and other candidate sampling points, and use the similarity matrix to select the candidate sampling point with the largest dissimilarity gain. This process uses a greedy strategy to iteratively select the optimal candidate identifier index vector v * , to maximize the total dissimilarity gain of S. In each iteration, candidate sampling points that maximize the diversity gain are selected until S reaches the set maximum capacity. This process is implemented by the following objective function:

[0070]

[0071] Among them, arg max represents the independent variable that makes a function reach its maximum value, v′∈T v \S means outside S, from T v Select a new candidate vector v′ from the matrix, det represents the determinant operation, and Represent the posterior covariance matrix of the current S and the updated candidate sampling point set S∪v′ respectively.

[0072] In each sampling iteration, candidate sampling points that have been evaluated in previous iterations are excluded.

[0073] S304: Input the candidate sampling points in S into the target model for evaluation. During the evaluation process, the objective function value of each candidate sampling point is recorded and the posterior distribution of the proxy model is updated. The process returns to S302 until the set budget limit is reached. The proxy model no longer updates and outputs the successful adversarial examples selected in each iteration.

[0074] Compared with the prior art, the present invention has at least the following advantages:

[0075] By combining a Bayesian optimization framework with diversity sampling techniques, the proposed method dynamically adjusts attack direction, fully leverages feedback information, balances exploration and exploitation to effectively avoid local optima, and significantly reduces the number of model calls to improve the success rate and efficiency of adversarial example generation. This provides a new technical approach for security testing of deep code models. This approach offers a highly efficient and reliable approach for security testing of deep code models, with significant application value in areas such as code quality testing and vulnerability analysis.

[0076] 1. Efficient Generation of Adversarial Examples: Through Bayesian Optimization, this invention can reduce the number of queries to the target model. By establishing a proxy model (Gaussian process), Bayesian Optimization can predict the most likely successful attack direction with fewer model queries, significantly reducing computing resource consumption and improving the efficiency of adversarial example generation.

[0077] 2. Avoiding Local Optima: Diversity Profiling (DPP) technology effectively expands the search range and avoids the local optimality problem caused by greedy algorithms. By using the DPP algorithm, the present invention selects multiple distinct candidate points in each sampling, maximizing the diversity of the current candidate set and increasing the probability of successful attacks. Diversity sampling ensures the breadth of the attack space, thereby reducing the risk of falling into local optimal solutions.

[0078] 3. Adaptability to Multiple Task Scenarios: This invention supports two major task scenarios: code comprehension and code generation. In the code comprehension task, the generation of adversarial examples is optimized by maximizing the negative margin function; in the code generation task, the BLEU score is optimized to ensure that the generated code variants maintain semantic consistency with the original code. This approach enables deep code models to cope with a variety of attack scenarios.

[0079] 4. Improved Model Robustness: This paper further enhances the robustness of deep code models against malicious attacks through adversarial fine-tuning techniques. Adversarial examples generated in each round of sampling are used to update the posterior distribution of the proxy model, thereby strengthening the model's resistance to malicious variants and improving the stability of deep code models in security-critical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0081] The present invention is described in further detail below.

[0082] See also Figure 1 , the present invention proposes an iterative generation method for adversarial samples for large models, referred to as ITGen. ITGen is suitable for code understanding tasks (such as vulnerability detection, clone detection) and code generation tasks (such as code summary generation, code completion). By combining the bit vector-based search space, Bayesian optimization framework and diversity sampling strategy, it significantly improves the efficiency and success rate of adversarial example generation and enhances the robustness of the model. Specifically, ITGen includes the following steps: constructing a search space through bit vector representation, using feedback information to guide optimization, generating optimal candidate variants based on Bayesian optimization, and expanding the search range in combination with a diversity sampling strategy. ITGen can effectively generate adversarial examples in a black box environment and improve the robustness of deep code models through adversarial fine-tuning.

[0083] The present invention utilizes search space construction, Bayesian optimization and diversity sampling (DPP) technology in adversarial testing of deep code models to solve the local optimal problem and efficiency dilemma in existing adversarial example generation methods.

[0084] 1. Local Optimum Problem: The local optimality problem refers to the fact that existing adversarial example generation methods are prone to falling into local optimal solutions during the attack process. That is, they only find a locally effective solution and cannot explore the global optimal solution, thus limiting the success rate and scope of the adversarial attack.

[0085] Limitations of existing methods: Traditional adversarial example generation methods, such as greedy algorithms, rely on step-by-step identifier replacement and local search. One identifier is replaced at a time, and if a step succeeds, the search continues. This approach is prone to previously selected identifiers influencing subsequent choices, causing the algorithm to become trapped in a local optimum. For example, in adversarial attacks on deep learning models, replacing certain identifiers may have a significant impact on the model. However, if this replacement causes the search space to become too restricted in a certain area, subsequent attacks may be unable to find more promising adversarial examples.

[0086] The solution provided by the present invention is:

[0087] (1) Multiple Identifier Replacement and CIIV Representation: This invention supports simultaneous replacement of multiple identifiers through the CIIV (Candidate Identifier Index Vector) representation. While traditional methods replace identifiers one by one, this invention effectively expands the search space by replacing multiple identifiers simultaneously. Because multiple identifiers can be replaced at once, path constraints are reduced, thus avoiding the search direction limitations caused by replacing a single identifier and reducing the risk of falling into a local optimum.

[0088] (2) Bayesian optimization framework: The present invention uses Bayesian optimization as the core framework for adversarial example generation. The Bayesian optimization framework predicts the objective function value through a Gaussian process proxy model and dynamically selects the most promising attack point, avoiding the greedy algorithm from falling into local optimality. The expected improvement (EI) acquisition function of Bayesian optimization dynamically selects new sampling points based on existing feedback information to maximize the success probability of the adversarial attack with the minimum number of samples. Through multiple optimizations and feedback updates, Bayesian optimization can guide the search direction towards a wider attack space, thereby avoiding local optimal solutions.

[0089] (3) DPP (Determinant Point Process) Sampling: To further avoid local optimality, the present invention introduces the DPP sampling algorithm, which can select diverse samples from multiple candidate sampling points. In each round of sampling, DPP calculates the similarity of candidate sampling points and selects samples that maximize dissimilarity, thereby ensuring that the selected attack points have a high diversity. In this way, the present invention not only increases the attack coverage in each sampling, but also avoids the pitfalls caused by local optimality.

[0090] 2. Efficiency dilemma: The efficiency dilemma refers to the fact that existing adversarial example generation methods require a large number of model queries during the attack process, which consumes a lot of computing resources and increases the attack time, reducing the efficiency of the attack.

[0091] Efficiency Issues of Existing Methods: Traditional methods typically rely on strategies such as exhaustive search and greedy algorithms. During the adversarial example generation process, each attack involves multiple queries of the target model. Especially when the attack space is large, the number of model queries increases exponentially, resulting in wasted computing resources and the triggering of defense mechanisms. To find adversarial examples, existing methods often require numerous model evaluations, resulting in low efficiency. Furthermore, traditional methods fail to fully utilize feedback information. After each evaluation, they simply select the next candidate without dynamically adjusting the search direction using existing information, exacerbating search redundancy and inefficiency.

[0092] The solution provided by the present invention is:

[0093] (1) Bayesian Optimization Framework: This invention significantly reduces the number of model queries by introducing a Bayesian optimization framework. Bayesian optimization predicts the potential value of the target function by establishing a proxy model (Gaussian process), thereby avoiding the situation where a large number of queries are required each time. The proxy model fits the distribution of the target function through historical data and dynamically adjusts the sampling points, which can effectively guide the search process and reduce the actual number of queries to the target model. The goal of Bayesian optimization is to maximize the expected improvement through fewer model calls, significantly improving the efficiency of attack generation.

[0094] (2) Expected Improvement (EI) Acquisition Function: The Expected Improvement (EI) acquisition function selects the most promising samples based on the current state of the model in each iteration, thus avoiding the computational cost of exhaustive search. By calculating the potential improvement value of the current sampling point, the EI acquisition function helps select the most effective attack point in each query, thereby reducing the occurrence of invalid queries and improving the efficiency of adversarial example generation.

[0095] (3) DPP Sampling and Redundant Evaluation Elimination: Through DPP sampling, the present invention can expand the search space by increasing sample diversity, thereby reducing redundant queries. When a new sampling point is selected, previously evaluated candidate points are excluded, avoiding repeated calculations. This strategy effectively reduces the number of redundant evaluations and significantly improves sampling and optimization efficiency.

[0096] A method for iteratively generating adversarial samples for large models, comprising the following steps:

[0097] S100: Construct candidate identifier index vector search space T v , obtain the original code, extract the identifier set I from the original code, generate a candidate replacement set for each identifier in the identifier set I, and use the candidate replacement set to construct the identifier search space T c , T c Each identifier combination in is mapped to a candidate identifier index vector CIIV, and CIIV is used to construct T v .

[0098] S200: Constructing a historical failed attack set through random sampling and historical records and use Train the Gaussian process GP as a proxy model and use the expected improvement acquisition function to select the optimal sampling point v′ opt , and according to v′ opt The optimal code variants sampled correspond to Update the evaluation results And continue training the proxy model until preliminary generative adversarial examples are obtained.

[0099] S300: The current optimal sampling point v′ obtained according to S200 opt , select all candidate sampling points whose Hamming distance does not exceed the set threshold to build a distance sphere H, and select e candidate sampling points from H to form the initial candidate sampling point set S; v The candidate sampling points that can maximize the diversity gain in S are selected until S reaches the set maximum capacity; the candidate sampling points in S are input into the target model for evaluation, and the posterior distribution of the proxy model is updated according to the objective function value until the set budget limit is reached. The proxy model is no longer updated and the adversarial examples of successful attacks selected in each iteration are output.

[0100] Specifically, in S100, a candidate identifier index vector search space T is constructed. v The steps are as follows:

[0101] S101: Map a piece of original code to an identifier set, extract the identifier set I={x1,x2,…,x l}, identifiers can be programming elements such as local variables and function names.

[0102] S102: Generate a candidate replacement set for each identifier in identifier set I, ensuring that the replacement item complies with grammatical rules and maintains semantic consistency of the code; the candidate replacement set is composed of candidate identifiers from open source test datasets, such as BigCloneBench and CodeSearchNet, and is used to replace the identifier in the original code.

[0103] S103: In order to represent all candidate replacement sets of the identifier, the candidate replacement sets are used to construct the identifier search space T c , a set of identifiers corresponds to a T c , T c The elements in are called identifier combinations; the code resulting from replacing the original code with an identifier combination with the same meaning is called a code variant. An identifier combination represents a code variant: a code variant is mapped to an identifier combination.

[0104]

[0105] Where l represents the number of identifiers, C i represents the set of candidate replacements for the i-th identifier.

[0106] For example, if the identifier set is {search, arr}, and the candidate replacement sets are {search, lookup} and {arr, array}, then the search space T cContains the following identifier combinations:

[0107] T c ={(search,arr),(search,array),(lookup,arr),(lookup,array)}

[0108] For example: Original code: int search; int arr; sort(search,arr)

[0109] Code variant: int lookup; int array; sort(lookup, array)

[0110] ↓Map

[0111] Identifier combination: (lookup, array)

[0112] S104: T c Each identifier combination in is mapped to a candidate identifier index vector CIIV, where each element in CIIV represents the index of an identifier in the identifier combination in the candidate replacement set. The CIIV of each identifier combination is expressed as:

[0113] v′=[v1,v2,…v i …,v l ] (2)

[0114] where v i represents the index of the ith identifier in its candidate replacement set. For example, the CIIV of the identifier combination (lookup, array) is v′=[1,1].

[0115] Then use the CIIV of each identifier combination to construct the candidate identifier index vector search space T v :

[0116]

[0117] in, A set of CIIVs representing all combinations of identifiers.

[0118] The search space T can be expressed as c All identifier combinations in are mapped to index vector representations. This step not only replaces all identifiers in the original code at the same time, but also fully represents the search space T in the form of CIIV cFor each identifier combination in , all possible code variants are ultimately mapped to CIIV. This representation not only comprehensively constructs all possible code variants, but also avoids the search from falling into local optimality by supporting multiple identifier replacements, thereby improving search efficiency and the success rate of adversarial example generation. For example:

[0119] T c ={(search,arr),(search,array),(lookup,arr),(lookup,array)}

[0120] ↓Map

[0121] T v ={(0,0),(0,1),(1,0),(1,1)}

[0122] Specifically, the process of obtaining the initial generation of adversarial examples is as follows:

[0123] S201: Establish candidate identifier index vector search space T v Finally, the original code is used to attack the deep code model. If the attack is successful, a valid adversarial example is obtained and defined as the preliminary generated adversarial example output. If the attack fails, the Bayesian optimization framework is used to mine the evaluation information of the failed attack to find the code variant x′ that is mapped to the optimal attack vector and has the potential to successfully carry out the adversarial attack.

[0124] Code understanding task: In order to find a code variant x′ that is likely to successfully conduct an adversarial attack, a function that maximizes the negative margin is defined to measure the difference in the prediction confidence of the attacked deep code model for the category, where the attacked deep code model is called the target model:

[0125]

[0126] Where Y represents the set of all possible categories of the target model, and Represents classes respectively and real class The predicted probability of .

[0127] Code generation task: Define the code variant that is subjected to adversarial attack as an adversarial example. In order to find the code variant x′ that is likely to successfully conduct adversarial attack, define the maximum negative BLEU score to measure the semantic similarity between the adversarial example and the original code:

[0128]

[0129] Among them, L BLEU (x′) is the BLEU score of x′.

[0130] S202: Construct a historical failed attack set through random sampling and historical records

[0131] From T c Randomly sample n identifier combinations, replace the identifiers in each identifier combination with the identifiers with corresponding meanings in the original code, and obtain code variants. The formula for determining the number of samples n is:

[0132]

[0133] Where e represents the total number of identifiers in the original code, and N represents the total number of identifiers and their candidate sets. This formula ensures that oversampling is avoided when there are few identifiers, while covering a wider search space when there are many identifiers.

[0134] The code variants obtained by random sampling and replacement are input into the target model for evaluation, and the objective function value output after each code variant is input into the target model is recorded. mar gin (x′ i ) and -L BLEU (x′ i );

[0135] For the comprehension task, if -L margin (x′ i )>0, it means that the code variant is a successful attack, otherwise it means that the code variant is a failed attack;

[0136] For build tasks, if -L BLEU (x′ i )=0, it means that the code variant is a successful attack, otherwise it means that the code variant is a failed attack;

[0137] Use the CIIV corresponding to the code variants that failed the attack and their corresponding objective function values ​​to construct the historical failed attack set

[0138]

[0139] Among them, v′ i CIIV,x′ corresponding to the code variant that indicates the failed attack i Code variant indicating a failed attack, where n represents the number of samples.

[0140] S203: Use The Gaussian process GP is trained as a proxy model. The Gaussian process as a proxy model can predict the target function value and dynamically adjust the search direction to generate effective adversarial examples. The posterior distribution of the proxy model is expressed as:

[0141]

[0142] μ=k(v′1,X)(k(X,X)+τ 2 I′) -1 Y (9)

[0143] ∑=k(v′1,v′2)-k(v′1,X)(k(X,X)+τ 2 I′) -1 k(X,v′2) (10)

[0144] in, is a normal distribution, v′1 and v′2 are two different CIIVs, and X is The matrix composed of CIIV, μ is the mean function, ∑ is the covariance matrix (part of the posterior distribution), τ 2 is the noise variance, I′ is the identity matrix used for regularization, k(v′1,v′2) is the kernel function, k(v′1,X) is the kernel function matrix, and k(X,X) is the kernel matrix.

[0145] The kernel function k(v′1,v′2) is:

[0146]

[0147] where δ(·) is the Kronecker delta function, v′ i is the code variant x′ i Corresponding to CIIV, l is the total number of identifiers in the code, indicating the dimension of CIIV, w 1i and w 2i Represents the i-th element in the two CIIV vectors, which is the identifier replacement corresponding to the two different code variants. is the signal variance, β i is the length scale parameter corresponding to the correlation of the ith element.

[0148] S204: Based on the trained proxy model, use the expected improved acquisition function to select the optimal sampling point v′ opt ,The expected improvement acquisition function aims to guide the selection of the next sampling point towards the direction of the maximum possible performance improvement and optimize the objective function. The calculation formula of the expected improvement acquisition function is :

[0149]

[0150] Among them, v′ opt represents the optimal sampling point, v′∈T v Indicates T v The sampling points v′,y in * Represents the current optimal objective function value; in the classification task, y* is max-L margin (x′ i ); in the generation task, y * is max-L BLEU (x′ i ), Represents the objective function value calculated by the surrogate model M The predicted distribution of max(y * -y,0) is the measure of improvement, which indicates the improvement value that the current sampling point v′ may bring.

[0151] S205: According to v′ opt The optimal code variants sampled correspond to The evaluation results of execution:

[0152] For the comprehension task, when -L margin (x′ i )>0, then stop the iteration and return the adversarial example x′ opt , and the adversarial example x′ opt Defined as the initial generation of adversarial example output; otherwise, the current sampling result and L margin (x′ i )join in and update And return to S203. Continue training the proxy model until the attack budget limit is reached or effective adversarial examples are initially generated.

[0153] For build tasks, when -L BLEU (x′ i )=0, then stop the iteration and return the adversarial example x′ opt , and the adversarial example x′ opt Defined as the initial generation of adversarial example output; otherwise, the current sampling result and L BLEU (x′ i )join in and update And return to S203 until the attack budget limit is reached or a valid adversarial example is preliminarily generated.

[0154] Specifically, the component uses the Determinant Point Process (DPP) algorithm to obtain the v The algorithm selects candidate sampling points with higher diversity, thereby expanding the search range and avoiding falling into local optimal solutions. Its main goal is to further optimize the generation process of adversarial examples by selecting candidate sampling points with maximum diversity, so as to efficiently generate effective adversarial examples.

[0155] Specifically, in S300, the process of outputting the successful adversarial examples selected in each round of iteration is as follows:

[0156] S301: Constructing a set of candidate sampling points: Calculating the current optimal sampling point v′ opt The Hamming distance with other candidate sampling points is based on the calculated Hamming distance at the current optimal sampling point v′ opt Construct a distance ball H around it, which contains the current optimal sampling point v′ opt The Hamming of all candidate sampling points does not exceed the set threshold, and the first e candidate sampling points with the largest expected improvement EI are selected from H to form the initial candidate sampling point set S:

[0157] S=[v′ i1 ,v′ i2 ,…,v′ ie ] (13)

[0158] Among them, v′ i1 Indicates that the position index of the i-th candidate acquisition point in S is 1.

[0159] The calculation method of Hamming distance is:

[0160]

[0161] Among them, v1 and v2 represent the optimal sampling point v′ respectively. opt The corresponding CIIV and CIIV corresponding to other candidate sampling points, v 1,i and v 2,i Indicates the element at the corresponding position. For example, suppose there is a v′ opt =(1, 0, 1, 0, 0, 1), construct a distance ball H with a threshold of 2, which contains all distances between v′ and opt The candidate sampling points whose Hamming distance is less than or equal to 2. The candidate sampling points include: (1, 0, 0, 0, 0, 1), (1, 1, 1, 0, 0, 0), (1, 0, 1, 0, 1, 1).

[0162] Maximum expected improvement (EI):

[0163]

[0164] Where v′∈T v Indicates T v The sampling points v′,y in * Represents the current optimal objective function value, y * is max-L margin (x′ i ); in the generation task, y * is max-L BLEU (x′ i ), Represents the objective function value calculated by the surrogate model M The predicted distribution of max(y * -y,0) is the measure of improvement, which indicates the improvement value that the current sampling point v′ may bring.

[0165] S302: For S, use formula (11) to calculate the similarity matrix It is used to quantify the similarity between candidate sampling points, that is, to measure the proximity of two candidate sampling points in the search space. Represents the candidate sampling point v i ′ and v j The similarity matrix is ​​calculated based on the CIIV of the candidate sampling points.

[0166] S303: After calculating the similarity matrix, use the matrix to perform diversity sampling. Specifically, calculate the similarity matrix between a candidate sampling point in S and other candidate sampling points, and use the similarity matrix to select the candidate sampling point with the maximum dissimilarity gain [using the similarity matrix to select the candidate sampling point with the maximum dissimilarity gain is to select the candidate sampling point that can maximize the diversity gain]. This process uses a greedy strategy to iteratively select the optimal candidate identifier index vector v * , to maximize the total dissimilarity gain of S. In each iteration, candidate sampling points that maximize the diversity gain are selected until S reaches the set maximum capacity. This process is implemented by the following objective function:

[0167]

[0168] Among them, arg max represents the independent variable that makes a function reach its maximum value, v′∈T v \S means outside S, from T v Select a new candidate vector v′ from the matrix, det represents the determinant operation, and Represent the posterior covariance matrix of the current S and the updated candidate sampling point set S∪v′ respectively.

[0169] In each sampling iteration, candidate sampling points that have already been evaluated in previous iterations are excluded. This strategy effectively avoids redundant evaluations, reduces unnecessary computation, and improves sampling efficiency. In this way, subsequent sampling can focus on newly generated candidate sampling points, avoiding repeated computations and thus accelerating the optimization process.

[0170] S304: Input the candidate sampling points in S into the target model for evaluation. During the evaluation process, the objective function value of each candidate sampling point is recorded, and the posterior distribution of the proxy model is updated, and the process returns to S302. The proxy model is no longer updated until the set budget limit is reached [until the set budget limit is reached or the feedback information of the attack failure is no longer valid, that is, when the update of the proxy model no longer significantly improves after several rounds of iterations, the proxy model is no longer updated], and the adversarial example selected in each iteration for the successful attack is output. In this way, the sampling results will directly affect the objective function value in the feedback optimization generation component, thus forming a closed-loop feedback loop, continuously improving the efficiency and success rate of adversarial example generation, and ultimately efficiently generating effective adversarial examples.

[0171] Experiment and analysis:

[0172] The present invention selects a variety of advanced identifier replacement and adversarial example generation methods to verify the advantages of the present invention in terms of both generation quantity and quality, which can be summarized into three categories: identifier replacement standard, optimized generation standard and diversity enhancement standard.

[0173] 1. Identifier replacement criteria: The identifier replacement criteria builds a comprehensive search space by generating semantically preserved code variants:

[0174] (1) Identifier replacement screens the candidate set based on syntactic correctness and semantic consistency to ensure that the generated code variants can be compiled and executed.

[0175] (2) Using coverage as an evaluation indicator, the coverage ratio of the search space is expanded as much as possible by replacing different identifiers. In the present invention, the coverage ratio is set to no less than 80%.

[0176] 2. Optimizing the Generation Criteria: The optimized generation criteria dynamically adjusts the generation direction through Bayesian optimization to guide the generation of more effective adversarial examples:

[0177] (1) Use a feedback mechanism to record successful and failed generation information to optimize the next round of adversarial example generation.

[0178] (2) Guided sampling is used to select the code variants that are most likely to deceive the target model to improve generation efficiency and attack success rate.

[0179] 3. Diversity Enhancement Criteria: The diversity enhancement criteria screens adversarial examples with wide coverage through the Determinant Point Process (DPP):

[0180] (1) Use the similarity matrix to evaluate the differences between candidate code variants.

[0181] (2) By maximizing diversity, candidate samples with the widest coverage are selected to avoid the generated examples from falling into local optimality.

[0182] Select Model:

[0183] The models used in this paper are CodeBERT, GraphCodeBERT, CodeT5, PLBART, and CodeGPT, and the tasks selected are vulnerability prediction, clone detection, and code summarization. The vulnerability prediction task uses a dataset containing code samples extracted from two C language projects to predict whether a code snippet contains vulnerabilities; the clone detection task uses the BigCloneBench dataset to detect the semantic equivalence of two code snippets; and the code summarization task uses the Java subset of CodeSearchNet to generate a natural language summary describing the functionality of the code snippet.

[0184] CodeBERT is a Transformer-based deep code model that focuses on code understanding tasks such as vulnerability prediction, code clone detection, and code classification. GraphCodeBERT introduces a graph neural network based on CodeBERT to capture the grammatical structure and dependency information of the code, which is suitable for more complex code semantic understanding tasks. CodeT5, based on the T5 architecture, is a powerful code generation and translation model that supports generation tasks from natural language to code and translation between programming languages. PLBART focuses on translation tasks from programming language to programming language, combining a pre-training framework for text and code. CodeGPT is a code generation model based on the GPT architecture that can efficiently handle code completion and generation tasks and is suitable for generation scenarios in multiple programming languages.

[0185] All selected models have been well pre-trained and fine-tuned to effectively handle their respective target tasks. These models and datasets are widely used in various deep coding model tasks, making them representative and widely adopted in previous research works.

[0186] Evaluation indicators:

[0187] This paper adopts three evaluation indicators: attack success rate (ASR), average model invocation number (AMI) and robustness analysis after adversarial fine-tuning.

[0188] Attack Success Rate (ASR): When the generated adversarial examples successfully mislead the victim model, the attack success rate increases. ASR is a direct evaluation metric that reflects the ability of the generated adversarial examples to change the model's prediction results in the target task. A higher ASR indicates that the attack method is more effective in testing the vulnerabilities of deep learning models.

[0189] Average Model Invocation (AMI): AMI measures the average number of model invocations required to generate adversarial examples and is an important metric for evaluating attack efficiency. In black-box scenarios, model invocations are often limited by time and cost. A reduction in AMI indicates a more cost-effective process for generating adversarial examples.

[0190] Robustness analysis after adversarial fine-tuning: This paper evaluates the robustness of the victim model through adversarial fine-tuning, specifically including the following:

[0191] Adversarial examples generated by ITGen, ALERT, and BeamAttack are used to fine-tune the model and evaluate its performance on vulnerability prediction and code summarization tasks.

[0192] The fine-tuned victim model achieves significantly improved accuracy in vulnerability prediction tasks and significantly improves the BLEU score in code summarization tasks.

[0193] The results show that ITGen's adversarial examples have a significant effect in enhancing model robustness and improving test coverage.

[0194] By combining the above evaluation indicators, the present invention can comprehensively measure the effectiveness and efficiency of the adversarial example generation method, while verifying the potential of adversarial examples in enhancing model robustness, providing comprehensive support for the security and robustness testing of deep code models.

[0195] Diversification rules:

[0196] The present invention adopts three main mutation rules to generate adversarial examples. These rules are identifier replacement, identifier permutation and reorganization, and code fragment adjustment, which are used to generate diverse mutant code examples.

[0197] Identifier substitution: Adversarial examples are generated by replacing identifiers in the code (such as variable and function names). The paper mentions that identifier substitution generates a set of candidate replacements based on context and mutates using combinations in the search space. For example, replacing the identifier "search" with "lookup" or "find" can generate diverse variants while maintaining semantic consistency.

[0198] Identifier permutation: This method adjusts the order of identifiers to explore the model's sensitivity to input order. By changing the order of identifiers into logically equivalent but different combinations, code variants are generated. This method is suitable for detecting whether the model misinterprets illogical changes in code input.

[0199] Code snippet adjustments: Generate mutated code by adding, deleting, or moving code snippets, such as inserting empty statements or adjusting the order of code blocks. These mutations do not change the semantics or logical functionality of the code, but can effectively increase input diversity and test the model's robustness to non-functional changes.

[0200] It is worth noting that the paper mentions that in the construction of the search space and the application of mutation rules, the generated code is always ensured to maintain syntactic and semantic consistency to avoid the failure of adversarial examples. At the same time, the determinant point process (DPP) is used to screen diverse mutation examples to further improve coverage and test effectiveness.

[0201] The results of the present invention are shown in Table 1. AL, Beam, and IT represent ALERT, BeamAttack, and ITGen, respectively.

[0202] Table 1 Test results

[0203]

[0204]

[0205] Observing the results in Table 1, it is worth noting that it is a challenging task to comprehensively optimize the attack success rate (ASR) and average model invocation (AMI) in the three tasks of vulnerability prediction, clone detection, and code summarization.

[0206] In the vulnerability prediction task, ITGen demonstrated a significant advantage, with its ASR being higher than ALERT and BeamAttack across all models. For example, on the CodeT5 model, ITGen achieved an ASR of 96.25%, significantly exceeding ALERT (83.21%) and BeamAttack (81.55%). This result demonstrates that ITGen is more capable of capturing key vulnerability signatures in code. However, because vulnerability prediction involves complex code semantic analysis, even ITGen requires a certain number of model calls (e.g., an AMI of 0.04 for CodeT5) to complete adversarial example generation.

[0207] In the clone detection task, ITGen's ASR also significantly outperforms the baseline method. For example, on the GraphCodeBERT model, ITGen's ASR is 33.68%, significantly higher than ALERT (12.92%) and BeamAttack (13.62%). However, the clone detection task involves semantic similarity analysis between code pairs, so the process of generating adversarial examples is more complicated, resulting in a higher AMI, such as 0.15 for GraphCodeBERT.

[0208] In code summarization tasks, ITGen performs well in both ASR and AMI. For example, on the PLBART model, ITGen achieves an ASR of 87.17%, significantly higher than ALERT (68.05%) and BeamAttack (57.93%). However, due to the natural language requirements of code summarization tasks, ITGen requires a more complex adversarial example generation strategy, resulting in a slightly higher AMI, but still significantly lower than other methods (such as BeamAttack).

[0209] Furthermore, ITGen excels at enhancing the robustness of victim models. Through adversarial fine-tuning, ITGen significantly reduces the attack success rate and improves the model's robustness to new adversarial examples. For example, in the vulnerability prediction task, after fine-tuning the CodeBERT model with ITGen, its ASR performance decreased from the original 65.15% to 58.92%.

[0210] When using our ITGen as a guide for adversarial example generation, the adversarial examples cover most key scenarios in vulnerability prediction, clone detection, and code summarization tasks. This highlights the effectiveness of our approach in generating diverse and representative adversarial examples that can trigger various vulnerabilities and model flaws across different tasks.

[0211] In vulnerability prediction tasks, adversarial examples generated by ITGen are able to cover a wider range of vulnerability patterns. In particular, on the CodeT5 model, ITGen's example coverage significantly outperforms other methods. This demonstrates that our method can effectively exploit the potential flaws of deep code models when handling complex vulnerability detection tasks.

[0212] In the clone detection task, the adversarial examples generated by our method exhibit a wider range of semantic variations and code mutations, enabling ITGen to significantly improve the model's ability to trigger errors. Although the clone detection task is relatively semantically sensitive, the diverse examples generated by ITGen effectively test the model's robustness in judging code similarity.

[0213] In the code summarization task, adversarial examples generated by ITGen affect the quality of the natural language summaries generated by the model through subtle code mutations, demonstrating strong coverage and challenge. Compared with baseline methods, the adversarial examples generated by our method are more able to highlight the model's potential weaknesses in language generation tasks.

[0214] Although the generation efficiency is slightly lower than other methods in some tasks, the adversarial examples generated by ITGen perform well in terms of diversity, fairness, and coverage. This shows that the present invention has the potential to identify a wide range of defects in deep coding models in real-world tasks, especially when applied to more complex and rich datasets.

[0215] In short, this invention not only generates diverse and more efficient adversarial examples, but also identifies potential vulnerabilities and deficiencies in the model. These adversarial examples can provide valuable insights for improving the robustness and reliability of deep learning models in real-world applications.

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for iteratively generating adversarial samples for large models, characterized by: The steps include: S100: Construct candidate identifier index vector search space T v , obtain the original code, extract the identifier set I from the original code, generate a candidate replacement set for each identifier in the identifier set I, and use the candidate replacement set to construct the identifier search space T c , T c Each identifier combination in is mapped to a candidate identifier index vector CIIV, and CIIV is used to construct T v ; S200: Constructing a historical failed attack set through random sampling and historical records and use Train the Gaussian process GP as a proxy model and use the expected improvement acquisition function to select the optimal sampling point v′ opt , and according to v′ opt The optimal code variants sampled correspond to Update the evaluation results And continue to train the proxy model until preliminary adversarial examples are generated; S300: The current optimal sampling point v′ obtained according to S200 opt , select all candidate sampling points whose Hamming distance does not exceed the set threshold to build a distance sphere H, and select e candidate sampling points from H to form the initial candidate sampling point set S; v The candidate sampling points that can maximize the diversity gain are selected until S reaches the set maximum capacity; the candidate sampling points in S are input into the target model for evaluation, and the posterior distribution of the proxy model is updated according to the objective function value until the set budget limit is reached. The proxy model is no longer updated and the adversarial examples of successful attacks selected in each iteration are output.

2. The iterative generation method for adversarial samples for large models according to claim 1, characterized in that: In S100, a candidate identifier index vector search space T is constructed. v The steps are as follows: S101: Map a piece of original code to an identifier set, and extract the identifier set from the original code S102: Generate a candidate replacement set for each identifier in the identifier set I, ensuring that the replacement item complies with grammatical rules and maintains semantic consistency of the code; the candidate replacement set is a set of candidate identifiers derived from an open source test dataset, and is used to replace the identifier in the original code; S103: Use the candidate replacement set to construct the identifier search space T c , a set of identifiers corresponds to a T c , T c The elements in are called identifier combinations; a piece of code obtained by replacing the identifiers with the same meaning in the original code with an identifier combination is called a code variant; an identifier combination represents a code variant: Where l represents the number of identifiers, C i represents the candidate replacement set for the i-th identifier; S104: T c Each identifier combination in is mapped to a candidate identifier index vector CIIV, where each element in CIIV represents the index of an identifier in the identifier combination in the candidate replacement set. The CIIV of each identifier combination is expressed as: where v i represents the index of the i-th identifier in its candidate replacement set; Then use the CIIV of each identifier combination to construct the candidate identifier index vector search space T v : in, A set of CIIVs representing all combinations of identifiers.

3. The iterative generation method for adversarial samples for large models according to claim 1, characterized in that: The process of obtaining the initial generation of adversarial examples is as follows: S201: First, use the original code to attack the deep code model. If the attack is successful, a valid adversarial example is obtained and defined as the initial generated adversarial example output; If the attack fails, a Bayesian optimization framework is used to mine the evaluation information of the failed attack to find a code variant x′ that is mapped to the optimal attack vector and has the potential to successfully conduct the adversarial attack; Code understanding task: Define a function that maximizes the negative margin to measure the difference in prediction confidence of the attacked deep code model for the category, where the attacked deep code model is called the target model: Where Y represents the set of all possible categories of the target model, and Represents classes respectively and real class The predicted probability of ; Code generation task: The code variants that undergo adversarial attacks are defined as adversarial examples, and the maximum negative BLEU score is defined to measure the semantic similarity between the adversarial example and the original code: Among them, L BLEU (x′) is the BLEU score of x′; S202: Construct a historical failed attack set through random sampling and historical records From T c Randomly sample n identifier combinations, replace the identifiers in each identifier combination with the identifiers with corresponding meanings in the original code, and obtain code variants. The formula for determining the number of samples n is: in, represents the total number of identifiers in the original code, and N represents the total number of identifiers and their candidate sets; The code variants obtained by random sampling and replacement are input into the target model for evaluation, and the objective function value output after each code variant is input into the target model is recorded. margin (x i ′) and -L BLEU (x i ′); For the comprehension task, if -L margin (x i ′)>0, it means that the code variant is a successful attack, otherwise it means that the code variant is a failed attack; For build tasks, if -L BLEU (x i ′)=0, it means that the code variant is a successful attack, otherwise it means that the code variant is a failed attack; Use the CIIV corresponding to the code variants that failed the attack and their corresponding objective function values ​​to construct the historical failed attack set Among them, v′ i CIIV,x′ corresponding to the code variant that indicates the failed attack i Code variant indicating a failed attack, n represents the number of samples; S203: Use Train the Gaussian process GP as the proxy model, and the posterior distribution of the proxy model is expressed as: μ=k(v′1,X)(k(X,X)+τ 2 I′) -1 Y (9) ∑=k(v′1,v′2)-k(v′1,X)(k(X,X)+τ 2 I′) -1 k(X,v′2) (10) in, is a normal distribution, v′1 and v′2 are two different CIIVs, and X is The matrix composed of CIIV, μ is the mean function, ∑ is the covariance matrix, τ 2 is the noise variance, I′ is the identity matrix used for regularization, k(v′1,v′2) is the kernel function, k(v′1,X) is the kernel function matrix, and k(X,X) is the kernel matrix; The kernel function k(v′1,v′2) is: where δ(·) is the Kronecker delta function, l is the total number of identifiers in the code, represents the dimension of CIIV, and w 1i and w 2i Represents the i-th element in the two CIIV vectors, which is the identifier replacement corresponding to the two different code variants. is the signal variance, β i is the length scale parameter corresponding to the correlation of the i-th element; S204: Based on the trained proxy model, use the expected improved acquisition function to select the optimal sampling point v′ opt ,The calculation formula of expected improved acquisition function is: where v′ opt represents the optimal sampling point, v′∈T v Indicates T v The sampling points v′,y in * Represents the current optimal objective function value; in the classification task, y * is max-L margin (x i ′); in the generation task, y * is max-L BIEU (x i ′), Represents the objective function value calculated by the surrogate model M The predicted distribution of max(y * -y,0) is the measure of improvement, which indicates the improvement value that the current sampling point v′ may bring; S205: According to v′ opt The optimal code variants sampled correspond to The evaluation results of execution: For the comprehension task, when -L margin (x i ′)>0, then stop the iteration and return the adversarial example x′ opt , and the adversarial example x′ opt Defined as the initial generation of adversarial example output; otherwise, the current sampling result and L margin (x i ')join in and update And return to S203; For build tasks, when -L BIEU (x i ′)=0, then stop the iteration and return the adversarial example x′ opt , and the adversarial example x′ opt Defined as the initial generation of adversarial example output; otherwise, the current sampling result and L BLEU (x i ')join in and update And return to S203.

4. The iterative generation method for adversarial samples for large models according to claim 3, characterized in that: In S300, the process of outputting the successful adversarial examples selected in each round of iteration is as follows: S301: Constructing a set of candidate sampling points: Calculating the current optimal sampling point v′ opt The Hamming distance with other candidate sampling points is based on the calculated Hamming distance at the current optimal sampling point v′ opt Construct a distance ball H around it, which contains the current optimal sampling point v′ opt The Hamming of all candidate sampling points does not exceed the set threshold, and the first e candidate sampling points with the largest expected improvement EI are selected from H to form the initial candidate sampling point set S: S=[v′ i1 ,v′ i2 ,…,v′ ie ] (13) Among them, v′ i1 Indicates that the position index of the i-th candidate acquisition point in S is 1; The calculation method of Hamming distance is: Among them, v1 and v2 represent the optimal sampling point v′ respectively. opt The corresponding CIIV and CIIV corresponding to other candidate sampling points, v 1,i and v 2,i Indicates the element at the corresponding position; Maximum expected improvement (EI): Where v′∈T v Indicates T v The sampling points v′,y in * Represents the current optimal objective function value, y * is max-L margin (x′ i ); in the generation task, y * is max-L BLEU (x′ i ), Represents the objective function value calculated by the surrogate model M The predicted distribution of max(y * -y,0) is the measure of improvement, which indicates the improvement value that the current sampling point v′ may bring; S302: For S, use formula (11) to calculate the similarity matrix Used to quantify the similarity between candidate sampling points, Represents the candidate sampling point v i ′ and v j ′, the similarity matrix is ​​calculated based on the CIIV of the candidate sampling points; S303: Calculate the similarity matrix between a candidate sampling point in S and other candidate sampling points, and use the similarity matrix to select the candidate sampling point with the largest dissimilarity gain. This process uses a greedy strategy to iteratively select the optimal candidate identifier index vector v * , to maximize the total dissimilarity gain of S; in each iteration, the candidate sampling point that can maximize the diversity gain is selected until S reaches the set maximum capacity; this process is achieved through the following objective function: Among them, arg max represents the independent variable that makes a function reach its maximum value, v′∈T v \S means outside S, from T v Select a new candidate vector v′ from the matrix, det represents the determinant operation, and Represent the posterior covariance matrix of the current S and the updated candidate sampling point set S∪v′ respectively; In each sampling iteration, candidate sampling points that have been evaluated in previous iterations are excluded; S304: Input the candidate sampling points in S into the target model for evaluation; during the evaluation process, record the objective function value of each candidate sampling point, and update the posterior distribution of the proxy model, and return to S302 until the set budget limit is reached. The proxy model no longer updates and outputs the successful adversarial examples selected in each iteration.

Images