Cross-domain antagonism attack method based on pre-training code model

Abstract

The invention discloses a cross-domain antagonism attack method based on a pre-training code model, and aims to overcome the defects of an existing method in challenges such as data difference and model feedback deficiency. The method comprises the following steps that source code data are obtained, the source code data are traversed to identify a basic constituent unit token of the source code data, and the token at least comprises an identifier and an operator; generating a natural replacement identifier set for the extracted identifiers based on a mask language model; applying equivalent transformation of a token level to preliminarily optimize a countermeasure example; and carrying out further conversion on the preliminarily optimized adversarial example by applying equivalent structure transformation to obtain a final adversarial example. According to the method, a natural replacement identifier is generated through structure and token-level disturbance in combination with a mask language model, and the migration ability and attack effect on resistance examples are enhanced in combination with greedy search and gradient optimization.

Description

Technical Field

[0001] The present invention belongs to the technical field of adversarial attacks, and specifically relates to a cross-domain adversarial attack method based on a pre-trained code model. Background Art

[0002] In recent years, pre-trained code models have been widely used in source code tasks, such as vulnerability detection and code author attribution, usually using the "pre-training-fine-tuning" paradigm: self-supervised pre-training on large-scale datasets, followed by supervised fine-tuning on downstream tasks. Although these fine-tuned pre-trained code models perform well in tasks, they are vulnerable to adversarial example attacks. Adversarial examples mislead model predictions through small perturbations, bringing security risks, such as bypassing code copyright authentication and vulnerability detection tools. Existing adversarial attack methods are divided into two environments: same-domain and cross-domain. In the same-domain environment, attack methods include transfer-based and query-based attacks. Transfer-based attacks assume that the attacker has access to the training data of the victim model and uses this data to generate adversarial examples; query-based attacks assume that the attacker can obtain feedback information from the victim model and generate adversarial examples through query results. However, it is difficult to obtain training data or model feedback in practice, which limits their application effect. In contrast, attacks in cross-domain environments are more challenging because attackers can only access open-source pre-trained models and cannot obtain training data or model feedback in the target domain, that is, they cannot obtain fine-tuned pre-trained code models. In this case, attackers need to generate adversarial examples based on limited test samples. In order to improve the adversarial effect, it is usually adopted to maximize the feature difference between adversarial examples and clean examples, and combine greedy algorithms to optimize the generation of adversarial examples to improve its attack capability and concealment.

[0003] Existing cross-domain adversarial attack methods can successfully attack fine-tuned pre-trained code models to a certain extent, but there are still some shortcomings and limitations when dealing with special challenges between different domains. First, these methods usually only perform simple transformations on identifiers (such as variable names and function names), resulting in insufficient diversity of adversarial examples. This makes these attack examples easier to detect and defend when they migrate between different domains, thereby reducing the effectiveness and concealment of the attack. Second, most current methods rely on the loss value of the embedding vector to generate adversarial examples, without making full use of gradient information to optimize the generation process, which limits the effectiveness of these examples in different domains and the success rate of attacks. In addition, existing methods mainly use genetic algorithms when generating alternative identifiers. Although some transformation effects can be produced, the generated identifiers are often not natural enough and do not conform to the context of the code or common programming habits. This unnatural code may cause some problems, such as affecting the readability of the code or introducing potential logical errors, reducing the concealment and success rate of the attack. Moreover, identifiers that do not conform to habits are more easily identified by the detection system during cross-domain migration, increasing the risk of attack failure. Overall, existing methods still have room for improvement in generating more diverse, natural and efficient adversarial examples. In order to improve the attack effect in actual cross-domain applications, we need to pay more attention to the naturalness and contextual consistency of adversarial examples, optimize the generation process, and explore more effective strategies to cope with the complex changes between different domains. Summary of the invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide a cross-domain adversarial attack method based on a pre-trained code model. By introducing perturbations at the equivalent structure level and token level, the masked language prediction function of the model is used to generate natural alternative identifiers, and greedy search and gradient optimization strategies are combined to improve the applicability and attack effect of adversarial examples in different fields.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a cross-domain adversarial attack method based on a pre-trained code model, comprising the following steps:

[0007] Acquire source code data, traverse the source code data to identify its basic constituent unit token, wherein the token includes at least an identifier and an operator;

[0008] Generate a set of natural alternative identifiers for the extracted identifiers based on a masked language model;

[0009] Apply token-level equivalent transformations to preliminarily optimize adversarial examples;

[0010] The initially optimized adversarial examples are further transformed by applying equivalent structural transformation to obtain the final adversarial examples.

[0011] As a preferred technical solution, the source code data is obtained, and the source code data is traversed to identify its basic constituent unit token, specifically:

[0012] Receive source code sets from different downstream task datasets, preprocess each source code, remove single-line comments, multi-line comments, document comments, and blank lines to reduce interference from irrelevant information;

[0013] The lexical analyzer traverses the source code character by character to identify its basic building block token;

[0014] Record the occurrence and frequency of identifiers, operators and control structures to provide a basis for subsequent semantically preserved code conversion.

[0015] As a preferred technical solution, the method of generating a natural set of alternative identifiers for the extracted identifiers based on the masked language model is specifically as follows:

[0016] Perform mask operations on all identifiers in each source code in turn;

[0017] Input the masked source code into a preset masked language model and make predictions based on context information to generate multiple potential alternative identifiers;

[0018] The generated alternative identifiers are screened so that the screened alternative identifiers meet the semantic requirements of programming specifications and contexts;

[0019] A set of natural alternative identifiers is constructed based on the screened alternative identifiers for the subsequent adversarial example generation process.

[0020] As a preferred technical solution, the application of equivalent structural transformation to generate preliminary adversarial examples is specifically as follows:

[0021] Calculate the importance of identifiers: replace each identified identifier with a placeholder to simulate its absence; then input the original code and the code containing the placeholder into the pre-trained code model to obtain the first feature vector and the second feature vector respectively; characterize the importance of the identifier by calculating the L2 norm of the first feature vector and the second feature vector; sort all identifiers by importance score;

[0022] Replace identifiers and use gradient information to optimize selection: replace the sorted identifiers in turn and decide whether to replace them based on their gradient information. Specifically, for each identifier, generate K replacement identifiers as a replacement set I, replace the original identifier with the K replacement identifiers in the set, and obtain K replaced code examples x'; find the optimal replacement identifier by calculating the difference between x' and the original code example x based on the gradient information; if the optimal replacement identifier is not found, do not replace it and continue the next round of iteration to determine whether the next identifier should be replaced;

[0023] Use gradient information to calculate the optimal replacement: For each replaced identifier code example x_sub, calculate the gradient of the example relative to the model loss function And through the symbolic function Extract the direction information of the gradient; multiply the sign function by a small perturbation coefficient ∈ to generate a perturbation And apply the perturbation to the original embedding representation x_orig to obtain the adversarial input Predict the logits output f(x_adv) and predicted label of the adversarial input x_adv through the pre-trained code model; if the maximum logits value drops significantly, it indicates that the adversarial example is effective, and the maximum logits value is the classification confidence of the pre-trained code model; by evaluating the impact of each alternative identifier on the model, select the identifier that makes the confidence of the pre-trained code model drop to the lowest as the optimal replacement solution; by replacing and evaluating the identifiers one by one, finally form a preliminary adversarial example;

[0024] Operator-level replacement optimization: After obtaining the preliminary adversarial example x_adv, operator-level replacement is further performed. Specifically, based on the number of occurrences and position information of the operator in the source code, it is decided in the form of probability whether to perform operator equivalent replacement; after operator-level replacement, the adversarial example after token-level transformation is further optimized.

[0025] As a preferred technical solution, the importance of the identifier is calculated by the following formula:

[0026]

[0027] Among them, ||.|| 2 2 It represents the L2 norm of the square, which is used to measure the change of the code feature after the placeholder is inserted. F(x) represents the first feature vector, and F(x') represents the second feature vector. OSI represents the importance of the identifier. The larger the OSI value, the greater the influence of the identifier on the code feature and the higher its importance.

[0028] As a preferred technical solution, the application of equivalent structural transformation further transforms the initially optimized adversarial example to obtain the final adversarial example, specifically:

[0029] Branch structure replacement: Analyze the source code structure, identify the parts that can be equivalently transformed, and decide whether to replace them in a probabilistic manner based on the number of times the branch structure appears in the code. These replacements can evade detection tools based on static analysis and generate more hidden adversarial examples.

[0030] Insertion of redundant code: Sort the positions of each line of code according to their importance, and determine whether to insert redundant code in turn, and decide the necessity of insertion based on random probability; the insertion of redundant code further increases the complexity of adversarial examples, making them more difficult to detect.

[0031] As a preferred technical solution, the step of evaluating the transfer ability of the optimized adversarial example is also included, specifically:

[0032] Generate adversarial examples using pre-trained code models on a variety of tasks and datasets, and then apply these examples to the pre-trained code model fine-tuned on the same task dataset;

[0033] By comparing the attack success rate and stability of these fine-tuned pre-trained code models on adversarial examples, we analyze the transfer effect and robustness of examples in different fine-tuned pre-trained code models and tasks.

[0034] If these examples can effectively reduce the prediction accuracy of the fine-tuned pre-trained code model in multiple environments, it means that these examples have good cross-domain migration capabilities. Based on the evaluation results, the generation strategy is further optimized to improve the applicability and attack effect of adversarial examples.

[0035] In a second aspect, the present invention further provides a cross-domain adversarial attack system based on a pre-trained code model, which is applied to the cross-domain adversarial attack method based on a pre-trained code model, and includes a source code data acquisition module, an alternative identifier generation module, a first equivalent transformation module, and a second equivalent transformation module;

[0036] The source code data acquisition module is used to acquire source code data, traverse the source code data to identify its basic constituent unit token, wherein the token includes at least an identifier and an operator;

[0037] The alternative identifier generation module is used to generate a natural alternative identifier set for the extracted identifier based on the masked language model;

[0038] The first equivalent transformation module applies token-level equivalent transformation to preliminarily optimize adversarial examples;

[0039] The second equivalent transformation module applies an equivalent structural transformation to further transform the initially optimized adversarial example to obtain a final adversarial example.

[0040] In a third aspect, the present invention further provides an electronic device, characterized in that the electronic device comprises:

[0041] at least one processor; and,

[0042] a memory communicatively connected to the at least one processor; wherein,

[0043] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the cross-domain adversarial attack method based on the pre-trained code model.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the cross-domain adversarial attack method based on a pre-trained code model.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] 1. Improve the diversity and concealment of adversarial examples: By introducing changes at different levels in the code (such as changes in code structure and basic elements (Tokens)), the possible range of adversarial attacks is expanded, making the generated attack examples more diverse and complex between different fields. Such diversity helps to evade the defense systems unique to each field and is more difficult to detect, thereby improving the concealment and effectiveness of the attack.

[0047] 2. Improve the efficiency and effectiveness of adversarial example generation: This invention optimizes the generation process by making full use of the gradient information of the code, rather than relying solely on traditional methods. This approach can generate adversarial examples more accurately, improve the success rate of attacks on victim models and the misleading effect, and greatly reduce the time and computing resources required for generation in different fields.

[0048] 3. Generate natural and effective alternative identifiers: This invention uses the masked language prediction function of the model to generate alternative identifiers that appear natural and reasonable in both the source and target domains, in line with programming habits and contextual consistency, thereby enhancing the concealment and practicality of adversarial examples in cross-domain attacks. At the same time, more suitable alternative identifiers are quickly found through greedy search, and the selection is further optimized to improve the efficiency of generation and the accuracy of the attack.

[0049] 4. Enhance the applicability of adversarial attacks in multiple fields and the robustness of models: The method of the present invention can not only be used for source code classification tasks, but also effectively apply adversarial examples in tasks in different fields, expanding the scope of use. The generated more natural and adaptable adversarial examples help enhance the stability of the model, making it more resistant to cross-domain adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 This is a flow chart of a cross-domain adversarial attack method based on a pre-trained code model according to an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a process for generating a final adversarial example in an embodiment of the present invention;

[0053] Figure 3 This is a block diagram of a cross-domain adversarial attack system based on a pre-trained code model according to an embodiment of the present invention.

[0054] Figure 4 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0056] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0057] The present invention aims to solve the problem of how to effectively generate adversarial samples and attack source code models when it is impossible to obtain data or model feedback in the target domain, thereby improving the applicability of adversarial attacks in different fields. Current source code classification models are usually self-supervised pre-trained in the source domain and then supervised fine-tuned in the target domain. Although they can perform well in migration tasks, they are easily misled by adversarial examples, which are generally generated by transformations that keep the code semantics unchanged. Existing attack methods for generating adversarial examples mostly rely on query feedback on the target model, which is difficult to implement in practical applications. To this end, the present invention proposes a cross-domain adversarial attack method based on a pre-trained code model, which uses an open source pre-trained code model and limited test samples to effectively attack code models in unknown domains without relying on target domain data.

[0058] See also Figure 1 This embodiment provides a cross-domain adversarial attack method based on a pre-trained code model, which generates natural replacement identifiers through structural and token-level perturbations and masked language, and combines greedy search and gradient optimization to enhance the migration ability and attack effect of adversarial examples, including the following steps:

[0059] S1. Obtain source code data, traverse the source code data and identify its basic constituent unit token;

[0060] Furthermore, step S1 is specifically as follows:

[0061] S11. First, receive a set of source code from different downstream task datasets, preprocess each source code, remove single-line comments, multi-line comments, document comments, and blank lines to reduce interference from irrelevant information, thereby ensuring the accuracy and naturalness of the subsequent generation of adversarial examples.

[0062] S12, through the lexical analyzer, traverse the source code character by character, identify its basic building blocks (tokens), including keywords, identifiers, operator separators and control constants, etc., in this embodiment, specifically using identifiers and operators. At the same time, the system will accurately record the occurrence position and frequency of identifiers, operators and control structures (such as if, while, etc.), providing a basis for subsequent semantically preserved code conversion.

[0063] S2, generating a set of natural alternative identifiers for the extracted identifiers based on the masked language model;

[0064] Furthermore, step S2 is specifically as follows:

[0065] S21, performing mask operations on all identifiers in each source code in turn;

[0066] S22, inputting the masked code into a preset masked language model, and performing prediction based on context information, thereby generating multiple potential alternative identifiers;

[0067] S23, screening the generated alternative identifier to ensure that it meets the semantic requirements of the programming specification and context;

[0068] S24. Construct a set of natural replacement identifiers for the subsequent adversarial example generation process.

[0069] In a specific example, the set of alternative identifiers for different identifiers in each source code can be represented as I i,j Where i is the i-th identifier, j is the j-th alternative identifier of the i-th identifier, and I is the set of all alternative identifiers, that is: I = {I i，j |i∈{1,...,N}, j∈{1,...,Q i}};

[0070] Where N is the total number of identifiers in the code, Q i is the number of alternative identifiers corresponding to the i-th identifier.

[0071] S3. Apply token-level equivalent transformation to preliminarily optimize adversarial examples. Token-level equivalent substitution focuses on the basic elements of the code (such as variable names, operators, etc.). The specific steps are as follows:

[0072] S31. Calculate the importance of the identifier. The importance of the identifier is measured by calculating its impact on the code characteristics.

[0073] For each identifier in the set of identifiers extracted in step S1, replace it with a placeholder to simulate its absence. Then, the original code x and the code x′ containing the placeholder are input into the pre-trained code model M to obtain their first feature vector F(x) and second feature vector F(x′), respectively. The importance of the identifier (OSI) is calculated by the following formula:

[0074]

[0075] where ||.|| 2 2 Represents the squared L2 norm (Euclidean distance), which is used to measure the change in code features after inserting a placeholder; the larger the OSI value, the greater the impact of the identifier on the code features and the higher its importance. Sort all identifiers by their importance scores.

[0076] S32. Replace identifiers and use gradient information to optimize selection:

[0077] Replace the sorted identifiers in turn, and decide whether to replace them based on their gradient information. For each identifier, generate K replacement identifiers as the replacement set I, replace the original identifier with the K replacement identifiers in the set, and obtain K replaced code examples x'. Find the optimal replacement identifier by calculating the difference between x' and x based on the gradient information. If the optimal replacement identifier is not found, no replacement is performed, and the next round of iteration is continued to determine whether the next identifier should be replaced.

[0078] S33, using gradient information to calculate the optimal replacement:

[0079] For each replaced identifier code example x_sub, calculate the gradient of the example relative to the loss function of the pre-trained code model And through the symbolic function Extract the direction information of the gradient (+1 means increase, -1 means decrease of the corresponding eigenvalue). Multiply this sign information by a small perturbation coefficient ∈ to generate perturbation And apply it to the original embedding representation x_orig to get the adversarial input The pre-trained code model is used to predict the logits output f(x_adv) and predicted label of the adversarial input x_adv. If the maximum logits value (classification confidence of the model) drops significantly, it indicates that the adversarial example is effective. By evaluating the impact of each replacement identifier on the pre-trained code model, the identifier that reduces the confidence of the pre-trained code model to the lowest is selected as the optimal replacement solution. By replacing and evaluating each identifier one by one, a complete adversarial code is finally formed.

[0080] S34, Operator level replacement optimization:

[0081] After obtaining the preliminary adversarial example x_adv, we further perform operator-level replacement. In step S1, we record the number of occurrences and position information of the operators in the code, and decide whether to perform operator equivalent replacement (such as replacing ++ with +=) in the form of probability based on this information. After the operator-level replacement, the adversarial example after the token-level transformation is further optimized.

[0082] S4. Apply equivalent structural transformation to further transform the initially optimized adversarial example to obtain the final adversarial example.

[0083] Furthermore, step S4 is specifically as follows:

[0084] S41. Replacement of branch structure:

[0085] The system analyzes the source code structure and identifies parts that can be equivalently transformed (such as loop structures, conditional branches, etc.). It decides whether to replace the branch structure in the code in a probabilistic manner based on the number of times the branch structure appears (such as rewriting a for loop into an equivalent while loop, or converting a nested if-else into an equivalent switch-case statement). These transformations can evade detection tools based on static analysis and generate more hidden adversarial examples.

[0086] S42. Insertion of redundant code:

[0087] The importance of each line of code is sorted, and whether to insert redundant code is determined in turn, and the necessity of insertion is determined based on random probability. The insertion of redundant code further increases the complexity of adversarial examples, making them more difficult to detect.

[0088] After the above token-level and equivalent structure-level replacement and optimization steps, a fully optimized adversarial example is finally obtained.

[0089] It can be understood that the process of generating the final adversarial example is as follows Figure 2 As shown in Figure 1, the process starts with preprocessed source code examples and gradually optimizes the diversity and concealment of adversarial examples through a series of equivalent transformations at the token level and structure level.

[0090] Furthermore, in order to seriously generate adversarial examples, the following steps are also included:

[0091] S5. Evaluate the transferability of adversarial examples. Evaluating the transferability of adversarial examples is a key step in verifying that they remain effective attacks on different tasks and models. Specifically:

[0092] S51. Generate adversarial examples on a variety of tasks (e.g., code classification, code similarity detection) and datasets using a pre-trained code model, and then apply these examples to the pre-trained code model that has been fine-tuned on the same task dataset;

[0093] S52. By comparing the attack success rate and stability of these models on adversarial examples, analyze the transfer effect and robustness of the examples in different fine-tuned pre-trained code models and tasks. If these examples can effectively reduce the prediction accuracy of the fine-tuned pre-trained code model in multiple environments, it means that they have good cross-domain transfer capabilities;

[0094] S53. Based on these evaluation results, further optimize the generation strategy to improve the applicability and attack effectiveness of adversarial examples.

[0095] The core of the present invention is to develop a cross-domain adversarial attack scheme based on pre-trained code models, which can create effective adversarial examples to challenge source code models in the absence of target domain data or model responses. The innovative elements that this scheme aims to protect cover three main aspects: first, by introducing equivalent perturbations at the structural and tag levels, the exploration scope of adversarial attacks is expanded and the diversity of adversarial examples is enhanced; second, by using the masked language model, that is, the mask prediction function of the model to generate natural replacement identifiers, ensure that the code follows programming standards and maintains contextual consistency; finally, by integrating gradient optimization and greedy search strategies, the efficiency of generating adversarial examples and the probability of attack success are improved.

[0096] Compared with traditional technical means, the present invention is not limited to the basic transformation of the identifier level, but further incorporates more complex structural level transformations and sophisticated label level perturbations to promote the diversification of adversarial examples. Traditional methods often rely on genetic algorithms or embedded vector loss information to generate adversarial examples, which may cause the generated instances to appear unnatural or unrecognizable. In contrast, the present invention generates natural replacement identifiers using a masked language model and optimizes the generation process using gradient information to ensure that in a cross-domain environment, adversarial examples exhibit higher concealment, naturalness, and attack effectiveness, while having stronger practical application value and robustness.

[0097] In summary, compared with the prior art, the present invention has the advantages of generating richer, more hidden and natural adversarial examples, and generating them faster. Its innovations are mainly reflected in: introducing equivalent perturbations at the structural level and the tag level, broadening the exploration space of adversarial attacks, allowing adversarial examples to better cope with defense measures in different fields; using a masked language model to generate replacement identifiers that meet programming specifications, and combining gradient optimization with a greedy search strategy to accelerate the selection process of optimized replacement identifiers, ensuring that the generated code is both natural and efficient; in addition, using gradient information to optimize the generation process, rather than relying solely on embedded vector loss, makes adversarial examples more accurate and efficient, greatly improving the success rate and efficiency of attacks in cross-domain environments.

[0098] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0099] Based on the same idea as the cross-domain adversarial attack method based on the pre-trained code model in the above-mentioned embodiment, the present invention also provides a cross-domain adversarial attack system based on a pre-trained code model, which can be used to execute the above-mentioned cross-domain adversarial attack method based on the pre-trained code model. For ease of explanation, the structural diagram of the embodiment of the cross-domain adversarial attack system based on the pre-trained code model only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than the illustrated structure, or combine certain components, or arrange the components differently.

[0100] See also Figure 3 In another embodiment of the present application, a cross-domain adversarial attack system 100 based on a pre-trained code model is provided, the system comprising a source code data acquisition module 101, a replacement identifier generation module 102, a first equivalent transformation module 103, and a second equivalent transformation module 104;

[0101] The source code data acquisition module 101 is used to acquire source code data, traverse the source code data to identify its basic constituent unit token, and the token at least includes an identifier and an operator;

[0102] The alternative identifier generation module 102 is used to generate a natural alternative identifier set for the extracted identifier based on the masked language model;

[0103] The first equivalent transformation module 103 applies equivalent transformation at the token level to preliminarily optimize the adversarial example;

[0104] The second equivalent transformation module 104 applies an equivalent structural transformation to further transform the initially optimized adversarial example to obtain a final adversarial example.

[0105] It should be noted that the cross-domain adversarial attack system based on a pre-trained code model of the present invention corresponds one-to-one to the cross-domain adversarial attack method based on a pre-trained code model of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the cross-domain adversarial attack method based on a pre-trained code model are applicable to the embodiment of the cross-domain adversarial attack based on a pre-trained code model. For specific contents, please refer to the description in the embodiment of the method of the present invention. It will not be repeated here. This is hereby declared.

[0106] In addition, in the implementation of the cross-domain adversarial attack system based on the pre-trained code model in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the cross-domain adversarial attack system based on the pre-trained code model is divided into different program modules to complete all or part of the functions described above.

[0107] See also Figure 4 In one embodiment, an electronic device for implementing a cross-domain adversarial attack method based on a pre-trained code model is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a cross-domain adversarial attack program 203 based on a pre-trained code model.

[0108] Among them, the first memory 202 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 200. Further, the first memory 202 can also include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can not only be used to store application software and various types of data installed in the electronic device 200, such as the code of the cross-domain adversarial attack program 203 based on the pre-trained code model, but also can be used to temporarily store data that has been output or is to be output.

[0109] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The first processor 201 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the first memory 202, and calling data stored in the first memory 202.

[0110] Figure 4 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the electronic device 200 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0111] The cross-domain adversarial attack program 203 based on the pre-trained code model stored in the first memory 202 in the electronic device 200 is a combination of multiple instructions. When running in the first processor 201, it can achieve:

[0112] Acquire source code data, traverse the source code data to identify its basic constituent unit token, wherein the token includes a keyword, an identifier, an operator, a separator, and a control constant;

[0113] Generate a set of natural alternative identifiers for the extracted identifiers based on a masked language model;

[0114] Apply token-level equivalent transformations to preliminarily optimize adversarial examples;

[0115] The initially optimized adversarial examples are further transformed by applying equivalent structural transformation to obtain the final adversarial examples.

[0116] Furthermore, if the module / unit integrated in the electronic device 200 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0117] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0118] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A cross-domain adversarial attack method based on a pre-trained code model, characterized in that: The steps include: Acquire source code data, traverse the source code data to identify its basic constituent unit token, wherein the token includes at least an identifier and an operator; Generate a set of natural alternative identifiers for the extracted identifiers based on a masked language model; Apply token-level equivalent transformations to preliminarily optimize adversarial examples; The initially optimized adversarial examples are further transformed by applying equivalent structural transformation to obtain the final adversarial examples.

2. The cross-domain adversarial attack method based on the pre-trained code model according to claim 1, characterized in that: The source code data is obtained, and the source code data is traversed to identify its basic constituent unit token, specifically: Receive source code sets from different downstream task datasets, preprocess each source code, remove single-line comments, multi-line comments, document comments, and blank lines to reduce interference from irrelevant information; The lexical analyzer traverses the source code character by character to identify its basic building block token; Record the occurrence and frequency of identifiers, operators and control structures to provide a basis for subsequent semantically preserved code conversion.

3. The cross-domain adversarial attack method based on the pre-trained code model according to claim 1, characterized in that: The method of generating a natural set of alternative identifiers for the extracted identifiers based on the masked language model is as follows: Perform mask operations on all identifiers in each source code in turn; Input the masked source code into a preset masked language model and make predictions based on context information to generate multiple potential alternative identifiers; The generated alternative identifiers are screened so that the screened alternative identifiers meet the semantic requirements of programming specifications and contexts; A set of natural alternative identifiers is constructed based on the screened alternative identifiers for the subsequent adversarial example generation process.

4. The cross-domain adversarial attack method based on the pre-trained code model according to claim 1, characterized in that: The application of equivalent structural transformation to generate preliminary adversarial examples is specifically: Calculate the importance of identifiers: replace each identified identifier with a placeholder to simulate its absence; Then, the original code and the code containing the placeholder are input into the pre-trained code model to obtain the first eigenvector and the second eigenvector respectively; the importance of the identifier is characterized by calculating the L2 norm of the first eigenvector and the second eigenvector; all the identifiers are sorted according to the importance score; Replace identifiers and use gradient information to optimize selection: replace the sorted identifiers in turn and decide whether to replace them based on their gradient information. Specifically, for each identifier, generate K replacement identifiers as a replacement set I, replace the original identifier with the K replacement identifiers in the set, and obtain K replaced code examples x'; The optimal replacement identifier is found by calculating the difference between x' and the original code example x based on the gradient information; if the optimal replacement identifier is not found, no replacement is performed, and the next round of iteration is continued to determine whether the next identifier should be replaced; Use gradient information to calculate the optimal replacement: For each replaced identifier code example x_sub, calculate the gradient of the example relative to the model loss function And through the symbolic function Extract the direction information of the gradient; multiply the sign function by a small perturbation coefficient ∈ to generate a perturbation And apply the perturbation to the original embedding representation x_orig to obtain the adversarial input Predict the logits output f(x_adv) and predicted label of the adversarial input x_adv through the pre-trained code model; if the maximum logits value drops significantly, it indicates that the adversarial example is effective, and the maximum logits value is the classification confidence of the pre-trained code model; by evaluating the impact of each alternative identifier on the model, select the identifier that makes the confidence of the pre-trained code model drop to the lowest as the optimal replacement solution; by replacing and evaluating the identifiers one by one, finally form a preliminary adversarial example; Operator-level replacement optimization: After obtaining the preliminary adversarial example x_adv, operator-level replacement is further performed. Specifically, based on the number of occurrences and position information of the operator in the source code, it is decided in the form of probability whether to perform operator equivalent replacement; after operator-level replacement, the adversarial example after token-level transformation is further optimized.

5. The cross-domain adversarial attack method based on the pre-trained code model according to claim 4 is characterized in that: The importance of an identifier is calculated using the following formula: Among them, ||.|| 2 2 represents the L2 norm of the square, which is used to measure the change in the code features after the placeholder is inserted. F(x) represents the first eigenvector, and F(x') represents the second eigenvector. OSI represents the importance of the identifier. The larger the OSI value, the greater the impact of the identifier on the code features and the higher its importance.

6. The cross-domain adversarial attack method based on the pre-trained code model according to claim 1, characterized in that: The application of equivalent structural transformation further transforms the initially optimized adversarial example to obtain the final adversarial example, specifically: Branch structure replacement: Analyze the source code structure, identify the parts that can be equivalently transformed, and decide whether to replace them in a probabilistic manner based on the number of times the branch structure appears in the code. These replacements can evade detection tools based on static analysis and generate more hidden adversarial examples. Insertion of redundant code: Sort the positions of each line of code according to their importance, and determine whether to insert redundant code in turn, and decide the necessity of insertion based on random probability; the insertion of redundant code further increases the complexity of adversarial examples, making them more difficult to detect.

7. The cross-domain adversarial attack method based on a pre-trained code model according to claim 1, characterized in that: It also includes a step to evaluate the transfer ability of the optimized adversarial examples, specifically Generate adversarial examples using pre-trained code models on a variety of tasks and datasets, and then apply these examples to the pre-trained code model fine-tuned on the same task dataset; By comparing the attack success rate and stability of these fine-tuned pre-trained code models on adversarial examples, we analyze the transfer effect and robustness of examples in different fine-tuned pre-trained code models and tasks. If these examples can effectively reduce the prediction accuracy of the fine-tuned pre-trained code model in multiple environments, it means that these examples have good cross-domain migration capabilities. Based on the evaluation results, the generation strategy is further optimized to improve the applicability and attack effect of adversarial examples.

8. A cross-domain adversarial attack system based on a pre-trained code model, characterized in that: A cross-domain adversarial attack method based on a pre-trained code model applied to any one of claims 1-7, comprising a source code data acquisition module, an alternative identifier generation module, a first equivalent transformation module, and a second equivalent transformation module; The source code data acquisition module is used to acquire source code data, traverse the source code data to identify its basic constituent unit token, wherein the token includes at least an identifier and an operator; The alternative identifier generation module is used to generate a natural alternative identifier set for the extracted identifier based on the masked language model; The first equivalent transformation module applies token-level equivalent transformation to preliminarily optimize adversarial examples; The second equivalent transformation module applies an equivalent structural transformation to further transform the initially optimized adversarial example to obtain a final adversarial example.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the cross-domain adversarial attack method based on the pre-trained code model as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the cross-domain adversarial attack method based on a pre-trained code model described in any one of claims 1 to 7 is implemented.

Images