Depth code model robustness improving method and system based on code standardization

Abstract

The invention discloses a depth code model robustness improvement method and system based on code normalization, and the method comprises the steps: recognizing all identifiers in a code through a static program analysis technology, and replacing the identifiers with meaningless placeholders, so as to eliminate potential attack features; on the basis of the context understanding capability of a large language model, semantic reconstruction is carried out on placeholders, meaningful identifier names are generated, and code semantic information is recovered; using standardized training data to perform distribution alignment fine adjustment on the model, and adopting the same preprocessing process in the reasoning stage to ensure input distribution consistency. According to the method, on three code tasks (clone detection, defect prediction and vulnerability detection) and four mainstream models, the average experience robustness of multiple attack methods is improved by 15.4%-126.0%, and the method is superior to an existing defense method under the condition of 87.9% in 33 test scenes.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence and software security, and particularly relates to a robustness improvement method of a deep code model. BACKGROUND

[0002] Deep code models (DCMs) have shown superior performance in various applications such as vulnerability detection and code generation. However, like all deep learning models, they are vulnerable to input perturbations and thus susceptible to attacks by malicious actors. For example, renaming a variable in a piece of flawed code, seemingly trivial, can actually mislead these models into incorrectly classifying it as safe code, which can lead to very serious security vulnerabilities. Given the widespread use of deep code models in security-sensitive applications, it is crucial to ensure their robustness.

[0003] To enhance the robustness and reliability of deep code models (DCMs) against adversarial inputs, many defense methods have been proposed in the past few years. These methods usually generate new inputs with adversarial features based on predefined mutation rules, and then fine-tune the model on an augmented dataset consisting of adversarial inputs generated by this process. The basic goal of this process is to alleviate the out-of-distribution problem by expanding the feature space of the training data, thereby enhancing the overall predictive robustness of the model, but such methods have a significant limitation: they rely on a fixed set of predefined mutation rules, which have limited generalization in different attack scenarios. Since the generated inputs are limited in themselves, they cannot fully cover the wide range of potential out-of-distribution inputs. Therefore, a model strengthened with some adversarial samples may still be vulnerable to other types of adversarial samples. Because the complete input space of deep code models is usually infinite, it is impossible to consider all possible inputs. How to improve the robustness of the model against different types of attacks has become an important challenge in current research.

[0004] In addition, existing methods cannot accurately identify all potential vulnerable identifiers in a huge search space. Traditional identifier generation methods (such as frequency-based methods) have obvious limitations: on the one hand, it is impossible to determine whether the identifier in the context has been maliciously contaminated, and blind reuse may introduce additional risks; on the other hand, it lacks a deep understanding of code functionality, making it difficult to generate identifiers that accurately express code semantics. SUMMARY

[0005] In view of the defects in the prior art, the application provides a deep code model robustness improvement method and system based on code normalization, which innovatively uses a large language model for code identifier semantic recovery through a code standardization processing flow combining code abstraction and instantiation, establishes a complete model robustness improvement technical system, solves the problem that a deep code model is vulnerable to adversarial attacks, and provides a unified defense scheme that can effectively improve model robustness.

[0006] In order to achieve the above-mentioned application purposes, the application provides the following technical solutions:

[0007] In the first aspect, the application provides a deep code model robustness improvement method based on code normalization, which comprises the following steps:

[0008] S1: receiving original code input, performing code abstraction processing on the original code to replace all identifiers in the code with standardized placeholders, and generating abstract code, wherein the code abstraction processing comprises identifying all identifiers by using a static program analysis technique;

[0009] S2: inputting the abstract code into a large language model, replacing the standardized placeholders in the abstract code with new identifiers with semantics based on a constructed prompt word, and generating instantiated code;

[0010] S3: using the standardized training data set obtained by the code abstraction processing and code instantiation processing to fine-tune a deep code representation model, so that the model adapts to the data distribution of standardized code;

[0011] S4: sequentially performing the code abstraction processing and code instantiation processing on the code to be predicted, and inputting the processed code into the fine-tuned deep code representation model for prediction.

[0012] In some embodiments, the code abstraction processing of S1 specifically comprises:

[0013] parsing the original code into an abstract syntax tree;

[0014] constructing a data flow graph based on the abstract syntax tree and recording the mapping relationship between phrases in the code and positions in the original code;

[0015] traversing the data flow graph to identify and collect all identifier nodes;

[0016] based on the mapping relationship, replacing the identifiers in the original code with standardized placeholders of a preset format, wherein the method declaration name is replaced with a uniform identifier.

[0017] In some implementations, the standardized placeholder is formatted as var followed by a numeric sequence number.

[0018] In some implementations, the prompt word includes a task definition section, a one-time example section, and an abstract code section to be instantiated;

[0019] The task definition section is used to instruct the replacement of all standardized placeholders while maintaining code functionality;

[0020] The one-off example section is used to demonstrate replacement paradigms to a large language model.

[0021] In some implementations, downstream tasks of the deep code representation model include at least one of code clone detection, software defect prediction, or vulnerability detection.

[0022] Second, a robustness improvement system for deep code models based on code normalization for implementing the method as described in any one of claims 1 to 5, characterized in that the system comprises:

[0023] The code abstraction module is configured to receive the original code, replace all identifiers in the code with standardized placeholders through static program analysis, and output abstract code.

[0024] A code instantiation module, connected to the code abstraction module, is configured to receive the abstract code and generate instantiation code that replaces the standardized placeholders based on prompt words using a large language model;

[0025] The model fine-tuning module, connected to the code instantiation module, is configured to fine-tune the deep code representation model using a standardized dataset composed of instantiated code;

[0026] The inference interface module is connected to the code abstraction module, the code instantiation module, and the fine-tuned deep code representation model. It is configured to preprocess the input code and call the model to obtain the prediction result during inference.

[0027] In some implementations, the code abstraction module further includes:

[0028] The syntax parsing unit is configured to parse raw code into an abstract syntax tree.

[0029] A data flow graph construction unit is configured to construct a data flow graph and position mapping relationships based on the abstract syntax tree;

[0030] The identifier collection unit is configured to traverse the data flow graph to identify and collect all identifier nodes.

[0031] The execution unit is replaced and configured to replace the identifier with a standardized placeholder based on the location mapping relationship.

[0032] In some implementations, the code instantiation module includes a prompt word management unit configured to generate and maintain prompt word templates containing task definitions and one-time examples.

[0033] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements any of the methods described above.

[0034] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods described herein.

[0035] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0036] 1) By combining large language models with static program analysis techniques, all potentially attackable locations in the code are standardized to eliminate potential polluting identifiers and improve the model's robustness against adversarial attacks.

[0037] 2) Fine-tuning the model using these uniformly formatted training data ensures that the model learns from a controlled and consistent data distribution; during inference, the same transformation is applied to the incoming input to make its distribution consistent with the distribution of the fine-tuned data; this alignment can significantly improve the model's prediction confidence, thereby enhancing the model's robustness.

[0038] 3) On three of the most widely used code tasks (clone detection, defect prediction, and vulnerability detection) and four mainstream models (CodeBert, Graphcodebert, CodeT5, and CodeT5plus), the average empirical robustness against various attack methods is improved by 15.4%–126.0%, outperforming existing defense methods in 87.9% of the 33 test scenarios; this verifies that the proposed model robustness improvement method can effectively eliminate interference and improve the inference accuracy of the model. Attached Figure Description

[0039] Figure 1 This is a flowchart of the robustness improvement method for deep code representation model based on code normalization according to the present invention;

[0040] Figure 2 This is a technical roadmap for an embodiment of the present invention.

[0041] Figure 3This is an example diagram of an abstract algorithm in an embodiment of the present invention.

[0042] Figure 4 This is an example diagram of the prompt words in an embodiment of the present invention.

[0043] Figure 5 This is a structural diagram of the robustness improvement system of the deep code representation model based on code normalization according to the present invention.

[0044] Figure 6 This is a structural diagram of the code abstraction module of the present invention.

[0045] Figure 7 This is a schematic diagram of the experimental results of the present invention. Detailed Implementation

[0046] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0047] Example 1: As Figure 1 As shown, this invention proposes a method for improving the robustness of deep code models based on code normalization, such as... Figure 2 The diagram shown is a technical roadmap of an embodiment of the present invention; the specific steps of the method are as follows:

[0048] Step S1, Code Abstraction: Perform static program analysis on the input raw code, identify and replace all identifiers with meaningless standardized placeholders, and generate abstract code; the specific process is as follows:

[0049] Input code abstraction involves converting input code into a standardized representation using static program analysis techniques.

[0050] For a given input code snippet, it is first parsed into an abstract syntax tree and a data flow graph (DFG) is constructed. By traversing the DFG, the Unicode method checks whether each node is an identifier node, and all identifier nodes are collected into a set named ID. Simultaneously, all code phrases and their position mapping pairs in the original code are recorded. Finally, based on the mapping pairs, the Unicode method uniformly replaces all identifiers in the set ID with a meaningless format of "var" followed by an integer number (e.g., var0, var1), thereby eliminating interference from the original identifiers while ensuring that the data dependencies of the code remain unchanged.

[0051] The core of the code abstraction phase lies in accurately identifying and replacing all identifiers in the code through static program analysis techniques. The specific algorithm and implementation process for "identifier replacement" in this code abstraction step are described below:

[0052] The process begins by parsing the input code snippet into an Abstract Syntax Tree (AST) and further constructing its Data Flow Graph (DFG). The DFG is then traversed to identify and replace all identifiers, ensuring that data dependencies remain unchanged during replacement. Algorithm 1 outlines the detailed process of code abstraction: Given a code snippet 𝑥, Unicode first parses it into an AST format (line 2), and then constructs a DFG based on the AST. This process records all phrases in the map (i.e., the map in line 3) and their associated positions in the original code. This step is crucial because Unicode aims to replace identifiers without altering irrelevant code functionality such as comments and formatting; converting the AST back to source code might unintentionally change these functionalities. Therefore, identifier replacement occurs on the original plain text format of the code. Crucially, the system synchronously records all phrases in the code and their positional mappings in the original code. This ensures that subsequent identifier replacement operations operate precisely on the original code text, completely avoiding unintentional changes to non-functional elements such as comments and formatting that might occur when reversing the AST to generate source code, thus guaranteeing the integrity of the code's functionality.

[0053] like Figure 3 As shown in the example of the abstraction algorithm in this embodiment of the invention, the specific replacement algorithm is as follows: The system traverses each node in the DFG, determines whether it is an identifier node, and collects all identified identifier nodes into a set named ID (lines 5-9). Subsequently, based on the previously recorded position mapping, all identifiers in the set are uniformly replaced with meaningless placeholders of "var" plus a numerical number (such as var1, var2) (lines 11-15). In order to maintain all code functions except for identifiers and to completely eliminate potential attack vectors, this process is performed on a new copy of the original code (i.e., 𝑥'), and all method names in the code are also uniformly replaced with "methodName" (line 16). Through this process, all identifiers (regardless of whether they were originally contaminated) are standardized, thereby eliminating interference while completely preserving the data dependencies and core functional logic of the code.

[0054] Step S2, Input Code Instantiation: The standardized placeholders in the abstract code are semantically restored using a large language model to generate instantiated code with new semantic identifiers; the specific process is as follows:

[0055] After removing all interfering identifiers during the code abstraction phase, this step leverages the semantic understanding capabilities of the large language model to perform semantic restoration and identifier reconstruction on standardized placeholders. The large language model prompts used in this process consist of three parts: 1) Task definition: The prompt begins by clearly defining the task objective, requiring the model to replace all meaningless placeholders (such as var1) with meaningful identifiers based on the contextual semantics of the given code, while ensuring the code's functionality remains unchanged. 2) One-time example: A complete input-output example is then provided, including an input code snippet with placeholders and the corresponding target code that successfully restores the identifier's semantics. The target code represents the semantic reconstruction result, guiding the large language model to accurately understand the replacement rules and semantic consistency requirements. 3) Instantiating the input code: The final part explicitly specifies the specific code snippet to be processed, i.e., the target code that needs to undergo identifier instantiation.

[0056] To balance code security and model performance, this invention uses meaningless placeholders to replace identifiers during the code abstraction stage to eliminate the attack surface, and then introduces a code instantiation mechanism based on a large language model. This aims to restore the semantic information of the identifiers and avoid the decline in model prediction performance caused by complete abstraction.

[0057] This invention uses a large language model to achieve semantic recovery of identifiers, and its advantages are:

[0058] Large language models demonstrate an exceptional ability to understand code semantics and can generate meaningful identifiers that are highly relevant to the functionality of the code.

[0059] Large language models possess powerful pattern recognition and generation norms, which can effectively reduce the probability of introducing polluting identifiers.

[0060] To effectively guide large language models in generating safe and semantically clear identifiers, this invention designs a structured prompt template based on best practices in prompt engineering. For example... Figure 4 As shown, the prompt word consists of three components:

[0061] Task Definition: Clearly state that the goal is to replace meaningless placeholders with identifiers that conform to the program's functionality by understanding the semantics of the code context. The goal is to clearly outline the objective at the beginning of the prompt, namely, to replace all meaningless placeholder identifiers (e.g., var1) by understanding the semantics of the given code snippet, with the aim of ensuring that this replacement maintains the program's functionality.

[0062] One-time example: Provides a complete example including input code and restored code to visually demonstrate the replacement rules and semantic consistency requirements; followed by a clear example of how to replace the identifier. This example includes an input code snippet with placeholders and the corresponding target code that successfully restores the identifier's semantics to further enhance the large model's understanding of the task requirements.

[0063] Instantiate input code: Specifies the specific code content to be processed, guiding the model to perform identifier recovery against the target code. This section contains the specific code that needs to be instantiated.

[0064] This prompt design ensures that the large language model can accurately understand the task intent, generate identifier names that are both semantically consistent with the code and resistant to attacks, and provide high-quality input representations for subsequent model inference.

[0065] Step S3, Model Fine-tuning: The deep code representation model is fine-tuned using the standardized training dataset processed in steps S1 and S2 to adapt the model to the data distribution of standardized code. The specific process is as follows:

[0066] First, using the aforementioned code standardization components of code abstraction and instantiation, the complete training dataset corresponding to the task is transformed into a standardized dataset with a uniform distribution. Then, the pre-trained deep code model is fine-tuned based on this dataset to help the model adapt to the input feature distribution of normalized code, enhance the model's ability to identify and generalize normalized code features, and ultimately improve its prediction accuracy and robustness in adversarial environments.

[0067] Step S4, Inference Preprocessing: During the model inference stage, steps S1 and S2 are executed sequentially on the new input code, and the processed code is input into the fine-tuned deep code representation model for prediction.

[0068] To improve model robustness, this invention employs a distribution alignment strategy: first, the model is fine-tuned using uniformly formatted training data to learn a consistent and standardized data distribution; during the inference phase, the same formatting process is applied to the input data to ensure its distribution remains consistent with the training distribution. This distribution alignment mechanism effectively enhances the model's prediction confidence and generalization ability when faced with out-of-distribution samples.

[0069] Furthermore, the specific configuration of the model fine-tuning parameters is as follows:

[0070] During the model fine-tuning phase, the hyperparameters were set as follows: For the large language model generation phase, the temperature parameter was set to 1.0, and the maximum generation length (max_tokens) was set to 4096. During the deep code model fine-tuning phase, the sequence length (block_size) was set to 512, and the gradient pruning threshold (max_grad_norm) was set to 1.0. Different learning rates were used for different downstream tasks: the learning rate for code clone detection and defect prediction tasks was 5×10⁻. 5 The learning rate for the vulnerability prediction task is 2×10⁻ 5 All parameters were set according to recommended settings from relevant work to ensure comparability and reproducibility of the experiments.

[0071] Compared to traditional model fine-tuning, the fine-tuning in this paper has specific technical implications:

[0072] The data foundation is different: training data that has undergone complete standardization (abstraction + instantiation);

[0073] Its goal orientation is unique: it aims to solve out-of-distribution (OOD) problems, rather than simply adapting to the task.

[0074] Unique technical effect: It focuses on improving the model's robustness against adversarial attacks, rather than general performance improvements.

[0075] In this invention, model fine-tuning achieves distribution alignment by enabling the model to learn the feature distribution of standardized code, ensuring stable predictive performance against various identifier renaming attacks. This fine-tuning is not a simple parameter adjustment, but a systematic distribution alignment strategy, a key step in achieving a unified defense mechanism.

[0076] The Necessity and Technical Principles of Fine-Tuning: In the model fine-tuning stage, the core issue to address is the data distribution shift caused by code standardization. After abstraction and instantiation, the data distribution of the code differs from the model's original training set. Directly using the original model for inference will lead to performance degradation due to this distribution mismatch. Related research also confirms that changes in identifiers can easily introduce interfering features that cause model inaccuracies. Therefore, adapting the model to the new data distribution of the standardized code is crucial to ensuring its predictive reliability.

[0077] During model fine-tuning, after obtaining uniform code snippets through code abstraction and instantiation, their distribution may differ from the distribution of the original training data because the instantiated identifiers may differ from the original code. Therefore, directly using the original code model to predict these uniform code snippets can lead to performance degradation due to out-of-distribution issues. Our research on sophisticated attack techniques also observed that identifier renaming often introduces vulnerable features. Therefore, ensuring that the distribution of standardized code is consistent with the distribution of the model's training data is crucial. To address this issue, Unicode has incorporated a model fine-tuning component.

[0078] Specifically, this invention first uses the two code normalization components mentioned earlier (i.e., code abstraction and instantiation) to transform the entire training dataset for a given task into a uniform distribution. Then, it uses this normalized dataset to fine-tune the model. This approach helps the model adapt to the input feature distribution of the normalized code, effectively improving its predictive performance on normalized inputs.

[0079] Secondly, Example 2: Figure 5 As shown, the structural diagram of the robustness improvement system for deep code models based on code normalization of the present invention is presented. The system includes:

[0080] Code abstraction module 100 is configured to receive the original code, replace all identifiers in the code with standardized placeholders through static program analysis, and output abstract code.

[0081] The code instantiation module 200 is connected to the code abstraction module and is configured to receive the abstract code and generate instantiation code that replaces the standardized placeholders based on prompt words using a large language model;

[0082] The model fine-tuning module 300 is connected to the code instantiation module and is configured to fine-tune the deep code representation model using a standardized dataset composed of instantiated code.

[0083] The inference interface module 400 is connected to the code abstraction module, the code instantiation module, and the fine-tuned deep code representation model, and is configured to preprocess the input code and call the model to obtain the prediction result during inference.

[0084] like Figure 6 As shown, the code abstraction module 100 further includes:

[0085] Syntax parsing unit 110 is configured to parse raw code into an abstract syntax tree;

[0086] The data flow graph construction unit 120 is configured to construct a data flow graph and position mapping relationship based on the abstract syntax tree;

[0087] Identifier collection unit 130 is configured to traverse the data flow graph to identify and collect all identifier nodes;

[0088] Replace execution unit 140 and configure it to replace identifiers with standardized placeholders based on the position mapping relationship.

[0089] Experimental results show that, under the aforementioned test conditions, the method proposed in this invention demonstrates significant performance advantages compared to five current mainstream defense methods in multiple typical tasks. Specifically, Unicode achieved the best defense performance in 29 out of 33 comparison scenarios (87.9%), with its defense effectiveness improving by an average of 15.4% to 126.0% compared to the baseline method. This fully verifies the superior effectiveness and universal applicability of this invention in improving the anti-interference capability of deep code models.

[0090] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements any of the methods described above.

[0091] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods described herein.

[0092] The invention was implemented as a tool called Unicode, and its effectiveness was evaluated by conducting extensive comparative experiments with five state-of-the-art defense methods on three different code-related tasks targeting four typical deep code models.

[0093] To comprehensively evaluate performance, this implementation compares Unicode with the best online method, CodeDenoise, and four offline adversarial training methods (ALERT, SODA, CodeTAE, and Marvel). The baseline selection takes into account both online detection performance and the representativeness of different techniques in offline training to ensure the breadth and depth of the comparison.

[0094] The selected baseline methods collectively constitute a comprehensive and reliable evaluation benchmark for the following reasons: First, they have broad technical coverage, encompassing various code modification methods ranging from identifier renaming to code structure variation; second, they are highly representative in performance, as these baselines all report state-of-the-art results in their respective fields; third, they offer methodological diversity, employing different adversarial example generation and defense techniques such as mutual learning and transfer learning; and finally, they are highly reproducible, with all methods being open-source, ensuring the fairness of experimental comparisons and the verifiability of results. These baseline methods can be compared with this invention in a complementary manner to provide a comprehensive and reliable evaluation of the invention.

[0095] To quantify the effectiveness of this patented technology in resisting adversarial attacks, empirical robustness (ER) is used as the core metric. This metric is a widely adopted benchmark in deep learning defense research, defined as the model's test accuracy on adversarial inputs, and can directly reflect the model's robust performance under actual attacks.

[0096] In terms of method configuration, the core settings for Unicode involve its selected Large Language Model (LLM) and its corresponding code instantiation and model fine-tuning hyperparameters. To verify the generalization ability of our method on different LLMs, we selected two representative models for evaluation: DeepSeek-V3 (released on December 26, 2024) and GPT-4o-mini (released on July 18, 2024). DeepSeek-V3 was used as the default model, while GPT-4o-mini was used to verify the applicability of the method in heterogeneous model architectures. Both support 128K context windows and exhibit excellent benchmark performance. All benchmark methods used the default configurations from their original papers or official code repositories to ensure the fairness and reproducibility of the comparative experiments.

[0097] like Figure 7As shown, the experimental results clearly demonstrate the superior performance of this method. The first two columns list the tasks and model structures, and the subsequent columns are organized into three blocks, each representing the defense results corresponding to a specific attack method. For example, the first block shows the empirical robustness (ER) values ​​of five defense methods against adversarial examples generated by CODA. For clarity, each model's defense against a specific adversarial attack is referred to as a "subject." There are a total of 33 "subjects" (12 victim models × 3 attack methods - 3 failures due to the high cost of CodeTAE to generate enough adversarial instances for CodeT5+). The best results for each "subject" are highlighted with shading. The tabular data shows that Unicode significantly outperforms all baseline methods. Specifically, the evaluation framework covers three code tasks with multiple models and reports defense results separately for three different attack methods (each model-attack combination is defined as an evaluation unit, for a total of 33 valid units). The data shows that Unicode achieved the highest empirical robustness (ER) values ​​in 87.9% of the evaluation units (29 out of 33), significantly outperforming all baseline methods. Compared to suboptimal methods, Unicode achieved an average relative performance improvement of 15.4% to 126.0%. To further verify the statistical significance of the improvement, a Wilcoxon symbolic rank test was performed, and the results showed that the p-values ​​for all comparisons were less than 0.05. This evidence collectively demonstrates the effectiveness of Unicode in defending against adversarial attacks.

[0098] In summary, this invention proposes a robustness enhancement method for deep code models based on code normalization. The technical solution involves four key stages: First, the input code is structured by using abstract syntax trees and data flow graph analysis to identify all identifiers and uniformly replace them with standardized placeholders, fundamentally eliminating potential attack vectors. Then, leveraging the semantic understanding capabilities of a large language model, context-appropriate semantic identifiers are generated for the abstract code, restoring readability while maintaining functionality. Next, the model is fine-tuned for distribution alignment using standardized training data to adapt to a unified data distribution. Finally, in the inference stage, the same standardized preprocessing procedure is applied to the input code to ensure its distribution aligns with the training data before feeding it into the fine-tuned model for prediction, thus forming a robustness guarantee throughout the entire process from training to inference.

[0099] The method and system of this invention effectively counteract various attack methods through a unified defense mechanism, ensuring that any input, regardless of its source, is transformed into a distributional representation that the model can handle with high confidence. This defense mechanism primarily comprises two stages: code abstraction and instantiation. In the first stage, code abstraction, static program analysis is used to identify all identifiers in the code and replace them with semantically meaningless placeholders (e.g., var), while preserving the original code structure and functionality. This step eliminates the adverse effects of potentially vulnerable identifiers on model predictions. In the second stage, code instantiation, a large language model (LLM) is used to generate new meaningful words from the placeholders in the abstract code for semantic recovery. This stage is crucial because it leverages the contextual understanding capabilities of the large model to recover the semantics of lost identifiers. Finally, the model is fine-tuned using these uniformly formatted training data to ensure that the model learns from a controlled and consistent data distribution. During inference, the same transformation is applied to the incoming input to preprocess its distribution to align with the distribution of the fine-tuned data. This alignment significantly improves the model's prediction confidence, thereby enhancing its robustness.

[0100] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0101] This invention is not limited to the embodiments described above. The above description of specific embodiments is intended to illustrate and explain the technical solutions of this invention. The specific embodiments described above are merely illustrative and not restrictive. Without departing from the spirit and scope of the claims, those skilled in the art can make many specific modifications based on the teachings of this invention, and these modifications all fall within the scope of protection of this invention.

Claims

1. A code normalization-based deep code model robustness improvement method, characterized in that, The method comprises: S1: receiving original code input, performing code abstraction processing on the original code to replace all identifiers in the code with standardized placeholders, and generating abstract code, wherein the code abstraction processing comprises identifying all identifiers using a static program analysis technique; S2: inputting the abstract code into a large language model, guiding the large language model to replace the standardized placeholders in the abstract code with new identifiers with semantics based on a constructed prompt, and generating instantiated code; S3: using the standardized training data set obtained by the code abstraction processing and code instantiation processing to fine-tune a deep code representation model, so that the model adapts to the data distribution of standardized code; S4: sequentially performing the code abstraction processing and code instantiation processing on the code to be predicted, and inputting the processed code into the fine-tuned deep code representation model for prediction.

2. The code normalization based deep code model robustness improvement method according to claim 1, characterized in that, The S1 of performing code abstraction processing on the original code specifically comprises: parsing the original code into an abstract syntax tree; constructing a data flow graph based on the abstract syntax tree and recording the mapping relationship between phrases in the code and positions in the original code; traversing the data flow graph to identify and collect all identifier nodes; based on the mapping relationship, replacing the identifiers in the original code with standardized placeholders of a preset format, wherein the method declaration name is replaced with a uniform identifier.

3. The code normalization based deep code model robustness improvement method according to claim 2, characterized in that, The format of the standardized placeholder is var followed by a serial number.

4. The code normalization based deep code model robustness improvement method according to claim 1, characterized in that, The prompt includes a task definition part, a one-time example part, and an abstract code part to be instantiated; the task definition part is used to instruct to replace all standardized placeholders and maintain code functionality; the one-time example part is used to show the large language model a replacement example.

5. The code normalization based deep code model robustness improvement method according to claim 1, characterized in that, The application downstream task of the deep code representation model includes at least one of code clone detection, software defect prediction, or vulnerability detection.

6. A code normalization based deep code model robustness improvement system for implementing the method of any one of claims 1 to 5, characterized in that, The system comprises: a code abstraction module configured to receive original code and replace all identifiers in the code with standardized placeholders through static program analysis, and output abstract code; a code instantiation module connected with the code abstraction module, configured to receive the abstract code and generate instantiated code with the standardized placeholders replaced according to a prompt using a large language model; a model fine-tuning module connected with the code instantiation module, configured to fine-tune a deep code representation model using a standardized data set composed of instantiated code; an inference interface module connected with the code abstraction module, the code instantiation module, and the fine-tuned deep code representation model, configured to preprocess input code and call the model to obtain a prediction result during inference.

7. The code-specification-based deep code model robustness improvement system according to claim 6, wherein, The code abstraction module further comprises: a syntax parsing unit configured to parse the original code into an abstract syntax tree; a data flow graph construction unit configured to construct a data flow graph and a position mapping relationship based on the abstract syntax tree; an identifier collection unit configured to traverse the data flow graph to identify and collect all identifier nodes; a replacement execution unit configured to replace the identifiers with standardized placeholders based on the position mapping relationship.

8. The code-specification-based deep code model robustness improvement system according to claim 6, wherein, The code instantiation module includes a prompt word management unit configured to generate and maintain prompt word templates containing task definitions and one-time examples.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the code normalization-based deep code model robustness improvement method in any one of claims 1 to 5.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the code normalization-based deep code model robustness improvement method in any one of claims 1 to 5.

Images