Program source code automatic vulnerability repairing method based on deep learning
Through deep learning technology, the context information of code changes is extracted and the Vulnerability repair is repaired using the Transformer-XL model, which solves the problem of insufficient accuracy and efficiency of vulnerability repair in the existing technology, and achieves a deep understanding of complex code structures and efficient repairs. The generation of solutions improves the reliability of software development and maintenance.
Patent Information
- Application Number
- CN202411001036.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When existing automated program repair technologies deal with complex and large-scale software systems, vulnerability repair is insufficient, especially the identification and repair of small sample size data sets, and it is difficult to deeply understand the code structure and syntax, resulting in The adaptability and reliability of the repair plan are insufficient.
Adopt automated vulnerability repair methods based on deep learning, extract the context information of code changes through the preprocessing stage, use CodeBERT and Transformer-XL models for data preprocessing and model training, and combine neural networks and syntax checking tools to generate a repair solution to ensure the repair solution accuracy and safety.
It significantly improves the accuracy and efficiency of vulnerability repair, enhances the model's understanding of complex code structures, and improves the reliability and practicality in software development and maintenance.
Smart Images

Figure CN118965361A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer program vulnerability repair, and in particular relates to a method for automatically repairing program source code vulnerabilities based on deep learning technology. Background Art
[0002] In the field of computer program vulnerability repair, traditional methods rely on manual operations by developers and rely on their expertise and understanding of software codes. As software becomes increasingly complex, this method has gradually shown its limitations. Especially in a rapidly evolving software environment, frequent function updates, code refactoring, and bug fixes require more efficient vulnerability management strategies.
[0003] In recent years, the research and application of automated program repair technology has attracted increasing attention due to the prevalence of vulnerabilities in large software systems. These vulnerabilities not only pose a threat to users' data security, but also reveal the shortcomings of traditional repair methods, especially in the ability to quickly identify and resolve emerging vulnerabilities.
[0004] In the field of automated program repair, researchers try to develop systems that can automatically detect and repair software vulnerabilities. The core idea of these systems is to use historical repair records, template recognition, and machine learning techniques to predict and implement effective repair strategies.
[0005] However, these automated techniques also face challenges. First, the complexity and diversity of vulnerabilities require systems that can understand a wide range of attack scenarios and vulnerability types. Second, the strict syntax and structure rules of programming languages make it more difficult to generate effective and feasible repair patches.
[0006] In summary, although the automated vulnerability repair technology has made progress in theory and practice, there are still many challenges and limitations. Therefore, there is an urgent need for a more accurate and efficient program source code automated vulnerability repair method and system to overcome the shortcomings of the existing technology and improve the stability of program operation. Summary of the invention
[0007] The present invention aims to solve key problems in the field of automated vulnerability repair, specifically including the following aspects:
[0008] 1. Improve the accuracy and efficiency of vulnerability repair, especially when dealing with large-scale and complex software systems.
[0009] 2. Solve the problem of vulnerability identification and repair on small sample size data sets.
[0010] 3. Strengthen the deep understanding of code structure and syntax to improve the adaptability and reliability of repair solutions.
[0011] To achieve the above objectives, the present invention introduces an improved automatic vulnerability repair method, the specific process of which is as follows:
[0012] Preprocessing phase: In this phase, we extract the context of code changes from the bug fix and vulnerability fix datasets. Through normalization and abstraction based on data flow dependencies, we extract the definition-use chain and provide the deep learning model with a dataset that can effectively capture the syntax and structure information around the vulnerability.
[0013] Training phase: The training process is divided into two parts: pre-training and fine-tuning. In the initial stage, pre-training is performed on the bug repair dataset, using its rich training samples to capture common repair features. Subsequently, the model is fine-tuned on the vulnerability repair dataset to adapt to specific types of vulnerability repair tasks. This stage focuses on adjusting the model weights and optimizing the model to adapt to the processing requirements of real vulnerability scenarios.
[0014] Prediction and patching phase: In this phase, the model conducts in-depth analysis of the input vulnerable files, locates potential suspicious code, and proposes repair suggestions based on model training. We combine advanced neural network technology and syntax checking tools to screen and optimize the final repair plan to ensure that the generated patch can not only solve the vulnerability problem, but also avoid introducing new errors.
[0015] In summary, the present invention proposes a program source code automatic vulnerability repair solution based on deep learning technology. This solution can significantly improve the accuracy and efficiency of vulnerability repair, while enhancing the model's ability to understand complex code structures, making it more reliable and practical in actual software development and maintenance processes. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 A schematic diagram of a framework of a method for automatically repairing program source code vulnerabilities based on deep learning technology provided by an embodiment of the present invention;
[0017] Figure 2 A schematic diagram of the data preprocessing operation steps provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0018] The core objective of this application is to provide a method for automatic vulnerability repair in program source code based on deep learning technology, which aims to effectively improve the accuracy and efficiency of vulnerability repair, especially when dealing with large-scale and complex software systems. In order to have a deeper understanding of the technical solution of the present invention, the exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be pointed out that the embodiments and other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0019] In the present invention, the implementation of the technical solution is based on the division of logical functions rather than physical structure or specific implementation method. This means that in practical applications, these functional modules can be combined, integrated, ignored or executed in different ways according to specific needs and conditions. In addition, the various technical features and solutions mentioned in the present invention are multiple possibilities proposed to achieve the invention objectives, and the possibility of other changes and modifications is not excluded.
[0020] refer to Figure 1 , which shows a schematic diagram of a method for automatic program source code vulnerability repair using deep learning technology in the present invention. The method includes the following important steps:
[0021] In step S1, the preprocessing phase: In this phase, the contextual information of the code changes is extracted through the dataset, and then normalized and abstracted based on the data flow dependencies to streamline the data. This phase involves the following subdivision steps:
[0022] Step S11, data set processing: This step processes the data set and converts it into a standard format, that is, the codes are divided into pairs of original problem codes and corrected codes. If the data set is already in this format, this step is skipped; otherwise, the data format is standardized by programming means. This usually includes writing scripts or programs to parse and reconstruct the information in the data set.
[0023] In step S12, the code difference extraction stage, we introduced CodeBERT technology, a deep learning-based code understanding and analysis tool that is specially trained on the semantics and structure of programming languages. By leveraging the capabilities of CodeBERT, we can more accurately grasp the true intention and deep impact of code modifications, thereby improving the accuracy and efficiency of difference extraction. The following are the specific steps we use CodeBERT for code difference extraction:
[0024] Code Embedding: CodeBERT is used to embed the original code and the revised code to generate vectors that represent the semantics and structure of the code. This helps capture the deep semantic relationships behind the code modifications.
[0025] Semantic difference analysis: By comparing the embedding vectors of the original code and the revised code, we can identify the semantic differences of the code modification. This comparison is not only based on the differences at the text level, but also focuses on the functional and logical changes of the code.
[0026] Contextual understanding: CodeBERT can understand the contextual information of the code, including function calls, variable scopes, etc. This helps us consider the impact of code modifications on the overall program structure when extracting code differences.
[0027] Difference extraction optimization: Based on the analysis results of CodeBERT, we can extract code differences more accurately, including identifying changes in key variables and logical structures in the code. In this step, we can also identify and filter out changes that have little impact on the code function, such as only formatting or comment modifications.
[0028] Enhanced data flow and control flow analysis: Combined with the semantic analysis results of CodeBERT, further in-depth analysis of data flow and control flow is performed. This helps to more fully understand the impact of code changes on program behavior.
[0029] Evaluation and correction: The code differences extracted by CodeBERT are initially evaluated and manually corrected if necessary. This step ensures that the extracted differences accurately reflect the essence of the code modification.
[0030] By integrating CodeBERT technology, our automated vulnerability remediation method has achieved significant optimization in the code difference extraction stage, making the subsequent model training and vulnerability remediation process more accurate and effective. This deep learning-assisted method provides strong support for handling complex code modifications, thereby improving the overall performance and reliability of the automated vulnerability remediation method.
[0031] For example:
[0032] Original code:
[0033]
[0034]
[0035] Step S13, normalization: replace code variable names, constants, and strings with standardized tags, such as replacing variable names with "var1", "var2", etc. This helps reduce the vocabulary and the frequency of specific tags. The normalized code is abstracted into a sequence of tokens. This method helps reduce the problem of performance degradation in long sequence processing and enables the model to learn common patterns from different code changes more effectively.
[0036] For example, compare the deleted code to normalize:
[0037]
[0038] Through the above steps, the preprocessing stage provides accurate and representative data for model training and prediction, thereby enhancing the effectiveness of the automated vulnerability remediation method. This stage is the basis of the model training and prediction stage, ensuring the accuracy and representativeness of the data.
[0039] In step S2, the training phase, we use the advanced Transformer-XL model to process and optimize the code repair task. This phase can be divided into two core parts: pre-training and fine-tuning.
[0040] (1) In the pre-training part, the Transformer-XL model is first trained on a general bug fix dataset, which has a large number of samples, allowing the model to learn and grasp the general features and experience of code repair. The Transformer-XL model can effectively process long code sequences and memorize previous information through its unique loop mechanism, which is especially useful when processing long code segments. We extract the Diff (after S13 optimization) from the dataset, which contains the difference context between the vulnerable version and the fixed version. Through pre-training, the model can learn how to maintain the continuity and consistency of context information over long sequences.
[0041] (2) In the fine-tuning part, the Transformer-XL model is further trained on a specific vulnerability repair dataset, which is relatively small, so the model needs to be adjusted to better suit the specific vulnerability repair task. The purpose of fine-tuning is to improve the generalization ability of the model on a smaller dataset so that it can handle real-world vulnerability scenarios more accurately. During the fine-tuning process, we only adjust the last few layers of the model to prevent overfitting and retain the knowledge learned from pre-training. In addition, in order to alleviate the "catastrophic forgetting" that occurs during the retraining process, we will mix general domain data and specific domain data to maintain the repair quality of the model.
[0042] Through the above steps, the preprocessing stage provides accurate and representative data for model training and prediction, thereby enhancing the effectiveness of the automated vulnerability repair method. The use of Transformer-XL not only improves the ability to process long code sequences, but also improves the model's performance in handling complex, long-distance dependency vulnerability repair tasks through its improved attention mechanism and memory function.
[0043] In step S3, the prediction and patching phase, our approach uses the deep learning capabilities of the Transformer-XL model to analyze the vulnerable code and generate a series of potential fixes. This phase includes the following sub-steps:
[0044] Step S31, input problematic code: In this step, vulnerable code is used as input for analysis by the Transformer-XL model. The input code may be a suspicious code snippet identified by a classifier, vulnerability detection tool, or manually.
[0045] Step S32, code preprocessing: preprocess the input problem code, including code standardization and abstraction. This step helps convert the code into a format that the model can effectively process, such as normalizing variable names into standardized tags (such as "var1", "var2", etc.).
[0046] Step S33, prediction and repair: Use the pre-trained and fine-tuned Transformer-XL model to analyze and predict the pre-processed code to generate multiple repair solutions. In this step, the model uses its improved attention mechanism and long-distance dependency processing capabilities to generate possible repair candidates for each vulnerable code.
[0047] Step S34, output patch code: Finally, the repair solution generated by the model undergoes syntax checking and static analysis (such as using the FindBugs tool) to ensure that the patch provided not only solves the problem but also does not introduce new errors. Through this series of steps, we not only provide a possible repair solution, but also provide multiple candidate solutions for developers to choose from, thereby increasing the reliability and accuracy of the repair solution.
[0048] Through step S3, our method can not only generate high-quality repair solutions, but also adapt to various complex vulnerability scenarios, thereby improving the efficiency and accuracy of automated vulnerability repair.
[0049] Through the above system and workflow, the present invention provides a method for automatic vulnerability repair of program source code based on deep learning technology. The core of the present invention is to use advanced machine learning technology to analyze and process vulnerabilities in software code, thereby automatically generating repair solutions. This method not only improves the efficiency and accuracy of vulnerability repair, but also can process large and complex data sets, effectively responding to challenges in modern software systems.
[0050] It is worth noting that, depending on specific needs and conditions, the above steps can be combined, integrated, ignored or executed in different ways to adapt to application needs in different scenarios. In this way, the present invention provides a comprehensive automated vulnerability repair solution, which aims to improve the accuracy and efficiency of vulnerability repair, while enhancing the model's ability to understand complex code structures, making it more reliable and practical in real-world software development and maintenance.
[0051] The application of the present invention is not limited to the specific embodiments; technicians may make adjustments without departing from the principles and spirit of the invention. These adjustments may include, but are not limited to, using different deep learning models, optimizing for specific types of software vulnerabilities, or adjusting preprocessing and training processes. All of these adjustments should be within the scope of protection of the claims attached hereto.
[0052] Therefore, for those skilled in the art, according to the basic principles of the present invention, the disclosed embodiments can be appropriately adjusted and changed to meet different application requirements. For example, optimization can be performed according to a specific programming language or software architecture, or special training can be performed for specific types of vulnerabilities. However, these adjustments and changes still fall within the scope of protection defined in the attached claims.
[0053] In summary, the present invention provides an innovative automated vulnerability repair solution that can adapt to the ever-changing technical environment and the growing software security requirements.
Claims
1. A method for automatically repairing program source code vulnerabilities based on deep learning, characterized in that: The preprocessing phase includes extracting contextual information of code changes from bug fix and vulnerability fix datasets and performing normalization and abstraction based on data flow dependencies to optimize the amount of training data; The training phase includes pre-training on the bug repair dataset and model fine-tuning on the vulnerability repair dataset to adapt to specific types of vulnerability repair tasks; The prediction and remediation phase includes analyzing the input vulnerable files, locating potential suspicious codes, and making repair suggestions based on model training.
2. The pretreatment stage according to claim 1, characterized in that: For data whose format does not satisfy the original vulnerability code-fix code pair format, the data set is parsed and reconstructed. CodeBERT technology is used to optimize the code difference extraction process to improve the accuracy and efficiency of difference extraction. The normalization step involves replacing code variable names, constants, and strings with standardized tokens and abstracting the normalized code into token sequences.
3. The training phase as claimed in claim 1, characterized in that: The Transformer-XL model is used to effectively process long code sequences through its unique loop mechanism, maintaining the continuity of contextual information in long sequences. First, pre-train on a common bug repair dataset to learn common features and experience of code repair; In the fine-tuning stage, by adjusting the last few layers of the model, the generalization ability of the model on a specific vulnerability repair dataset is enhanced, thereby improving the performance of the model in real vulnerability scenarios.
4. The prediction and repair phase as claimed in claim 1, characterized in that: In the problem code input stage, the vulnerable code is used as input for the Transformer-XL model to perform in-depth analysis; In the code preprocessing stage, the input code is standardized and abstracted to adapt to the processing format of the model; In the prediction and repair phase, multiple possible repair solutions are generated based on the pre-trained and fine-tuned Transformer-XL model; In the patch code output phase, the generated fix solution is subjected to syntax checking and static analysis to ensure that the provided patch can solve the problem without introducing new errors.
Citation Information
Patent Citations
Information security vulnerability classification method based on improved Bert model
CN113961933A
Program vulnerability automatic repairing method based on generative adversarial network
CN117828615A
Method for enhancing vulnerability detection based on prompt learning through reinforcement learning
CN117972716A
Vulnerability code repairing method and system based on pre-training large model
CN118092998A
Source code vulnerability detection method and system based on large model
CN118332557A
