System and method for obtaining enhanced patch generation for automatic program repair

Abstract

Systems and methods for automatic program repair using neural network models are described herein. After receiving the first vulnerability code patch, a first representation of the first vulnerability code patch is generated using an acquirer encoder of the patch acquirer. The patch acquirer acquires a first vulnerability repair code pair from a first plurality of vulnerability repair code pairs based on the first representation. And generating a first enhanced patch with the vulnerability code based on the first patch with the vulnerability code and the first vulnerability repair code pair. The patch generator generates a repaired code patch based on the first enhanced vulnerability code patch.

Description

[0002] Cross-references

[0003] The disclosed text claims priority to U.S. non-provisional patent application No. 17 / 896,873 filed on August 26, 2022, which in turn claims priority to U.S. Provisional Patent Application No. 63 / 343,264 filed on May 18, 2022 under 35 U.S.C119, and these applications are hereby incorporated by reference in their entirety. Technical Field

[0004] Embodiments relate generally to machine learning and automatic code generation, and more particularly to systems and methods for automatic program repair (APR) using acquisition-enhanced patch generation (RAP-Gen). Background Art

[0005] Software developers often spend a lot of time and energy debugging and repairing their source code, which makes software development expensive and time-consuming. Some existing automatic program repair tools can use use cases to alleviate the difficulty and cost of program repair, including searching for patches at development time, build time, or runtime. For example, some search-based (also known as generation and verification) methods can search for repairs based on repair patterns mined via manual heuristic rules or redundancy-based techniques. Redundancy-based techniques generally make redundant assumptions, that is, the repaired patches can usually be found (or reconstructed) from other places in the code base (donor code fragment). Therefore, these conventional search-based techniques are limited in accuracy and efficiency in repairing programs.

[0006] Therefore, a more efficient way to perform automatic program repair is needed. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is used to realize Figure 3 and a simplified diagram of a computing device that implements the automatic program repair framework described in other embodiments described herein.

[0008] Figure 2 is suitable for implementing Figure 3 A simplified block diagram of a networked system of an automatic program repair framework as described in and other embodiments described herein.

[0009] Figure 3 is an exemplary block diagram illustrating an exemplary architecture of an automatic program repair framework using acquisition-enhanced patch generation, according to some embodiments described herein.

[0010] Figure 4A is an example of training according to some embodiments described herein. Figure 3An exemplary logical flow chart of a method for obtaining an enhanced patch generation framework for automatic program repair is shown; Figure 4B is an exemplary logical flow diagram illustrating a method of performing an inference process using a trained acquisition-enhanced patch generation framework according to some embodiments described herein.

[0011] Figures 5 to 15 Provides exemplary data tables according to some embodiments described herein, these exemplary data tables show Figures 1 to 4B The described automatic program repair framework uses exemplary data performance for obtaining enhanced patch generation.

[0012] In the drawings, elements with the same reference numerals have the same or similar functions. DETAILED DESCRIPTION

[0013] As used herein, the term "network" may include any hardware or software based framework, including any artificial intelligence network or system, neural network or system, and / or any training or learning model implemented thereon or in conjunction with it.

[0014] As used herein, the term "module" may include a hardware or software-based framework that performs one or more functions. In some embodiments, a module may be implemented on one or more neural networks.

[0015] Existing automatic program repair systems can reduce manual debugging efforts and improve software reliability. Conventional search-based techniques usually rely on heuristic rules or redundant assumptions to mine repair patterns. Some deep learning-based methods can automate the program repair process by training learning models to generate code repair patches. However, the performance of such learning models is usually limited by a fixed set of parameters used to model the highly complex search space of program repair.

[0016] In view of the need for efficient and accurate code repair systems, the embodiments described herein provide an acquisition-enhanced patch generation framework to acquire code patches based on relevant repair patterns using a patch acquirer. Specifically, a hybrid patch acquirer can be configured for repair pattern mining, which considers both vocabulary and semantic matching by performing sparse and dense acquisition based on the original source code. The acquirer is also a language-independent acquirer because it does not require any language-specific features, such as an abstract syntax tree. An improvement to the previous repair pattern mining model is that the acquirer utilizes the top one related vulnerability repair pair as the guiding repair pattern for each patch with a vulnerability, rather than clustering various repair templates. This strategy is aligned with the debugging behavior of human developers, who often search for relevant vulnerability repair examples to refine some repair clues for vulnerability repair.

[0017] In one embodiment, a pre-trained converter-based encoder-decoder model (e.g., CodeT5 model) can be used as a basic patch generator. CodeT5 is a general programming language model that is pre-trained on a large source code corpus using a code-aware language modeling objective. A two-stage training strategy can be used to train the pre-trained encoder-decoder model to connect the patch acquirer and the CodeT5 patch generator. The patch acquirer first searches for relevant vulnerability repair patterns and then passes them to the patch generator for synthesizing repaired patches based on the source-band vulnerability code and external (acquired) vulnerability repair knowledge. The acquired repair pattern can then be directly attached to the source-band vulnerability patch. In this way, the acquirer can be integrated with any sequence-to-sequence learning-based model for acquisition in repair pattern mining for program repair.

[0018] Figure 1 According to some embodiments, Figure 3 A simplified diagram of a computing device 100 for an automatic program repair framework is shown in FIG. Figure 1 As shown, the computing device 100 includes a processor 110 coupled to a memory 120. The operation of the computing device 100 is controlled by the processor 110. Although the computing device 100 is shown as having only one processor 110, it should be understood that the processor 110 may represent one or more central processing units, multi-core processors, microprocessors, microcontrollers, digital signal processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing units (GPUs), etc. in the computing device 100. The computing device 100 may be implemented as a standalone subsystem, a board added to a computing device, and / or a virtual machine.

[0019] Memory 120 may be used to store software executed by computing device 100 and / or one or more data structures used during operation of computing device 100. Memory 120 may include one or more types of machine-readable media. Some common forms of machine-readable media may include floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tape, any other physical media with a pattern of holes, RAM, PROM, EPROM, flash EPROM, any other memory chip or cartridge, and / or any other media from which a processor or computer is adapted to read.

[0020] The processor 110 and / or the memory 120 may be arranged in any suitable physical arrangement. In some embodiments, the processor 110 and / or the memory 120 may be implemented on the same board, in the same package (e.g., a system-in-package), on the same chip (e.g., a system-on-chip), etc. In some embodiments, the processor 110 and / or the memory 120 may include distributed, virtualized, and / or containerized computing resources. According to such embodiments, the processor 110 and / or the memory 120 may be located in one or more data centers and / or cloud computing facilities.

[0021] In some examples, the memory 120 may include a non-transient tangible machine-readable medium including an executable code that, when run by one or more processors (e.g., processor 110), may cause the one or more processors to perform the methods described in more detail herein. For example, as shown, the memory 120 includes instructions for an automatic program repair module 130 that may be used to implement and / or simulate systems and models, and / or implement any method further described herein. The automatic program repair module 130 may receive inputs 140 via the data interface 115, including inputs such as program vulnerabilities, etc. The automatic program repair module 130 may generate outputs 150, such as code patches.

[0022] In some embodiments, the automatic program repair module 130 includes an acquirer encoder submodule 131, a patch acquirer submodule 132, and a patch generator submodule 133. In one embodiment, the automatic program repair module 130 and its submodules 131 to 133 may be implemented by hardware, software, and / or a combination thereof.

[0023] Some examples of computing devices, such as computing device 200, may include a non-transitory tangible machine-readable medium including executable code that, when executed by one or more processors (e.g., processor 110), may cause the one or more processors to perform the processes of the method. Some common forms of machine-readable media that may include the processes of the method are, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media with patterns of holes, RAM, PROMs, EPROMs, flash EPROMs, any other memory chips or cartridges, and / or any other media from which a processor or computer is suitable for reading.

[0024] Figure 2 is suitable for implementing Figure 3200 illustrates a simplified block diagram of a networked system for an automatic program repair framework as described in other embodiments described herein. In one embodiment, block diagram 200 illustrates a system that includes a user device 210 (which may be operated by a user 240), data provider servers 245, 270, and 280, a server 230, and other forms of devices, servers, and / or software components that operate to perform various methods according to the described embodiments. Exemplary devices and servers may include devices that may be similar to Figure 1 The operating OS described in OS, OS, OS) of computing device 100, independent enterprise-level server, or other suitable device and / or server-based OS. It is understood that Figure 2 The devices and / or servers shown in the drawings may be deployed in other ways, and the operations performed and / or services provided by such devices and / or servers may be combined or separated for a given implementation and may be performed by a greater or lesser number of devices and / or servers. One or more devices and / or servers may be operated and / or maintained by the same or different entities.

[0025] User device 210, data provider servers 245, 270, and 280, and server 230 may communicate with one another via network 260. User 240 (e.g., a driver, system administrator, etc.) may utilize user device 210 to access various features available to user device 210, which may include processes and / or applications associated with server 230 to receive output data anomaly reports.

[0026] The user device 210, the data provider server 245, and the server 230 may each include one or more processors, memories, and other suitable components for executing instructions (such as program code and / or data) stored on one or more computer-readable media to implement the various applications, data, and steps described herein. For example, such instructions may be stored in one or more computer-readable media, such as memory or data storage devices that are internal and / or external to various components of the system 200 and / or accessible via the network 260.

[0027] The user device 210 may be implemented as a communication device that may utilize appropriate hardware and software configured for wired and / or wireless communication with the data provider server 245 and / or the server 230. For example, in one embodiment, the user device 210 may be implemented as an autonomous vehicle, a personal computer (PC), a smart phone, a laptop / tablet computer, a watch with appropriate computer hardware resources, glasses with appropriate computer hardware (e.g., GOOGLE ), other types of wearable computing devices, implantable communication devices, and / or other types of computing devices capable of transmitting and / or receiving data (such as from of ). Although only one communication device is shown, multiple communication devices may function similarly.

[0028] Figure 2 The user device 210 includes a user interface (UI) application 212 and / or other applications 216, which may correspond to executable processes, programs, and / or applications with associated hardware. For example, the user device 210 may receive a message indicating vulnerable code and / or fixed code from the server 230 and display the message via the UI application 212. In other embodiments, the user device 210 may include additional or different modules with required specialized hardware and / or software.

[0029] In various embodiments, the user device 210 includes other applications 216 that may be desired in a particular embodiment to provide features to the user device 210. For example, other applications 216 may include security applications for implementing client-side security features, programming client applications for interfacing with appropriate application programming interfaces (APIs) through the network 260, or other types of applications. Other applications 216 may also include communication applications, such as email, text, voice, social networking, and IM applications, which allow users to send and receive emails, calls, texts, and other notifications through the network 260. For example, other applications 216 may be email or instant messaging applications that receive prediction result messages from the server 230. Other applications 216 may include device interfaces and other display modules that can receive input and / or output information. For example, other applications 216 may include software programs for asset management that can be executed by a processor, including a graphical user interface (GUI) configured to provide an interface to the user 240 to view vulnerable code and / or repaired code.

[0030] The user device 210 may also include a database 218 stored in a transient and / or non-transitory memory of the user device 210, which may store various applications and data and be utilized during execution of various modules of the user device 210. The database 218 may store user profiles related to the user 240, predictions previously viewed or saved by the user 240, historical data received from the server 230, etc. In some embodiments, the database 218 may be local to the user device 210. However, in other embodiments, the database 218 may be external to the user device 210 and accessible by the user device 210, including a cloud storage system and / or a database accessible via the network 260.

[0031] The user device 210 includes at least one network interface component 219 suitable for communicating with the data provider server 245 and / or the server 230. In various embodiments, the network interface component 219 may include a DSL (e.g., digital subscriber line) modem, a PSTN (public switched telephone network) modem, an Ethernet device, a broadband device, a satellite device, and / or various other types of wired and / or wireless network communication devices, including microwave, radio frequency, infrared, Bluetooth, and near field communication devices.

[0032] The data supplier server 245 may correspond to a server hosting one or more databases in the databases 203a-n (or collectively referred to as 203) to provide a training data set including pairs of vulnerable codes and repaired codes to the server 230. The database 203 may be implemented by one or more relational databases, distributed databases, cloud databases, etc.

[0033] The data provider server 245 includes at least one network interface component 226 adapted to communicate with the user device 210 and / or the server 230. In various embodiments, the network interface component 226 may include a DSL (e.g., digital subscriber line) modem, a PSTN (public switched telephone network) modem, an Ethernet device, a broadband device, a satellite device, and / or various other types of wired and / or wireless network communication devices, including microwave, radio frequency, infrared, Bluetooth, and near field communication devices. For example, in one implementation, the data provider server 245 may send asset information from the database 203 to the server 230 via the network interface 226.

[0034] The server 230 may accommodate Figure 1 The automatic program repair module 130 and its submodules described in . In some implementations, the module 130 can receive data from the database 219 at the data supplier server 245 via the network 260 to generate a repaired code patch. The generated repaired code patch can also be sent to the user device 210 via the network 260 for the user 240 to review.

[0035] The database 232 may be stored in a transient and / or non-transitory memory of the transmitter 230. In one implementation, the database 232 may store data obtained from the data vendor server 245. In one implementation, the database 232 may store parameters of the automatic program repair module 130. In one implementation, the database 232 may store previously generated repaired code patches and corresponding input feature vectors.

[0036] In some embodiments, database 232 may be local to server 230. However, in other embodiments, database 232 may be external to and accessible by server 230, including cloud storage systems and / or databases accessible via network 260.

[0037] The server 230 includes at least one network interface component 233 adapted to communicate with the user device 210 and / or the data provider servers 245, 270, or 280 via the network 260. In various embodiments, the network interface component 233 may include a DSL (e.g., digital subscriber line) modem, a PSTN (public switched telephone network) modem, an Ethernet device, a broadband device, a satellite device, and / or various other types of wired and / or wireless network communication devices, including microwave, radio frequency (RF), and infrared (IR) communication devices.

[0038] The network 260 may be implemented as a single network or a combination of multiple networks. For example, in various embodiments, the network 260 may include the Internet or one or more intranets, landline networks, wireless networks, and / or other suitable types of networks. Thus, the network 260 may correspond to a small-scale communication network (such as a private network or a local area network) or a larger-scale network (such as a wide area network or the Internet) that can be accessed by various components of the system 200.

[0039] Figure 3 is an exemplary block diagram illustrating an exemplary architecture of an automatic program repair framework 300 (also referred to as RAP-Gen framework 300) using acquisition-enhanced patch generation (RAP-Gen) according to embodiments described herein. RAP-Gen framework 300 is intended to generate a target program patch based on an input vulnerability patch and associated vulnerability repair patterns acquired via acquisition.

[0040] The following describes the task formulation for acquisition-enhanced patch generation for automatic program repair. For |D| bug fixes (X i Y i ) composed of a program repair data set, where X i and Y i are the ith vulnerability patch and the fixed patch, respectively. Codebase C (e.g., codebase 302) contains a large set of previous vulnerability fixes. Among them (B j , F j ) represents the jth vulnerability repair pair. Given a vulnerable program patch X in D i 308, the patch acquirer 304 is based on the relevance scoring function f parameterized by φ φ (X i , Bj ) Get one or more most relevant vulnerability fix pairs in code base C (B j , F j ).

[0041] In some embodiments, the original input sequence X i 308 uses the obtained vulnerability repair pairs to enhance to form a new input sequence 312, for example, in Then, the patch generator 306 (e.g., using a sequence-to-sequence (seq2seq) generator, and also referred to as the sequence generator 306) can be autoregressively derived from 312 Generate Y i 316. Framework 300 can utilize patch generator 306 to learn the probability parameterized by θ where Y i，1 : Y i，k-1 is the previous sequence before the kth token, and n represents the target sequence Y i In some embodiments, the external code base C 302 can be considered as a non-parametric memory, and the obtained vulnerability repair pair 310 can be considered as a guiding repair pattern for the patch generation model 306. In terms of probability, obtaining Z j (B j , F j ) can be formulated as a latent variable, which in some cases can be approximated by the top-1. Formally,

[0042]

[0043] in is from the acquirer P φ (Z j |X i ) to obtain the output. A top-1 approximation can be used to improve efficiency, because marginalization on k>1 makes training and inference complex and inefficient. In some embodiments, a top-k method (e.g., k=2, 3, 5) with Fréchet starting distance (FiD) can be used.

[0044] like Figure 3 As shown in the example of , the RAP-Gen framework 300 includes a patch acquirer 304 and a code-aware pre-trained patch generator 306. The patch acquirer 304 is configured to acquire relevant repair patterns to benefit automatic program repair. It can be built on the relevance scoring function f φ (X i , B j) to calculate (query) the vulnerability patch Xi 308 and the previous (key) vulnerability patch B in the code base C 302 j In various embodiments, the patch acquirer 304 may include a vocabulary-based acquirer (e.g., BM25) and / or a semantics-based acquirer (e.g., dense paragraph acquisition (DPR)). Figure 3 In the example of , the patch acquirer 304 includes a neural network model (e.g., acquirer encoder 318), and a hybrid approach is used to combine a vocabulary-based acquirer (e.g., BM25) and a semantics-based acquirer (e.g., DPR) to consider both lexical and semantic information.

[0045] Vocabulary-based acquirer. In some embodiments, a vocabulary-based acquirer (e.g., BM25) can be implemented using a term-based acquirer and can use sparse vector representations for vocabulary matching. The vocabulary-based acquirer can convert each code patch into a bag-of-words representation and compute the query patch X i With candidate patch B j The similarity score calculated is denoted as f φ (X i , B j )=BM2S(X i , B j ). In one example, a sparse term based getter may be sensitive to choices in the naming of identifiers in the source code that do not affect the semantics of the code.

[0046] Semantic-based acquirer. In some embodiments, the semantic-based acquirer can be implemented using a dense paragraph acquirer (DPR), and related patches can be acquired by measuring the semantic similarity of related patches. In some embodiments, in order to encode the code patch, an encoder (e.g., a transformer-based encoder) can be used to map each patch to a dense vector of a fixed size. The DPR can be initialized from an encoder of a pre-trained transformer-based neural network model (e.g., Code Bidirectional Encoder Representation from Transformer (CodeBERT) etc.). The encoder can be pre-trained using a large code base in one or more programming languages ​​(e.g., GitHub code bases in six programming languages). In one example, the final layer hidden state of the [CLS] token from the encoder is used as a patch representation. In some embodiments, the query patch X can be individually mapped to the query patch using a shared DPR. i Candidate patch B in 308 and C j Encoded as and The similarity is then calculated by the inner product between these two patch representations as follows:

[0047]

[0048] In some embodiments, the query patch X may be individually updated using a shared DPR. i 308 and candidate patch F in C j Encoded as and The similarity is then calculated by the inner product between these two patch representations as follows:

[0049]

[0050] Although this article generally uses X i With B j The similarity between (for example, using f φ (X i , B j )) to obtain, but it should be noted that the similarity used for acquisition can include X i With B j The similarity between (for example, using f φ (X i , B j ))、X i With F j The similarity between (for example, using f φ (X i , F j )) and / or combinations thereof.

[0051] In some embodiments, a training dataset including pairs of vulnerable patches and repaired patches is used to further train a semantics-based acquirer (e.g., DPR). j Treat it as a query and set the corresponding repaired code F j The code base 302 including vulnerability-fix pairs is used as a key. This can be performed based on the assumption that a patch with a vulnerability and its fixed patch usually share similar semantics (e.g., identifiers, data flow, and code structure). This technique can be used to avoid the large amount of manual annotation work required to curate a vulnerability-to-vulnerability search dataset.

[0052] In an example where vulnerability fix pairs are used as queries and corresponding keys, a semantic-based acquirer is trained using a batch negative contrastive learning method 314, where batch negative samples are used to optimize a contrastive loss (e.g., InfoNCE contrastive loss), as shown below:

[0053]

[0054] Where M is the current mini-batch and N represents the number of positive training examples in the mini-batch. The goal is to maximize the similarity between positive examples while minimizing the similarity between negative examples. Each positive example can have |M|-1 negative examples. It should be noted that various contrastive learning techniques can be used, for example, batch negative sample strategies, hard negative example mining strategies, etc., and in some embodiments, batch negative sample contrastive learning as described above provides better performance for noisier training data than hard negative example mining strategies.

[0055] In some embodiments, during the inference phase, given a query with vulnerability patch X i 308, a semantics-based acquirer (e.g., DPR) calculates X i (Query) and B j (keys) to obtain relevant vulnerability fix pairs (B j , F j In some embodiments, the semantics-based acquirer can be based on X i (Query) and F j (key) similarity between and / or with X i (Query) and B j The combination of similarities between (keys) is used to obtain related vulnerability fix pairs.

[0056] Mixed getters. Figure 3 As shown in the example of , in some embodiments, in order to consider vocabulary and semantic information, a hybrid method is used to combine a vocabulary acquirer (e.g., BM25) and a semantic acquirer (e.g., DPR). For example, the similarity score can be calculated as The function is the weight used to balance the two acquirers and can be set to 1 based on experience. Compared with acquirers that rely only on lexical or semantic information, hybrid acquirers are more robust.

[0057] exist Figure 3 In the example of RAP-Gen framework 300, patch generator 306 is used to generate a patch of repaired code. In some embodiments, the patch generator 306 pre-trained by code perception uses the input vulnerability patch 308 (denoted as X i ) (also referred to as source with vulnerability patch 308 or query with vulnerability patch 308) and the obtained vulnerability repair mode 310 (denoted as B j , F j ) to generate enhanced vulnerable code patch 312, for example, using the following cascade: The patch generator 306 is constructed to generate the repaired code patch Y i316 (e.g., using a seq2seq model). The patch generator 306 in the RAP-Gen framework 300 can include any suitable neural network model for sequence generation (also referred to as a sequence generation model or sequence generator). In some embodiments, the patch generator 306 includes a sequence generator optimized on a natural language implementation.

[0058] In some embodiments, the patch generator 306 includes a code-aware programming language model pre-trained on a large-scale source code corpus. In one example, the sequence generator uses CodeT5, which is a unified pre-trained transformer-based encoder-decoder model that can achieve state-of-the-art (SoTA) results in a variety of code intelligence tasks, such as defect detection and code error correction. It can be pre-trained on 8.3 million functions in 8 different programming languages ​​(including JavaScript and Java) collected from GitHub. CodeT5 can adopt identifier-aware pre-training objectives to incorporate code-specific knowledge into the language model. It can provide a code-specific byte pair encoding (BPE) tokenizer optimized for code, and can be able to avoid unknown words (OoV) problems. CodeT5 can be used in the patch generator 306, which can provide powerful code understanding capabilities.

[0059] like Figure 3 As shown in the example of , the acquired enhanced input 312 to the patch generator 306 (eg, CodeT5) can be prepared as [BUG] and [FIX] are special tokens used to separate the acquired vulnerability fix pair from the source vulnerability patch 308. The patch generator 306 (e.g., CodeT5) may include 312 as input to the encoder 318, and synthesizes and generates the repaired patch Y i 316 decoder 320. In some embodiments, during training of the patch generator 306, a teacher forcing algorithm can be used to minimize the language modeling loss. In some embodiments, during inference using the trained patch generator, a beam search (e.g., of size 5) is used to generate a ranked list of candidate repair patches.

[0060] In various embodiments, the RAP-Gen framework 300 utilizes general code understanding knowledge encoded via pre-training on a large-scale code corpus (e.g., using CodeT5). For example, a source input sequence 312 can be generated by concatenating the original vulnerable code patch 308 and the top-ranked vulnerability fix pairs 310 from the patch extractor 304. In some embodiments, an enhanced source input vulnerability patch 312 can be generated by concatenating the top k (e.g., k=2, 3, 5) acquired vulnerability fix pairs to the input vulnerability patch 308.

[0061] Figure 4A is an example of training according to some embodiments described herein. Figure 3 1 is an exemplary logical flow chart of a method for obtaining an enhanced patch generation framework for automatic program repair. One or more of the processes of method 400 may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which executable code, when executed by one or more processors, may cause the one or more processors to perform one or more of the processes. In some embodiments, method 400 corresponds to automatic program repair module 130 (e.g., Figure 1 ) for performing automatic program repair using Get-Enhanced Patch Generation.

[0062] At step 402, a patch acquirer including an acquirer encoder is provided. Figure 3 In the example of , in the acquisition-enhanced patch generation framework 300, a patch acquirer 304 including an acquirer encoder 318 is provided. In some embodiments, as shown at step 404, the acquirer encoder 318 is pre-trained, for example, using a first training dataset including a large programming language corpus (e.g., a GitHub code repository or other suitable code repository in one or more programming languages).

[0063] At step 406, a patch generator including a sequence generator neural network model is provided. Figure 3 In the example of , in the acquisition enhanced patch generation framework 300, the patch generator 306 includes a sequence generator neural network model, specifically a transformer-based encoder-decoder model including a generator encoder 318 and a generator decoder 320. In some embodiments, as shown at step 404, the patch generator 306 is pre-trained using, for example, a second training dataset including a large programming language corpus (e.g., a GitHub code base or other suitable code base in one or more programming languages).

[0064] At step 410, a RAP-Gen framework including a patch acquirer and a patch generator (eg, Figure 3 The two-stage training process includes step 412, at which the first stage training is performed by training the patch acquirer using the third training data set. In some embodiments, the third training data set may use the vulnerability repair pairs in the code base 302. For example, when the semantic acquirer of the patch acquirer 304 is trained using the third training data set, the vulnerable code Bj of the vulnerability repair pair in the code base can be considered as a query, and the corresponding repaired code Fj can be considered as a key. In another example, the repaired code Fj of the vulnerability repair pair in the code base can be considered as a query, and the corresponding vulnerable code Bj can be considered as a key. This is based on the assumption that the vulnerability patch and its repaired patch usually share similar semantics (e.g., identifiers, data flows, and code structures). By using the vulnerability repair pairs for the third training data set in the code base 302, a large amount of manual annotation work required to select the vulnerability to vulnerability search data set as the third training data set can be avoided. In one example, the first stage training can use a contrastive learning algorithm by optimizing the contrastive loss.

[0065] The two-stage training process includes step 414, at which the second stage training is performed by training the patch generator using the patch getter trained by the first stage training using the fourth training data set. In one example, when the input of the patch generator is generated using the original input vulnerable code patch and the top-ranked vulnerability fix pairs from the trained patch getter, a teacher forcing algorithm is used to minimize the language modeling loss.

[0066] During the second stage of training, in the example where the fourth training set is generated from the vulnerability fix pair code base, the patch getter (which has been trained using the first stage training) is not allowed to access the true value vulnerability fix pairs, otherwise the training loss will easily drop to near zero because the patch generator can directly copy the obtained fixes as the target output. In this example, each sample of the fourth training set is a vulnerability patch of the corresponding vulnerability fix pair (also called the true value vulnerability fix pair) from the code base, and the corresponding truth is the repaired patch of the corresponding vulnerability fix pair. For each sample vulnerability patch input, the patch getter obtains another vulnerability fix pair (not the true value vulnerability fix pair) from the code base. The obtained vulnerability fix pair is appended to the vulnerability patch input to generate an enhanced sequence input for the patch generator. It should be noted that when the code base is used to provide the fourth training set, the requirement of not accessing the true value vulnerability fix pairs only applies to the second stage of training, and when the code base is used to provide the third training set, the requirement does not apply to the first stage of training the patch getter.

[0067] How to generate the third and fourth datasets? Recall that we provide vulnerability fix pairs for each downstream dataset, which happens to be the third training set.

[0068] refer to Figure 4B , wherein an exemplary logic flow diagram of a method 450 illustrating an inference process using a trained acquisition-enhanced patch generation framework according to some embodiments described herein is shown. One or more of the processes of method 450 may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which when executed by one or more processors may cause the one or more processors to perform one or more of the processes. In some embodiments, method 450 corresponds to an automatic program repair module 130 (e.g., Figure 1 ) for performing automatic program repair using Get-Enhanced Patch Generation.

[0069] At step 452, a first vulnerability patch is received by the trained acquisition-enhanced patch generation framework. Figure 3 In the example of , the trained acquisition-enhanced patch generation framework 300 receives a first vulnerable patch 308 and provides it as an input to its trained patch acquirer 304 .

[0070] At step 454, one or more vulnerability repair pairs are provided based on the first vulnerability patch. Figure 3 In the example of , the trained patch acquirer 304 receives the first vulnerability patch 308 and acquires one or more vulnerability repair pairs from the code base 302, for example, based on the similarity between the first vulnerability patch 308 and the vulnerability repair pair. In various embodiments, the similarity is determined based on the similarity between the first vulnerability patch 308 and the vulnerability patch with the vulnerability repair pair, the similarity between the first vulnerability patch 308 and the repaired patch of the vulnerability repair pair, or a combination thereof. The similarity may include lexical similarity, semantic similarity, or a combination thereof.

[0071] At step 456, a first enhanced vulnerability patch is generated based on the first vulnerability patch and the obtained one or more vulnerability repair pairs. Figure 3 In the example of , the first enhanced vulnerability patch 312 is generated using the first vulnerability patch 308 and the one or more vulnerability repair pairs 310 provided by the patch acquirer 304 . The first enhanced vulnerability patch 312 is provided to the patch generator 306 .

[0072] At step 458, the first enhanced vulnerability patch is used to generate a first repaired patch for the first vulnerability patch. Figure 3In the example of , the patch generator 306 receives the first enhanced vulnerability patch 312 and generates a first repaired patch 316 based on the first enhanced vulnerability patch 312 .

[0073] Example Data Experiments and Performance

[0074] refer to Figure 5 In some experiments, the RAP-Gen framework is evaluated on two popular APR datasets, namely: TFix in JavaScript (Berkay Berabi, Jingxuan He, Veselin Raychev, and Martin T. Vechev, "TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer", Proceedings of Machine Learning Research (PMLR), Vol. 139, pp. 780-791) and Code Refinement in Java (Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk, "An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation", ACM Trans. Softw. Eng. Methodol., Vol. 28, No. 4 (2019), pp. 19:1-19:29). Both datasets were originally collected from GitHub commits, but the main difference is that the bug fix pairs in TFix can be verified by static analyzers, while the pairs in Code Refinement are verified by checking whether the commit message contains keywords like "fix vulnerability". The statistics of the TFix and Code Refinement benchmarks are shown in Figure 5 In Table 1.

[0075] TFix. Specifically, TFix is ​​a large-scale program repair dataset that includes JavaScript code-patch pairs selected from 5.5 million GitHub commits. It comprehensively covers 52 unique error types detected by the static analyzer ESLint. In addition to the error types, it also provides rich error annotations such as error messages and located error lines, making it unnecessary to perform fault localization as in existing work. In TFix, they use T5-Large to treat the APR task as a text-to-text generation problem. In the source input sequence, they combine all error information together with the vulnerable code patch into a single piece of text:

[0076] Fix {error type} {error message} {error context}

[0077] The error context consists of a given located error line, and its two adjacent code lines are used to form a vulnerable code patch. The target sequence will replace the error line with the repaired line in the error context. The same data format is used in the experiment, and can be used in Figure 6 (showing a vulnerability repair example on the TFix test set where the RAP-Gen framework correctly repairs the vulnerability).

[0078] During data processing, duplication issues within and between data splits were observed. Specifically, there were 114, 2, and 4 duplications in the training, validation, and test splits, respectively. For inter-split duplications, there were 28, 34, and 4 duplications between the training and test, training and test, and validation and test splits, respectively. Those duplications (243) were filtered out, and Figure 5 The deduplicated version TFix (Dedup) is shown in Table 1 of .

[0079] Code Refinement. Tufano et al. released two code correction datasets containing function-level bug fix pairs collected from the public GitHub Archive (https: / / www.gharchive.org / ) between March 2011 and October 2017. They used the Google BigQuery API to identify all Java commits with messages containing the following patterns: ("fix" or "solved") and ("bug" or "problem" or "problem" or "error") to ensure the quality of the collected bug fix function pairs. They normalized the functions by fuzzying identifiers with indexed tokens (such as TYPE1, VAR1, METHOD1, etc.). Figure 7(showing a bug fix example on the Refinement Small test set where the RAP-Gen framework gave a correct prediction). The two data subsets are determined by the number of tokens, i.e., # of code tokens <= 50 for the Small set and 50 < # of code tokens <= 100 for the Medium set.

[0080] In some embodiments, the RAP-Gen framework 300 can be fine-tuned (e.g., for up to 30 epochs) using a sequence-to-sequence generation loss for each benchmark, for example, using the AdamW optimizer (Decoupled Weight Decay Regularization by Ilia Loshchilov and Frank Hutter, published at ICLR in 2019). A grid search can be performed to achieve hyperparameter tuning using various batch sizes (e.g., 16, 32, 64) and learning rates (e.g., 1e-4, 5e-5, 2e-5). For example, for TFix, a batch size of 64 and a learning rate of 1e-4 can be used, and for Code Refinement, a batch size of 32 and a learning rate of 5e-5 can be used. In one example, the training time for the RAP-Gen-base on each benchmark using an A100 GPU was within 2 days. During inference, a beam search with a beam size of five can be used to generate a ranked list of synthesized repaired patches.

[0081] In some embodiments, the vulnerability fix pairs in the training set are adopted as the search code base for building the patch obtainer 304. For the vocabulary-based obtainer, an exemplary open source Python library (e.g., BM25’s https: / / pypi.org / project / rank-bm25 ). As a sparse term based extractor, the choice of tokenizer will greatly affect the extraction performance. In the experiments, CodeT5 tokenizer is used, which is a code-specific BPE tokenizer optimized for code tokenization. BM25 search engine is applied on a machine with 95 CPUs and 600G memory for the benchmarks TFix and Code Refinement. Each experiment is completed in one hour with multiprocessing.

[0082] In the experiments, for the semantic-based acquirer, CodeBERT initialized with DPR is used to encode each patch into a dense vector for semantic matching. Separately, the DPR model is fine-tuned on each benchmark for 50 epochs using InfoNCE contrastive loss. Fine-tuning is performed on an A100 GPU with 40G memory using a batch size of 64 and a learning rate of 2e-5. The training time for TFix and Code Refinement is approximately 9 GPU hours and 5 GPU hours, respectively.

[0083] For the hybrid acquirer, the ranking scores of BM25 and DPR are calculated, and these normalized scores are linearly combined with equal weights to construct a hybrid acquirer, namely "Hybrid". For all acquirers, the patches are encoded using the CodeT5 tokenizer with a maximum sequence length of 256.

[0084] Evaluation Metrics. For evaluation metrics, smoothed BLEU-4 (Chin-Yew Lin and Franz Josef Och, ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation, COLING, 2004) scores and exact match (EM) accuracy are used to evaluate program repair performance (e.g., see CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation, Yue Wang, Weishi Wang, Shafiq R. Joty, and Steven CHHoi, EMNLP, Association for Computational Linguistics, pp. 8696–8708). BLEU-4 is a looser metric that evaluates the degree of subword overlap, while EM is a stricter metric that requires the prediction to be the same as the true patch in the real submission. Since vulnerable programs may have different repair methods, the Error Removal metric (e.g., as used in TFix) is used to account for various forms of repair. A prediction is counted as correct for Error Removal if the existing error is removed and no new errors are introduced after the repair. For all metrics, the results are presented on a scale of 0-100 (%), and higher scores indicate better performance.

[0085] Baseline Models. The RAP-Gen framework is compared with learning-based models in two program repair benchmarks. CoCoNuT is a context-aware neural machine translation framework based on a convolutional encoder-decoder model. SequenceR is an LSTM-based sequence-to-sequence generation model with a replication mechanism. In addition, the RAP-Gen framework is compared with pre-trained programming language models based on a transformer architecture. One set of these models are encoder-only models, such as RoBERTa(code), CodeBERT, and GraphCodeBERT. These encoder-only models require a randomly initialized decoder for program repair tasks.

[0086] In addition, the RAP-Gen framework is compared with an encoder-decoder transformer model. PLBART is a unified pre-trained model whose denoising objectives include token masking, token removal, and token padding. TFix is ​​initialized with the T5-large checkpoint and continues to be fine-tuned on the TFix dataset. CoTexT is another T5-based model that is pre-trained on text and code. NSEdit is a language model with an encoder and decoder initialized from CodeBERT and CodeGPT, respectively. It is fine-tuned to generate repairs via neural symbolic editing sequences and is ranked as the current SoTA model on the Code Refinement benchmark. The results of all baseline models are obtained from their original papers.

[0087] Experiments confirm that acquisition-enhanced patch generation is an effective approach for program repair. Comprehensive experiments are conducted to compare RAP-Gen with existing learning-based methods on two benchmarks. First, the CodeT5 model is evaluated on TFix, and its evaluation is improved by providing a deduplicated version of the dataset and more reasonable metrics, as well as by additionally introducing a looser metric of exact match alignment with the BLEU-4 score. The results show that CodeT5-base establishes new SoTA performance on this task, improving T5-large from 49.70 to 53.57 in EM and from 76.98 to 78.85 in BLEU-4. In addition, the RAP-Gen model is evaluated using both TFix and Code Refinement datasets. It is observed that RAP-Gen significantly improves the performance together with vocabulary- and semantic-based acquirers. Specifically, RAP-Gen-base with "hybrid" improves exact matching over the best performing baseline in TFix (49.70→54.15), while RAP-Gen-base with "hybrid" improves exact matching in the small set of Code Refinement benchmark (24.04→24.80) and in the medium set (14.18→15.84). All these results confirm that acquisition-enhanced patch generation (RAP-Gen) is an effective method for APR.

[0088] Experiments show that acquisition-enhanced patch generation with CodeT5 is an effective approach for program repair. First, CodeT5 is compared with traditional APR techniques on the TFix benchmark, which is improved by using a deduplicated version of the data and more appropriate evaluation metrics. Then, the RAP-Gen framework integrated with two sizes of CodeT5 is evaluated on the TFix and Code Refinement benchmarks. Moreover, the experiments show that the patch acquirer finds relevant patches in terms of lexical and semantic similarity. In addition, a case study is provided to illustrate how the acquired vulnerability repair patterns can help program repair. In addition, as shown in the experiments, the RAP-Gen framework provides improved performance for a variety of bug types and repair patterns. A detailed performance breakdown of 52 bug types is listed, and the bug types that do not benefit from the acquisition enhancement of RAP-Gen are examined. In addition, the way the model performs with a shallow but dominant bug-line removal repair pattern of simply removing the buggy lines from the vulnerable code is studied.

[0089] Experiments show that the acquisition-enhanced patch generation using CodeT5 is an effective approach for program repair. First, it provides an improved TFix evaluation. The original TFix benchmark uses the direct average of the exact match (EM) accuracy across 52 error types as the main evaluation metric. However, as Fig.14As shown in Table 7 of , these error types have a rather unbalanced distribution, for example, the main error type "no-invalid-this" has 16,217 instances, while the smallest error type "no-new-symbol" has only 10 instances. Therefore, in some embodiments, a weighted average is used to consider the error type distribution. In addition, after examining how TFix calculates the published code for exact matches, another limitation is that if the predicted repair contains one more blank (such as a space or a line break) than the true value repair, it will be regarded as an exact match of the error. However, in the JavaScript language, the extra blank does not affect the correctness of the program. Therefore, a better metric of EM weighted average without spaces is proposed, which normalizes the blanks before calculating EM to exclude the impact of mismatches caused by multiple blanks. Due to the duplication problem in the TFix dataset, the results on its deduplication version are also included. In addition to the exact match accuracy, a looser metric of the BLEU-4 score is used to measure the subsequence overlap between the predicted repair and its true value repair. It should be noted that the BLEU-4 score is also calculated after blank normalization.

[0090] like Fig. 9 As shown in Table 2, the CodeT5 model is compared with other learning-based baselines on TFix. A key observation is that for the original average EM with spaces metric, CodeT5-base (50.88) also produces better accuracy than T5-large (49.33) (considering its much larger model size (~3.5x of CodeT5-base). Moreover, focusing on the more reasonable direct average EM without spaces, CodeT5-base significantly improves the performance, with an absolute accuracy improvement of about 5% compared to T5-large (49.35→54.30). Based on weighted average EM without spaces, both CodeT5-small (50.31) and CodeT5-base (53.57) outperform all baselines including T5-large (49.70). This indicates that the code-aware pre-trained CodeT5 model on a large-scale code corpus has a better understanding of the program. For TFix evaluation, EM is used to represent weighted average EM without spaces, unless otherwise stated. For the BLEU-4 metric, which has good alignment with the exact match metric, CodeT5-base also gives a state-of-the-art (SoTA) performance of 78.85 on the original TFix.

[0091] Next, we describe the ablation study observations. On the deduplicated TFix dataset, performance across various metrics is consistently slightly degraded. This is an expected phenomenon because the duplication between training and test splits in the original data (34 instances) will cause data leakage issues and inappropriately improve performance. If the error information including error type and error message is removed, both CodeT5-Small and CodeT5-Base models experience consistent performance degradation, which reveals that it is helpful for program repair models to be informed of which types of errors they need to fix.

[0092] refer to Fig.10 Table 3 of , describes the RAP-Gen evaluation on TFix. Table 3 shows the results of the RAP-Gen framework on the deduplication version of the TFix benchmark. First, a random baseline is established by randomly acquiring vulnerability repair pairs from the code base. The performance degradation of small and basic RAP-Gen in random acquisition suggests that the randomly acquired repair patterns cannot provide useful guidance signals for program repair. Then, RAP-Gen integrated with different acquirers is compared, including using a vocabulary-based acquirer BM25, a semantic-based acquirer DPR based on dense vector matching, and two ensemble methods combining them. The results show that all acquisition enhancement methods significantly improve the performance of both small models and base models on both exact matching and BLEU-4. This indicates that acquisition enhancement generation is a feasible and effective method for APR, and both semantic information and vocabulary information are crucial for acquiring relevant repair patterns. For the ensemble method, the RAP-Gen-base with "hybrid" produces the best improvement over T5-large (49.58→54.15EM). It verifies that ensemble methods that consider both lexical and semantic information can combine the best of both worlds. Another observation is that the performance gain of leveraging acquisition-augmentation on RAP-Gen-Small is larger than that on RAP-Gen-Base, which means that the improvement tends to reach a saturation point as the model size increases. Both RAP-Gen-Small and RAP-Gen-Base use the RAP-Gen framework with different patch generator backbones, specifically, CodeT5-Base and CodeT5-Small with different model sizes, respectively.

[0093] In some implementations, there may be multiple ways to fix a vulnerability. Therefore, an exact match to a true patch would be an overly strict metric, failing to consider other forms of correct fixes. To deal with this issue, a looser evaluation is used, where the error removal metric follows TFix. Under this metric, a fixed patch is considered correct as long as it resolves the errors in the source vulnerability patch and does not introduce new errors (detected by the static analyzer ESLint). When trying to reproduce this metric on 10,465 test instances, there are two difficulties: (1) Applying ESLint requires the full file context of each code patch, but we found that 95 code files are no longer available for acquisition. (2) When applying ESLint with a published configuration (https: / / github.com / eth-sri / TFix) to some data samples, there are parser errors. As a result, by excluding those unavailable code files and samples with parser errors, a filtered subset of 6,793 instances was selected, where it was also found that fixes generated from TFix tend to have more parser errors. Reference Fig.11 Table 4 shows the error removal comparison. It is observed that the RAP-Gen-Small model outperforms the T5-Large model in terms of error removal by a good margin, which means that the RAP-Gen model is more capable of synthesizing good repairs of different forms. In addition, it is observed that there is a misalignment between the error removal metric and the exact match metric, where RAP-Gen-Small achieves lower exact match but higher error removal accuracy. Such misalignment is also observed in TFix.

[0094] refer to Fig.12 Table 5 shows the Code Refinement results and comparison with previously discussed methods. All baseline results (including the CodeT5 model) are taken directly from their original papers. It is observed that “Naive Copy” gives a very high BLEU-4 score, but the exact match (EM) is zero, which indicates that the vulnerable code and its fix have a large overlap and exact match should be used as the primary evaluation metric. Among the baselines, NSEdit is very competitive, with the best result (24.04EM) on a small subset, while the multi-task trained CodeT5-base achieves the best result (14.18EM) on a medium set.

[0095] From the RAP-Gen model comparison, it is observed that RAP-Gen with various acquirers consistently improves the performance compared to its CodeT5 counterpart. The best model establishes new SoTA results on both subsets (24.80EM for small and 15.84EM for medium), especially surpassing NSEdit by about 2 absolute points on the more challenging medium set. This reaffirms that the acquired repair patterns provide useful signals to guide program repair. Among the various acquirers, DPR gives better results than BM25 for both RAP-Gen-Small and RAP-Gen-Base, revealing that semantic information may play a more important role than semantic information for this benchmark. In addition, "Hybrid" outperforms BM25 and DPR, which means that the hybrid set method is a more robust acquirer that balances semantic information and semantic information for this benchmark.

[0096] In summary, comprehensive experiments are performed to compare RAP-Gen with existing learning-based methods on two benchmarks. First, the CodeT5 model is evaluated on TFix, and its evaluation is improved by providing a deduplicated version of the dataset and more reasonable metrics, as well as additionally introducing a looser metric of exact match alignment with the BLEU-4 score. The results show that CodeT5-base establishes new SoTA performance on the task, improving T5-large from 49.70 to 53.57 in EM and from 76.98 to 78.85 in BLEU-4. We then evaluate the RAP-Gen model on TFix and Code Refinement datasets, and observe that RAP-Gen with lexical and semantic-based acquirers significantly improves the performance. Specifically, RAP-Gen-base with "hybrid" improves exact matching over the best performing baseline in TFix (49.70→54.15), while RAP-Gen-base with "hybrid" improves exact matching in the small set of Code Refinement benchmark (24.04→24.80) and in the medium set (14.18→15.84). All these results confirm that acquisition-augmented patch generation (RAP-Gen) with CodeT5 is an effective approach for APR.

[0097] Next, experiments are performed to evaluate whether the patch extractor can find relevant repair patterns that are beneficial for program repair. First, an automatic evaluation is provided to measure the relevance between queries and the extracted patches in terms of lexical and semantic similarity. In addition, specific cases are provided to understand how the extracted repair patterns can help achieve better APR.

[0098] refer to Fig.13Table 6 shows the evaluation of the extractors. The extractors are analyzed in terms of lexical and semantic matching between the query and the previously extracted patches. For lexical matching, the BLEU-4 score is used to measure their sub-token overlap, while for semantic matching, the cosine similarity (CosSim) between their dense vectors encoded by the fine-tuned DPR extractor is used. Fig.13 Table 6 shows the performance of the patch grabber on both TFix and Code Refinement benchmarks. The first row indicates the lower bound performance via randomly acquiring vulnerability fix pairs from the code base, where it is observed that the random baseline achieves much lower scores in both lexical and semantic matching. For lexical matching, BM25 outperforms DPR on TFix, but performs poorly on the two CodeRefinement subsets, which may be caused by the data differences between TFix and Code Refinement, where the latter adopts ambiguous identifiers (e.g., VAR1, VAR2, ...) that hinder the performance of the lexical-based BM25 grabber. The hybrid grabber achieves the best lexical matching for all datasets, which reveals that semantic information can supplement lexical matching.

[0099] For semantic matching, DPR achieves the best results on all datasets, which is not surprising since it is optimized for the same goal. It is worth noting that the hybrid acquirer achieves slightly lower results than DPR, but much better results than BM25, which means that it balances lexical and semantic information and is more robust than lexical-based acquirers (which are sensitive to the choice of identifier naming).

[0100] Return to reference Figure 6 and Figure 7 , using case studies to illustrate how the acquired repair patterns can help program repair, including a case study of TFix ( Figure 6 ) and a case study on Code Refinement ( Figure 7 ), where the RAP-Gen model with acquisition-enhancement predicts the correct repair, while CodeT5 without acquisition-enhancement fails to do so. Figure 6 As shown in , the acquired vulnerability repair pattern is exactly the pattern needed to repair the source vulnerable code. Without acquisition-enhancement, CodeT5 may mistakenly remove ".classify()" from the vulnerable line by learning from the previous adjacent line. Figure 7 In the Code Refinement case, the acquired vulnerability-fix pairs provide enough information to guide the RAP-Gen model to repair the source vulnerable code. Without acquisition-enhancement, CodeT5 performs incorrect repair by simply removing the last line of code.

[0101] Therefore, the quantitative ( Fig.13 Table 6) and qualitative ( Figure 6 and Figure 7 ) results are used to evaluate the performance of the patch grabber and the corresponding automatic program repair system. The results show that the hybrid patch grabber is more robust and can find lexically and semantically related patches to assist the program repair system.

[0102] refer to Figure 8 and Fig.15 Table 7 describes the performance of RAP-Gen for various error types and repair modes. First, regarding the performance breakdown of different error types, Fig.15 A detailed program repair performance breakdown on the deduplicated TFix dataset is listed in Table 7 of . CodeT5-Base outperforms the previous SoTA T5-Large on 44 / 52 bug types. In particular, for the major bug type “no-invalid-this”, CodeT5-Base improves its exact match from 37.48 of T5-Large to 43.57, which corresponds to repairing 98 more instances. While T5-Large can repair at least 50% of the bugs for 44% of the 52 bug types, CodeT5-Base significantly increases this percentage to 60%, and RAP-Gen-Small further improves it to 63%. Overall, RAP-Gen-Base correctly repairs 478 more vulnerable programs than T5-Large with a much smaller model size.

[0103] In addition, the impact of acquisition-enhancement in RAP-Gen is compared with the CodeT5 model for various error types. Fig.14 As shown in Table 7, it is observed that the acquisition-enhancement technique has different effects on various error types, and sometimes even hurts the program repair performance for a specific subset of error types. Specifically, in the experiment, for CodeT5-small, it degrades APR performance for 10 error types, and for CodeT5-base, it degrades APR performance for 18 error types. Based on the number of exact match repairs, it is observed that the largest performance degradation for RAP-Gen-small comes from the error type "no-extra-semi" (497→490), and the largest performance degradation for RAP-Gen-base comes from the error type "no-console" (228→220).

[0104] To explore why acquisition-augmentation sometimes hinders exact matching performance in the RAP-Gen model, Figure 8A case study for the “no-console” bug type is provided in . In this case, the ground truth fix is ​​to directly remove the buggy line in the vulnerable patch, while the RAP-Gen model fixes it differently based on the acquired fix pattern, which is counted as a false prediction in terms of exact match. This reaffirms the limitation of the exact match metric for evaluating program repair systems.

[0105] Next, we used the TFix benchmark to analyze which repair patterns the model performed. After manually inspecting the vulnerability repair pairs, we observed that the majority of repairs consisted of deletion operations compared to code insertion and replacement operations. The vulnerability repair operations consisted of the following: code insertion (12.5%), replacement (8.1%), deletion (47.9%), insertion and replacement (6.9%), insertion and deletion (8.2%), replacement and deletion (7.2%), and all three (9.2%). Earlier studies also reflected that deletion operations were one of the most common repair patterns. Among deletion operations, a major vulnerability repair pattern is bad line removal, which is simply removing the bad line (such as _ ... Figure 8 ). This shallow repair pattern accounts for about 23% of the deduplicated TFix test set. To further analyze this pattern, we compared how different models perform using the bad row removal pattern and present the results in Table 8. We observe that RAP-Gen-base achieves the lowest false positive count of 56 (corresponding to the highest precision of 97.09) with fetch-augmentation, compared to 67 for CodeT5-base and 71 for T5-large. This suggests that RAP-Gen is able to learn more diverse vulnerability repair patterns instead of over-relying on the shallow bad row removal pattern. In addition, RAP-Gen-small achieves the best recall score and F1 score, but at the cost of producing more false positive predictions.

[0106] In summary, the difficulty of program repair varies depending on the type of bug. The best RAP-Gen-Base in the experiments can repair 456 more vulnerable programs than the best performing baseline T5-Large. Error analysis is performed to analyze why fetch-augmentation sometimes degrades performance, and a case study is provided to illustrate that this may be due to the limitation of the exact match metric. In addition, a high-frequency repair pattern of buggy line removal is studied to show that RAP-Gen-Base gives the best precision score and RAP-Gen-Small achieves the best recall score and F1 score when handling this pattern.

[0107] This specification and the drawings showing aspects of the invention, embodiments, implementations or applications should not be considered restrictive. Various mechanical, compositional, structural, electrical and operational changes may be made without departing from the spirit and scope of this specification and claims. In some cases, in order not to obscure the embodiments of the disclosure, known circuits, structures or technologies are not shown or described in detail. Similar reference numerals in two or more figures represent identical or similar elements.

[0108] In this specification, the specific details of some embodiments consistent with the disclosure are described. Many specific details are described in order to provide a thorough understanding of the embodiments. However, it is obvious to those skilled in the art that some embodiments can be put into practice without some or all of these specific details. The specific embodiments disclosed herein are intended to be illustrative rather than restrictive. Those skilled in the art will appreciate that other elements fall within the scope and spirit of the disclosure, although the other elements are not specifically described herein. In addition, in order to avoid unnecessary repetition, unless specifically described otherwise or when one or more features will make the embodiment inoperative, the one or more features shown and described in association with an embodiment may be incorporated into other embodiments.

[0109] Although exemplary embodiments have been shown and described, various modifications, changes and substitutions are contemplated in the foregoing disclosure, and in some cases, some features of the embodiments may be employed without corresponding use of other features. Those of ordinary skill in the art will recognize many variations, substitutions and modifications. Therefore, the scope of the present invention should be limited only by the appended claims, and the claims should be interpreted broadly in a manner consistent with the scope of the embodiments disclosed herein.

Claims

1. A method for automatic program repair, the method comprising: Receive the first batch of vulnerability code patches; Using an acquirer encoder of a patch acquirer, generating a first representation of the first vulnerable code patch; Using the patch acquirer, based on the first representation, acquire a first vulnerability fix code pair from a first plurality of vulnerability fix code pairs; generating a first enhanced vulnerability code patch based on the first vulnerability code patch and the first vulnerability repair code pair; and A patch generator is used to generate a repaired code patch based on the first enhanced vulnerability code patch. 2 . The method according to claim 1 , wherein the patch acquirer is configured to: acquire based on at least one of lexical similarity and semantic similarity with the first vulnerable code patch. 3 . The method according to claim 2 , wherein the patch acquirer is configured to: perform acquisition based on a combination of lexical similarity and semantic similarity with the first vulnerable code patch.

4. The method of claim 1, wherein the patch generator comprises a transformer-based neural network model for sequence generation. 5 . The method according to claim 1 , wherein the first vulnerability repair pair of the first enhancement with a vulnerability is used as a guiding repair pattern of the patch generator.

6. The method according to claim 1, further comprising: Before receiving the first vulnerable code patch, a two-stage training process is performed, the two-stage training process comprising: Training the patch acquirer using a first training set in a first stage; and In the second stage, the patch generator is trained using the trained patch acquirer and the second training set. The method according to claim 6 , wherein the first training set includes vulnerable patches and corresponding repaired patches.

8. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions, the plurality of machine-readable instructions being adapted, when executed by one or more processors, to cause the one or more processors to perform a method comprising: Receive the first batch of vulnerability code patches; Using an acquirer encoder of a patch acquirer, generating a first representation of the first vulnerable code patch; Using the patch acquirer, based on the first representation, acquire a first vulnerability fix code pair from a first plurality of vulnerability fix code pairs; generating a first enhanced vulnerability code patch based on the first vulnerability code patch and the first vulnerability repair code pair; and A patch generator is used to generate a repaired code patch based on the first enhanced vulnerability code patch. 9 . The non-transitory machine-readable medium of claim 8 , wherein the patch obtainer is configured to obtain based on at least one of lexical similarity and semantic similarity with the first vulnerable code patch. 10 . The non-transitory machine-readable medium of claim 9 , wherein the patch retriever is configured to perform the retrieval based on a combination of the lexical similarity and the semantic similarity with the first vulnerable code patch.

11. The non-transitory machine-readable medium of claim 8, wherein the patch generator comprises a transformer-based neural network model for sequence generation. 12 . The non-transitory machine-readable medium of claim 8 , wherein the first vulnerability-repair pair of the first enhancement with a vulnerability is used as a guiding repair pattern for the patch generator.

13. The non-transitory machine-readable medium of claim 8, further comprising: Before receiving the first vulnerable code patch, a two-stage training process is performed, the two-stage training process comprising: Training the patch acquirer using a first training set in a first stage; and In the second stage, the patch generator is trained using the trained patch acquirer and the second training set. 14 . The non-transitory machine-readable medium of claim 13 , wherein the first training set includes vulnerable patches and corresponding fixed patches.

15. A system comprising: non-transitory memory; and one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform a method comprising: Receive the first batch of vulnerability code patches; Using an acquirer encoder of a patch acquirer, generating a first representation of the first vulnerable code patch; Using the patch acquirer, acquiring a first vulnerability repair code pair from a first plurality of vulnerability repair code pairs based on the first representation; generating a first enhanced vulnerability code patch based on the first vulnerability code patch and the first vulnerability repair code pair; and A patch generator is used to generate a repaired code patch based on the first enhanced vulnerability code patch. 16 . The system according to claim 15 , wherein the patch acquirer is configured to: acquire based on at least one of lexical similarity and semantic similarity with the first vulnerable code patch. 17 . The system of claim 16 , wherein the patch retriever is configured to perform the retrieval based on a combination of the lexical similarity and the semantic similarity with the first vulnerable code patch.

18. The system of claim 15, wherein the patch generator comprises a transformer-based neural network model for sequence generation.

19. The system of claim 15, wherein the first vulnerability repair pair of the first enhancement band vulnerability is used as a guiding repair pattern for the patch generator.

20. The system of claim 15, wherein the method further comprises: Before receiving the first vulnerable code patch, a two-stage training process is performed, the two-stage training process comprising: Training the patch acquirer using a first training set in a first stage; and In the second stage, the patch generator is trained using the trained patch acquirer and the second training set.

Images